[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:01:02.185313  7526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.89.190:44281
I20260709 13:01:02.187517  7526 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:01:02.188886  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:02.212347  7535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:02.212374  7532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:02.214459  7526 server_base.cc:1061] running on GCE node
W20260709 13:01:02.212383  7533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:03.489596  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:03.490340  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:03.490757  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602063490685 us; error 0 us; skew 500 ppm
I20260709 13:01:03.492681  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:03.500257  7526 webserver.cc:533] Webserver started at http://127.7.89.190:42021/ using document root <none> and password file <none>
I20260709 13:01:03.501535  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:03.501852  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:03.502527  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:03.511257  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/master-0-root/instance:
uuid: "e46f255b825443eeba2f9ff5dd6cd4e9"
format_stamp: "Formatted at 2026-07-09 13:01:03 on dist-test-slave-h0rz"
I20260709 13:01:03.526717  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.014s	user 0.013s	sys 0.000s
I20260709 13:01:03.534999  7541 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:03.537308  7526 fs_manager.cc:730] Time spent opening block manager: real 0.007s	user 0.001s	sys 0.003s
I20260709 13:01:03.537871  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/master-0-root
uuid: "e46f255b825443eeba2f9ff5dd6cd4e9"
format_stamp: "Formatted at 2026-07-09 13:01:03 on dist-test-slave-h0rz"
I20260709 13:01:03.538276  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:03.613031  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:03.615304  7526 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:01:03.615943  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:03.712661  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.190:44281
I20260709 13:01:03.712759  7592 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.190:44281 every 8 connection(s)
I20260709 13:01:03.723518  7593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:03.753723  7593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9: Bootstrap starting.
I20260709 13:01:03.761533  7593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:03.764041  7593 log.cc:826] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9: Log is configured to *not* fsync() on all Append() calls
I20260709 13:01:03.770563  7593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9: No bootstrap required, opened a new log
I20260709 13:01:03.796933  7593 raft_consensus.cc:359] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46f255b825443eeba2f9ff5dd6cd4e9" member_type: VOTER }
I20260709 13:01:03.797716  7593 raft_consensus.cc:385] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:03.797973  7593 raft_consensus.cc:740] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e46f255b825443eeba2f9ff5dd6cd4e9, State: Initialized, Role: FOLLOWER
I20260709 13:01:03.799284  7593 consensus_queue.cc:260] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46f255b825443eeba2f9ff5dd6cd4e9" member_type: VOTER }
I20260709 13:01:03.799916  7593 raft_consensus.cc:399] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:01:03.800370  7593 raft_consensus.cc:493] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:01:03.800958  7593 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:03.807363  7593 raft_consensus.cc:515] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46f255b825443eeba2f9ff5dd6cd4e9" member_type: VOTER }
I20260709 13:01:03.808346  7593 leader_election.cc:304] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e46f255b825443eeba2f9ff5dd6cd4e9; no voters: 
I20260709 13:01:03.810863  7593 leader_election.cc:290] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:01:03.811175  7596 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:03.814831  7596 raft_consensus.cc:697] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [term 1 LEADER]: Becoming Leader. State: Replica: e46f255b825443eeba2f9ff5dd6cd4e9, State: Running, Role: LEADER
I20260709 13:01:03.816069  7596 consensus_queue.cc:237] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46f255b825443eeba2f9ff5dd6cd4e9" member_type: VOTER }
I20260709 13:01:03.817517  7593 sys_catalog.cc:565] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:01:03.829838  7598 sys_catalog.cc:455] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e46f255b825443eeba2f9ff5dd6cd4e9. Latest consensus state: current_term: 1 leader_uuid: "e46f255b825443eeba2f9ff5dd6cd4e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46f255b825443eeba2f9ff5dd6cd4e9" member_type: VOTER } }
I20260709 13:01:03.829952  7597 sys_catalog.cc:455] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e46f255b825443eeba2f9ff5dd6cd4e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46f255b825443eeba2f9ff5dd6cd4e9" member_type: VOTER } }
I20260709 13:01:03.830885  7598 sys_catalog.cc:458] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:03.831053  7597 sys_catalog.cc:458] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:03.835096  7605 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:01:03.849164  7605 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:01:03.852859  7526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:01:03.871806  7605 catalog_manager.cc:1383] Generated new cluster ID: 7eb1546fa7b747a4bfc59403c3d45696
I20260709 13:01:03.872201  7605 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:01:03.921279  7605 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:01:03.923352  7605 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:01:03.942477  7605 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9: Generated new TSK 0
I20260709 13:01:03.943996  7605 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:01:03.984753  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:03.997114  7616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:03.997910  7615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:04.001658  7526 server_base.cc:1061] running on GCE node
W20260709 13:01:04.001590  7618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:04.002915  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:04.003129  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:04.003376  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602064003334 us; error 0 us; skew 500 ppm
I20260709 13:01:04.004076  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:04.009328  7526 webserver.cc:533] Webserver started at http://127.7.89.129:37153/ using document root <none> and password file <none>
I20260709 13:01:04.009987  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:04.010222  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:04.010556  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:04.011950  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-0-root/instance:
uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
format_stamp: "Formatted at 2026-07-09 13:01:04 on dist-test-slave-h0rz"
I20260709 13:01:04.019475  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:01:04.025172  7623 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:04.026247  7526 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:01:04.026687  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-0-root
uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
format_stamp: "Formatted at 2026-07-09 13:01:04 on dist-test-slave-h0rz"
I20260709 13:01:04.027055  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:04.051143  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:04.053241  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:04.057516  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:04.064679  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:04.065006  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:04.065420  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:04.065590  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:04.191174  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.129:32899
I20260709 13:01:04.191428  7685 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.129:32899 every 8 connection(s)
I20260709 13:01:04.202394  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:04.216688  7690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:04.223965  7691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:04.232724  7693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:04.235354  7526 server_base.cc:1061] running on GCE node
I20260709 13:01:04.237334  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:04.237658  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:04.237896  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602064237873 us; error 0 us; skew 500 ppm
I20260709 13:01:04.238819  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:04.242604  7686 heartbeater.cc:344] Connected to a master server at 127.7.89.190:44281
I20260709 13:01:04.242885  7526 webserver.cc:533] Webserver started at http://127.7.89.130:34275/ using document root <none> and password file <none>
I20260709 13:01:04.243268  7686 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:04.243676  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:04.243971  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:04.244367  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:04.244715  7686 heartbeater.cc:507] Master 127.7.89.190:44281 requested a full tablet report, sending...
I20260709 13:01:04.246093  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-1-root/instance:
uuid: "07fc46d8587b48f7b5015b4d639028a2"
format_stamp: "Formatted at 2026-07-09 13:01:04 on dist-test-slave-h0rz"
I20260709 13:01:04.249295  7558 ts_manager.cc:194] Registered new tserver with Master: 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899)
I20260709 13:01:04.253414  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.004s	sys 0.002s
I20260709 13:01:04.253270  7558 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52426
I20260709 13:01:04.259016  7698 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:04.260553  7526 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:01:04.260881  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-1-root
uuid: "07fc46d8587b48f7b5015b4d639028a2"
format_stamp: "Formatted at 2026-07-09 13:01:04 on dist-test-slave-h0rz"
I20260709 13:01:04.261219  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:04.285782  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:04.287746  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:04.291091  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:04.294958  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:04.295202  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:04.295485  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:04.295637  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:04.417858  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.130:43993
I20260709 13:01:04.418040  7760 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.130:43993 every 8 connection(s)
I20260709 13:01:04.426419  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:04.439769  7764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:04.444856  7765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:04.444947  7761 heartbeater.cc:344] Connected to a master server at 127.7.89.190:44281
I20260709 13:01:04.445767  7761 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:04.447145  7761 heartbeater.cc:507] Master 127.7.89.190:44281 requested a full tablet report, sending...
W20260709 13:01:04.449015  7767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:04.450659  7558 ts_manager.cc:194] Registered new tserver with Master: 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:04.452189  7526 server_base.cc:1061] running on GCE node
I20260709 13:01:04.453073  7558 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58688
I20260709 13:01:04.453450  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:04.453742  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:04.453939  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602064453922 us; error 0 us; skew 500 ppm
I20260709 13:01:04.454814  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:04.458333  7526 webserver.cc:533] Webserver started at http://127.7.89.131:44653/ using document root <none> and password file <none>
I20260709 13:01:04.459134  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:04.459390  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:04.459712  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:04.461198  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-2-root/instance:
uuid: "0e62bb9587c142d9a58b1934bcea3a24"
format_stamp: "Formatted at 2026-07-09 13:01:04 on dist-test-slave-h0rz"
I20260709 13:01:04.467028  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.005s
I20260709 13:01:04.471314  7772 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:04.472509  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:01:04.472890  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-2-root
uuid: "0e62bb9587c142d9a58b1934bcea3a24"
format_stamp: "Formatted at 2026-07-09 13:01:04 on dist-test-slave-h0rz"
I20260709 13:01:04.473745  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602061986899-7526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:04.504267  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:04.505813  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:04.508877  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:04.513448  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:04.513717  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:04.514426  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:04.514743  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.002s	sys 0.000s
I20260709 13:01:04.643904  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.131:43031
I20260709 13:01:04.644562  7834 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.131:43031 every 8 connection(s)
I20260709 13:01:04.668424  7835 heartbeater.cc:344] Connected to a master server at 127.7.89.190:44281
I20260709 13:01:04.668933  7835 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:04.670066  7835 heartbeater.cc:507] Master 127.7.89.190:44281 requested a full tablet report, sending...
I20260709 13:01:04.673120  7558 ts_manager.cc:194] Registered new tserver with Master: 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:04.674436  7526 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.020535439s
I20260709 13:01:04.675436  7558 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58696
I20260709 13:01:04.754348  7558 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58708:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:01:04.762192  7558 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260709 13:01:04.838830  7606 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:04.906488  7724 tablet_service.cc:1511] Processing CreateTablet for tablet 08769b2911b946138991f109d8c538ff (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:01:04.908565  7724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08769b2911b946138991f109d8c538ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:04.909773  7723 tablet_service.cc:1511] Processing CreateTablet for tablet 514f802d953f454aa82f02440a5b0830 (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:01:04.912356  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 514f802d953f454aa82f02440a5b0830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:04.905927  7726 tablet_service.cc:1511] Processing CreateTablet for tablet 8f8df6cb823b42648888082885992fce (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:01:04.914973  7725 tablet_service.cc:1511] Processing CreateTablet for tablet 68993f87452b4c7c9920f2898da0a86c (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:01:04.920534  7725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68993f87452b4c7c9920f2898da0a86c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:04.921959  7726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f8df6cb823b42648888082885992fce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:04.949295  7798 tablet_service.cc:1511] Processing CreateTablet for tablet 08769b2911b946138991f109d8c538ff (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:01:04.941598  7799 tablet_service.cc:1511] Processing CreateTablet for tablet 68993f87452b4c7c9920f2898da0a86c (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:01:04.966095  7798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08769b2911b946138991f109d8c538ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:04.968094  7650 tablet_service.cc:1511] Processing CreateTablet for tablet 68993f87452b4c7c9920f2898da0a86c (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:01:04.959777  7800 tablet_service.cc:1511] Processing CreateTablet for tablet 8f8df6cb823b42648888082885992fce (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:01:04.970568  7650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68993f87452b4c7c9920f2898da0a86c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:04.972718  7722 tablet_service.cc:1511] Processing CreateTablet for tablet a3dcf4a6e49144739a6e8dd5d0038911 (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:01:04.981079  7722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3dcf4a6e49144739a6e8dd5d0038911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:04.973378  7651 tablet_service.cc:1511] Processing CreateTablet for tablet 8f8df6cb823b42648888082885992fce (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:01:04.984190  7651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f8df6cb823b42648888082885992fce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:04.977193  7799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68993f87452b4c7c9920f2898da0a86c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:04.988171  7800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f8df6cb823b42648888082885992fce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.007619  7797 tablet_service.cc:1511] Processing CreateTablet for tablet 514f802d953f454aa82f02440a5b0830 (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:01:05.011967  7649 tablet_service.cc:1511] Processing CreateTablet for tablet 08769b2911b946138991f109d8c538ff (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:01:05.013362  7797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 514f802d953f454aa82f02440a5b0830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.014046  7649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08769b2911b946138991f109d8c538ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.023403  7648 tablet_service.cc:1511] Processing CreateTablet for tablet 514f802d953f454aa82f02440a5b0830 (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:01:05.032490  7648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 514f802d953f454aa82f02440a5b0830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.029748  7796 tablet_service.cc:1511] Processing CreateTablet for tablet a3dcf4a6e49144739a6e8dd5d0038911 (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:01:05.041128  7796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3dcf4a6e49144739a6e8dd5d0038911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.063539  7647 tablet_service.cc:1511] Processing CreateTablet for tablet a3dcf4a6e49144739a6e8dd5d0038911 (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:01:05.069218  7647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3dcf4a6e49144739a6e8dd5d0038911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.096895  7646 tablet_service.cc:1511] Processing CreateTablet for tablet fd26df543a8b435ba0bc8ccdfac81287 (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:01:05.100785  7721 tablet_service.cc:1511] Processing CreateTablet for tablet fd26df543a8b435ba0bc8ccdfac81287 (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:01:05.103032  7721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd26df543a8b435ba0bc8ccdfac81287. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.104807  7646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd26df543a8b435ba0bc8ccdfac81287. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.135434  7853 tablet_bootstrap.cc:492] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2: Bootstrap starting.
I20260709 13:01:05.146956  7795 tablet_service.cc:1511] Processing CreateTablet for tablet fd26df543a8b435ba0bc8ccdfac81287 (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:01:05.148797  7795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd26df543a8b435ba0bc8ccdfac81287. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.148928  7645 tablet_service.cc:1511] Processing CreateTablet for tablet 2c42151fcb7c494abc2aa68118789a8d (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:01:05.150836  7645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c42151fcb7c494abc2aa68118789a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.151005  7853 tablet_bootstrap.cc:654] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.163355  7723 tablet_service.cc:1511] Processing CreateTablet for tablet 2c42151fcb7c494abc2aa68118789a8d (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:01:05.164964  7723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c42151fcb7c494abc2aa68118789a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.173342  7794 tablet_service.cc:1511] Processing CreateTablet for tablet 2c42151fcb7c494abc2aa68118789a8d (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:01:05.174981  7794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c42151fcb7c494abc2aa68118789a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.186268  7644 tablet_service.cc:1511] Processing CreateTablet for tablet e4243864b8434ee4bb2f34ae8383fab1 (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:01:05.190994  7644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4243864b8434ee4bb2f34ae8383fab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.217902  7725 tablet_service.cc:1511] Processing CreateTablet for tablet e4243864b8434ee4bb2f34ae8383fab1 (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:01:05.221385  7725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4243864b8434ee4bb2f34ae8383fab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.229027  7853 tablet_bootstrap.cc:492] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2: No bootstrap required, opened a new log
I20260709 13:01:05.229682  7853 ts_tablet_manager.cc:1403] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2: Time spent bootstrapping tablet: real 0.095s	user 0.019s	sys 0.004s
I20260709 13:01:05.258904  7686 heartbeater.cc:499] Master 127.7.89.190:44281 was elected leader, sending a full tablet report...
I20260709 13:01:05.268461  7853 raft_consensus.cc:359] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.270349  7853 raft_consensus.cc:385] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:05.270767  7853 raft_consensus.cc:740] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Initialized, Role: FOLLOWER
I20260709 13:01:05.273290  7853 consensus_queue.cc:260] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.290838  7855 tablet_bootstrap.cc:492] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de: Bootstrap starting.
I20260709 13:01:05.297760  7797 tablet_service.cc:1511] Processing CreateTablet for tablet e4243864b8434ee4bb2f34ae8383fab1 (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:01:05.299410  7797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4243864b8434ee4bb2f34ae8383fab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.305375  7856 tablet_bootstrap.cc:492] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24: Bootstrap starting.
I20260709 13:01:05.310084  7761 heartbeater.cc:499] Master 127.7.89.190:44281 was elected leader, sending a full tablet report...
I20260709 13:01:05.320561  7646 tablet_service.cc:1511] Processing CreateTablet for tablet 4b496f2cfc1b4f69ac86961166c72bdc (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:01:05.322031  7646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b496f2cfc1b4f69ac86961166c72bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.333837  7855 tablet_bootstrap.cc:654] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.333920  7724 tablet_service.cc:1511] Processing CreateTablet for tablet 4b496f2cfc1b4f69ac86961166c72bdc (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:01:05.336410  7724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b496f2cfc1b4f69ac86961166c72bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.333123  7853 ts_tablet_manager.cc:1434] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2: Time spent starting tablet: real 0.103s	user 0.014s	sys 0.025s
I20260709 13:01:05.348452  7853 tablet_bootstrap.cc:492] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2: Bootstrap starting.
I20260709 13:01:05.356545  7855 tablet_bootstrap.cc:492] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de: No bootstrap required, opened a new log
I20260709 13:01:05.357244  7855 ts_tablet_manager.cc:1403] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent bootstrapping tablet: real 0.077s	user 0.019s	sys 0.000s
I20260709 13:01:05.353191  7800 tablet_service.cc:1511] Processing CreateTablet for tablet 4b496f2cfc1b4f69ac86961166c72bdc (DEFAULT_TABLE table=test-workload [id=e0af478bb28f40da9ef5585ea734d061]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:01:05.359710  7800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b496f2cfc1b4f69ac86961166c72bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:05.360252  7853 tablet_bootstrap.cc:654] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.363039  7855 raft_consensus.cc:359] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.363880  7855 raft_consensus.cc:385] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:05.364342  7855 raft_consensus.cc:740] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Initialized, Role: FOLLOWER
I20260709 13:01:05.365119  7855 consensus_queue.cc:260] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.388482  7855 ts_tablet_manager.cc:1434] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent starting tablet: real 0.031s	user 0.024s	sys 0.006s
I20260709 13:01:05.393647  7856 tablet_bootstrap.cc:654] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.404290  7855 tablet_bootstrap.cc:492] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de: Bootstrap starting.
I20260709 13:01:05.411408  7855 tablet_bootstrap.cc:654] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.412564  7856 tablet_bootstrap.cc:492] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24: No bootstrap required, opened a new log
I20260709 13:01:05.413195  7856 ts_tablet_manager.cc:1403] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24: Time spent bootstrapping tablet: real 0.109s	user 0.009s	sys 0.007s
I20260709 13:01:05.417804  7853 tablet_bootstrap.cc:492] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2: No bootstrap required, opened a new log
I20260709 13:01:05.418526  7853 ts_tablet_manager.cc:1403] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2: Time spent bootstrapping tablet: real 0.075s	user 0.010s	sys 0.009s
I20260709 13:01:05.418383  7856 raft_consensus.cc:359] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.420056  7856 raft_consensus.cc:385] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:05.420770  7856 raft_consensus.cc:740] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Initialized, Role: FOLLOWER
I20260709 13:01:05.421985  7856 consensus_queue.cc:260] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.422551  7853 raft_consensus.cc:359] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.424474  7853 raft_consensus.cc:385] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:05.424958  7853 raft_consensus.cc:740] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Initialized, Role: FOLLOWER
I20260709 13:01:05.425997  7853 consensus_queue.cc:260] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
W20260709 13:01:05.427400  7762 tablet.cc:2406] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:01:05.429962  7853 ts_tablet_manager.cc:1434] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 13:01:05.433228  7856 ts_tablet_manager.cc:1434] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.010s
I20260709 13:01:05.434381  7856 tablet_bootstrap.cc:492] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24: Bootstrap starting.
I20260709 13:01:05.434948  7853 tablet_bootstrap.cc:492] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2: Bootstrap starting.
I20260709 13:01:05.435796  7835 heartbeater.cc:499] Master 127.7.89.190:44281 was elected leader, sending a full tablet report...
I20260709 13:01:05.441900  7855 tablet_bootstrap.cc:492] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de: No bootstrap required, opened a new log
I20260709 13:01:05.442557  7856 tablet_bootstrap.cc:654] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.442597  7853 tablet_bootstrap.cc:654] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.442674  7855 ts_tablet_manager.cc:1403] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent bootstrapping tablet: real 0.039s	user 0.000s	sys 0.025s
I20260709 13:01:05.447150  7855 raft_consensus.cc:359] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.448457  7855 raft_consensus.cc:385] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:05.448983  7855 raft_consensus.cc:740] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Initialized, Role: FOLLOWER
I20260709 13:01:05.457715  7855 consensus_queue.cc:260] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.458901  7857 raft_consensus.cc:493] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.459774  7857 raft_consensus.cc:515] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.462214  7855 ts_tablet_manager.cc:1434] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.003s
I20260709 13:01:05.463788  7855 tablet_bootstrap.cc:492] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de: Bootstrap starting.
I20260709 13:01:05.474632  7855 tablet_bootstrap.cc:654] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.474628  7857 leader_election.cc:290] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:05.490531  7855 tablet_bootstrap.cc:492] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de: No bootstrap required, opened a new log
I20260709 13:01:05.491849  7855 ts_tablet_manager.cc:1403] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.011s
I20260709 13:01:05.493505  7853 tablet_bootstrap.cc:492] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2: No bootstrap required, opened a new log
I20260709 13:01:05.494180  7853 ts_tablet_manager.cc:1403] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2: Time spent bootstrapping tablet: real 0.059s	user 0.017s	sys 0.003s
I20260709 13:01:05.499226  7855 raft_consensus.cc:359] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.500638  7855 raft_consensus.cc:385] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:05.501374  7855 raft_consensus.cc:740] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Initialized, Role: FOLLOWER
I20260709 13:01:05.502912  7855 consensus_queue.cc:260] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.507020  7855 ts_tablet_manager.cc:1434] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.005s
I20260709 13:01:05.508730  7855 tablet_bootstrap.cc:492] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de: Bootstrap starting.
I20260709 13:01:05.506821  7853 raft_consensus.cc:359] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.512748  7853 raft_consensus.cc:385] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:05.513183  7853 raft_consensus.cc:740] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Initialized, Role: FOLLOWER
I20260709 13:01:05.520720  7855 tablet_bootstrap.cc:654] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.514294  7853 consensus_queue.cc:260] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.523909  7853 ts_tablet_manager.cc:1434] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.005s
I20260709 13:01:05.525203  7853 tablet_bootstrap.cc:492] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2: Bootstrap starting.
I20260709 13:01:05.527697  7856 tablet_bootstrap.cc:492] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24: No bootstrap required, opened a new log
I20260709 13:01:05.528365  7856 ts_tablet_manager.cc:1403] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24: Time spent bootstrapping tablet: real 0.094s	user 0.057s	sys 0.033s
I20260709 13:01:05.535669  7856 raft_consensus.cc:359] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.537093  7856 raft_consensus.cc:385] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:05.537511  7856 raft_consensus.cc:740] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Initialized, Role: FOLLOWER
I20260709 13:01:05.543082  7856 consensus_queue.cc:260] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.548355  7856 ts_tablet_manager.cc:1434] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.001s
I20260709 13:01:05.549527  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514f802d953f454aa82f02440a5b0830" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" is_pre_election: true
I20260709 13:01:05.550020  7856 tablet_bootstrap.cc:492] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24: Bootstrap starting.
I20260709 13:01:05.550554  7661 raft_consensus.cc:2468] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 0.
I20260709 13:01:05.552950  7699 leader_election.cc:304] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:05.553964  7853 tablet_bootstrap.cc:654] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.554051  7857 raft_consensus.cc:2804] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:05.554535  7857 raft_consensus.cc:493] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.554948  7857 raft_consensus.cc:3060] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:05.561751  7856 tablet_bootstrap.cc:654] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.576097  7857 raft_consensus.cc:515] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.579653  7855 tablet_bootstrap.cc:492] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de: No bootstrap required, opened a new log
I20260709 13:01:05.580456  7855 ts_tablet_manager.cc:1403] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent bootstrapping tablet: real 0.072s	user 0.012s	sys 0.020s
I20260709 13:01:05.596320  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514f802d953f454aa82f02440a5b0830" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:05.597066  7661 raft_consensus.cc:3060] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:05.598436  7859 raft_consensus.cc:493] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.599510  7857 leader_election.cc:290] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:05.599171  7859 raft_consensus.cc:515] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.605124  7859 leader_election.cc:290] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:05.600682  7855 raft_consensus.cc:359] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.606688  7855 raft_consensus.cc:385] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:05.607399  7855 raft_consensus.cc:740] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Initialized, Role: FOLLOWER
I20260709 13:01:05.600739  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514f802d953f454aa82f02440a5b0830" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24" is_pre_election: true
I20260709 13:01:05.609977  7855 consensus_queue.cc:260] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.600114  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514f802d953f454aa82f02440a5b0830" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:05.637025  7855 ts_tablet_manager.cc:1434] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.004s
I20260709 13:01:05.644624  7855 tablet_bootstrap.cc:492] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de: Bootstrap starting.
W20260709 13:01:05.648280  7700 leader_election.cc:343] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:05.657258  7859 raft_consensus.cc:493] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.658100  7859 raft_consensus.cc:515] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.661422  7859 leader_election.cc:290] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:05.663967  7661 raft_consensus.cc:2468] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 1.
W20260709 13:01:05.682011  7700 leader_election.cc:343] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:05.682781  7699 leader_election.cc:304] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 0e62bb9587c142d9a58b1934bcea3a24
I20260709 13:01:05.685570  7857 raft_consensus.cc:2804] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:05.694576  7853 tablet_bootstrap.cc:492] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2: No bootstrap required, opened a new log
I20260709 13:01:05.695254  7853 ts_tablet_manager.cc:1403] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2: Time spent bootstrapping tablet: real 0.170s	user 0.051s	sys 0.076s
I20260709 13:01:05.700920  7855 tablet_bootstrap.cc:654] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.702396  7853 raft_consensus.cc:359] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.705125  7853 raft_consensus.cc:385] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:05.705473  7853 raft_consensus.cc:740] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Initialized, Role: FOLLOWER
I20260709 13:01:05.703014  7856 tablet_bootstrap.cc:492] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24: No bootstrap required, opened a new log
I20260709 13:01:05.704160  7857 raft_consensus.cc:697] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 LEADER]: Becoming Leader. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:05.707448  7856 ts_tablet_manager.cc:1403] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24: Time spent bootstrapping tablet: real 0.158s	user 0.028s	sys 0.045s
I20260709 13:01:05.708024  7853 consensus_queue.cc:260] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.711795  7857 consensus_queue.cc:237] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.714188  7853 ts_tablet_manager.cc:1434] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.002s
I20260709 13:01:05.715346  7853 tablet_bootstrap.cc:492] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2: Bootstrap starting.
I20260709 13:01:05.720302  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3dcf4a6e49144739a6e8dd5d0038911" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24" is_pre_election: true
I20260709 13:01:05.720681  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24" is_pre_election: true
I20260709 13:01:05.726505  7856 raft_consensus.cc:359] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.727435  7856 raft_consensus.cc:385] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:05.727756  7856 raft_consensus.cc:740] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Initialized, Role: FOLLOWER
W20260709 13:01:05.736693  7625 leader_election.cc:343] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:05.736621  7877 raft_consensus.cc:493] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.737577  7877 raft_consensus.cc:515] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.731830  7856 consensus_queue.cc:260] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.740417  7809 raft_consensus.cc:2468] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 0.
I20260709 13:01:05.743603  7877 leader_election.cc:290] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:05.744892  7856 ts_tablet_manager.cc:1434] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.006s
I20260709 13:01:05.744819  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24" is_pre_election: true
I20260709 13:01:05.746043  7856 tablet_bootstrap.cc:492] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24: Bootstrap starting.
W20260709 13:01:05.746752  7700 leader_election.cc:343] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:05.748196  7863 raft_consensus.cc:493] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.748261  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" is_pre_election: true
I20260709 13:01:05.749068  7863 raft_consensus.cc:515] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.752130  7855 tablet_bootstrap.cc:492] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de: No bootstrap required, opened a new log
W20260709 13:01:05.753677  7699 leader_election.cc:343] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:05.754307  7625 leader_election.cc:304] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:05.754456  7699 leader_election.cc:304] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2; no voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de
I20260709 13:01:05.755331  7859 raft_consensus.cc:2804] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:05.753765  7855 ts_tablet_manager.cc:1403] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent bootstrapping tablet: real 0.110s	user 0.019s	sys 0.014s
I20260709 13:01:05.755729  7859 raft_consensus.cc:493] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.758764  7859 raft_consensus.cc:3060] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:05.756942  7877 raft_consensus.cc:2749] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:05.770282  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3dcf4a6e49144739a6e8dd5d0038911" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2" is_pre_election: true
I20260709 13:01:05.774875  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2" is_pre_election: true
I20260709 13:01:05.773516  7877 raft_consensus.cc:493] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.776726  7877 raft_consensus.cc:515] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.775952  7735 raft_consensus.cc:2468] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 0.
I20260709 13:01:05.782275  7859 raft_consensus.cc:515] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
W20260709 13:01:05.783372  7626 leader_election.cc:343] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:05.785236  7859 leader_election.cc:290] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 election: Requested vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:05.786603  7626 leader_election.cc:304] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 0e62bb9587c142d9a58b1934bcea3a24
I20260709 13:01:05.787562  7859 raft_consensus.cc:2804] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:05.788066  7859 raft_consensus.cc:493] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.788187  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3dcf4a6e49144739a6e8dd5d0038911" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:05.788525  7859 raft_consensus.cc:3060] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:05.789995  7626 leader_election.cc:343] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:05.790889  7863 leader_election.cc:290] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:05.791682  7877 leader_election.cc:290] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:05.795733  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8df6cb823b42648888082885992fce" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24" is_pre_election: true
W20260709 13:01:05.797914  7700 leader_election.cc:343] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:05.808789  7853 tablet_bootstrap.cc:654] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.813041  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3dcf4a6e49144739a6e8dd5d0038911" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:05.815106  7809 raft_consensus.cc:3060] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:05.817607  7856 tablet_bootstrap.cc:654] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.818203  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8df6cb823b42648888082885992fce" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" is_pre_election: true
W20260709 13:01:05.823926  7699 leader_election.cc:343] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:05.826423  7809 raft_consensus.cc:2468] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 1.
I20260709 13:01:05.826684  7699 leader_election.cc:304] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2; no voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de
I20260709 13:01:05.826453  7859 raft_consensus.cc:515] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.831508  7879 raft_consensus.cc:2749] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:05.833443  7625 leader_election.cc:304] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 07fc46d8587b48f7b5015b4d639028a2
I20260709 13:01:05.838553  7859 leader_election.cc:290] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 election: Requested vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:05.839291  7859 raft_consensus.cc:2804] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:05.839465  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:05.839815  7859 raft_consensus.cc:697] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Becoming Leader. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:05.840138  7735 raft_consensus.cc:3060] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:05.840750  7859 consensus_queue.cc:237] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
W20260709 13:01:05.848258  7606 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:05.858598  7853 tablet_bootstrap.cc:492] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2: No bootstrap required, opened a new log
I20260709 13:01:05.865001  7853 ts_tablet_manager.cc:1403] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2: Time spent bootstrapping tablet: real 0.150s	user 0.024s	sys 0.009s
I20260709 13:01:05.864040  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:05.865900  7856 tablet_bootstrap.cc:492] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24: No bootstrap required, opened a new log
I20260709 13:01:05.868176  7856 ts_tablet_manager.cc:1403] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24: Time spent bootstrapping tablet: real 0.122s	user 0.011s	sys 0.006s
I20260709 13:01:05.868120  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd26df543a8b435ba0bc8ccdfac81287" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" is_pre_election: true
I20260709 13:01:05.859635  7855 raft_consensus.cc:359] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.869309  7661 raft_consensus.cc:2468] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 0.
I20260709 13:01:05.870653  7855 raft_consensus.cc:385] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:01:05.870834  7625 leader_election.cc:343] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:05.871269  7853 raft_consensus.cc:359] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.872107  7855 raft_consensus.cc:740] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Initialized, Role: FOLLOWER
I20260709 13:01:05.872910  7773 leader_election.cc:304] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:05.873220  7853 raft_consensus.cc:385] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:05.873651  7853 raft_consensus.cc:740] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Initialized, Role: FOLLOWER
I20260709 13:01:05.873963  7863 raft_consensus.cc:2804] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:05.874601  7863 raft_consensus.cc:493] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.875921  7863 raft_consensus.cc:3060] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:05.873838  7855 consensus_queue.cc:260] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.875999  7853 consensus_queue.cc:260] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.881572  7735 raft_consensus.cc:2468] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 1.
I20260709 13:01:05.883517  7626 leader_election.cc:304] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 0e62bb9587c142d9a58b1934bcea3a24
I20260709 13:01:05.887775  7884 raft_consensus.cc:493] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.890368  7856 raft_consensus.cc:359] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.891314  7856 raft_consensus.cc:385] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:05.891629  7856 raft_consensus.cc:740] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Initialized, Role: FOLLOWER
I20260709 13:01:05.892521  7856 consensus_queue.cc:260] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.908547  7853 ts_tablet_manager.cc:1434] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.004s
I20260709 13:01:05.903228  7884 raft_consensus.cc:515] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.919329  7855 ts_tablet_manager.cc:1434] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent starting tablet: real 0.163s	user 0.014s	sys 0.017s
I20260709 13:01:05.920502  7884 leader_election.cc:290] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:05.921415  7884 raft_consensus.cc:2804] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:05.921607  7853 tablet_bootstrap.cc:492] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2: Bootstrap starting.
I20260709 13:01:05.922032  7884 raft_consensus.cc:697] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Becoming Leader. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:05.922190  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24" is_pre_election: true
I20260709 13:01:05.923032  7884 consensus_queue.cc:237] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.913749  7863 raft_consensus.cc:515] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
W20260709 13:01:05.924481  7625 leader_election.cc:343] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:05.925853  7863 leader_election.cc:290] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:05.927484  7863 raft_consensus.cc:493] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.927901  7855 tablet_bootstrap.cc:492] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de: Bootstrap starting.
I20260709 13:01:05.927984  7863 raft_consensus.cc:515] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.930459  7863 leader_election.cc:290] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
W20260709 13:01:05.966128  7626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:43993, user_credentials={real_user=slave}} blocked reactor thread for 82688.4us
I20260709 13:01:05.971408  7735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd26df543a8b435ba0bc8ccdfac81287" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2" is_pre_election: true
I20260709 13:01:05.973824  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2" is_pre_election: true
I20260709 13:01:05.976897  7855 tablet_bootstrap.cc:654] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.973491  7891 raft_consensus.cc:493] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.977663  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd26df543a8b435ba0bc8ccdfac81287" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:05.978034  7736 raft_consensus.cc:2468] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 0.
W20260709 13:01:05.978385  7775 leader_election.cc:343] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:05.981923  7661 raft_consensus.cc:3060] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:05.983155  7856 ts_tablet_manager.cc:1434] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24: Time spent starting tablet: real 0.114s	user 0.012s	sys 0.007s
I20260709 13:01:05.982954  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08769b2911b946138991f109d8c538ff" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" is_pre_election: true
I20260709 13:01:05.984966  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd26df543a8b435ba0bc8ccdfac81287" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:05.984972  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08769b2911b946138991f109d8c538ff" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2" is_pre_election: true
I20260709 13:01:05.987428  7626 leader_election.cc:304] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 0e62bb9587c142d9a58b1934bcea3a24
I20260709 13:01:05.989696  7856 tablet_bootstrap.cc:492] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24: Bootstrap starting.
W20260709 13:01:05.990536  7775 leader_election.cc:343] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:05.991681  7775 leader_election.cc:343] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:05.993347  7853 tablet_bootstrap.cc:654] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:05.980749  7891 raft_consensus.cc:515] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:05.993726  7888 raft_consensus.cc:2804] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:01:05.995095  7773 leader_election.cc:343] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:05.995260  7888 raft_consensus.cc:493] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.995944  7773 leader_election.cc:304] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24; no voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de
I20260709 13:01:05.997073  7888 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.004164  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514f802d953f454aa82f02440a5b0830" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2" is_pre_election: true
I20260709 13:01:06.005714  7891 leader_election.cc:290] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:06.019842  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514f802d953f454aa82f02440a5b0830" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" is_pre_election: true
I20260709 13:01:06.020925  7660 raft_consensus.cc:2393] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e62bb9587c142d9a58b1934bcea3a24 in current term 1: Already voted for candidate 07fc46d8587b48f7b5015b4d639028a2 in this term.
I20260709 13:01:06.001538  7889 raft_consensus.cc:2749] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:06.020991  7661 raft_consensus.cc:2468] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 1.
I20260709 13:01:06.017635  7556 catalog_manager.cc:5697] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "07fc46d8587b48f7b5015b4d639028a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:06.024590  7558 catalog_manager.cc:5697] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de reported cstate change: term changed from 0 to 1, leader changed from <none> to 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129). New cstate: current_term: 1 leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:06.046878  7856 tablet_bootstrap.cc:654] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:06.047586  7773 leader_election.cc:304] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24; no voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de
I20260709 13:01:06.053735  7889 raft_consensus.cc:3060] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.057065  7773 leader_election.cc:304] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 07fc46d8587b48f7b5015b4d639028a2
I20260709 13:01:06.058532  7891 raft_consensus.cc:2804] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:06.059418  7891 raft_consensus.cc:697] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 LEADER]: Becoming Leader. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Running, Role: LEADER
I20260709 13:01:06.060487  7891 consensus_queue.cc:237] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.064107  7855 tablet_bootstrap.cc:492] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de: No bootstrap required, opened a new log
I20260709 13:01:06.068104  7855 ts_tablet_manager.cc:1403] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent bootstrapping tablet: real 0.140s	user 0.037s	sys 0.020s
I20260709 13:01:06.070801  7889 raft_consensus.cc:2749] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:06.072757  7855 raft_consensus.cc:359] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.073894  7855 raft_consensus.cc:385] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:06.073834  7888 raft_consensus.cc:515] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.074427  7855 raft_consensus.cc:740] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Initialized, Role: FOLLOWER
I20260709 13:01:06.077082  7855 consensus_queue.cc:260] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.080897  7853 tablet_bootstrap.cc:492] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2: No bootstrap required, opened a new log
I20260709 13:01:06.081179  7856 tablet_bootstrap.cc:492] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24: No bootstrap required, opened a new log
I20260709 13:01:06.081564  7853 ts_tablet_manager.cc:1403] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2: Time spent bootstrapping tablet: real 0.160s	user 0.012s	sys 0.027s
I20260709 13:01:06.081796  7856 ts_tablet_manager.cc:1403] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24: Time spent bootstrapping tablet: real 0.095s	user 0.017s	sys 0.009s
I20260709 13:01:06.084662  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:06.085587  7733 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.084801  7888 leader_election.cc:290] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 election: Requested vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:06.086974  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:06.087689  7855 ts_tablet_manager.cc:1434] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.001s
W20260709 13:01:06.089078  7625 leader_election.cc:343] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:06.089681  7853 raft_consensus.cc:359] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.090468  7853 raft_consensus.cc:385] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:06.090965  7853 raft_consensus.cc:740] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Initialized, Role: FOLLOWER
I20260709 13:01:06.091557  7855 tablet_bootstrap.cc:492] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de: Bootstrap starting.
I20260709 13:01:06.090560  7856 raft_consensus.cc:359] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.094856  7856 raft_consensus.cc:385] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:06.094939  7733 raft_consensus.cc:2468] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 1.
I20260709 13:01:06.095289  7856 raft_consensus.cc:740] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Initialized, Role: FOLLOWER
I20260709 13:01:06.091918  7853 consensus_queue.cc:260] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.097421  7853 ts_tablet_manager.cc:1434] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.004s
I20260709 13:01:06.098443  7853 tablet_bootstrap.cc:492] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2: Bootstrap starting.
I20260709 13:01:06.100592  7626 leader_election.cc:304] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 0e62bb9587c142d9a58b1934bcea3a24
I20260709 13:01:06.102349  7888 raft_consensus.cc:2804] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:06.102981  7888 raft_consensus.cc:697] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Becoming Leader. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:06.096843  7856 consensus_queue.cc:260] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.104117  7855 tablet_bootstrap.cc:654] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:06.103973  7888 consensus_queue.cc:237] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.106220  7856 ts_tablet_manager.cc:1434] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.003s
I20260709 13:01:06.107695  7856 tablet_bootstrap.cc:492] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24: Bootstrap starting.
I20260709 13:01:06.113963  7855 tablet_bootstrap.cc:492] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de: No bootstrap required, opened a new log
I20260709 13:01:06.114579  7855 ts_tablet_manager.cc:1403] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.001s
I20260709 13:01:06.116482  7853 tablet_bootstrap.cc:654] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:06.118319  7855 raft_consensus.cc:359] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.119411  7855 raft_consensus.cc:385] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:06.119728  7855 raft_consensus.cc:740] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Initialized, Role: FOLLOWER
I20260709 13:01:06.120687  7855 consensus_queue.cc:260] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.122691  7856 tablet_bootstrap.cc:654] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:06.129419  7855 ts_tablet_manager.cc:1434] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:01:06.130798  7855 tablet_bootstrap.cc:492] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de: Bootstrap starting.
I20260709 13:01:06.138306  7855 tablet_bootstrap.cc:654] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:06.146091  7856 tablet_bootstrap.cc:492] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24: No bootstrap required, opened a new log
I20260709 13:01:06.146925  7856 ts_tablet_manager.cc:1403] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24: Time spent bootstrapping tablet: real 0.039s	user 0.005s	sys 0.012s
I20260709 13:01:06.151221  7856 raft_consensus.cc:359] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.152401  7856 raft_consensus.cc:385] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:06.152824  7856 raft_consensus.cc:740] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Initialized, Role: FOLLOWER
I20260709 13:01:06.153765  7856 consensus_queue.cc:260] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.155442  7853 tablet_bootstrap.cc:492] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2: No bootstrap required, opened a new log
I20260709 13:01:06.155155  7556 catalog_manager.cc:5697] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "0e62bb9587c142d9a58b1934bcea3a24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:06.156585  7853 ts_tablet_manager.cc:1403] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2: Time spent bootstrapping tablet: real 0.058s	user 0.007s	sys 0.010s
I20260709 13:01:06.160298  7853 raft_consensus.cc:359] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.161001  7891 raft_consensus.cc:493] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.161180  7853 raft_consensus.cc:385] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:06.161392  7856 ts_tablet_manager.cc:1434] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.004s
I20260709 13:01:06.161571  7853 raft_consensus.cc:740] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Initialized, Role: FOLLOWER
I20260709 13:01:06.161572  7891 raft_consensus.cc:515] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.162444  7853 consensus_queue.cc:260] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.165683  7855 tablet_bootstrap.cc:492] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de: No bootstrap required, opened a new log
I20260709 13:01:06.165935  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8df6cb823b42648888082885992fce" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" is_pre_election: true
I20260709 13:01:06.166695  7661 raft_consensus.cc:2468] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 0.
I20260709 13:01:06.167927  7853 ts_tablet_manager.cc:1434] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:01:06.168432  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8df6cb823b42648888082885992fce" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2" is_pre_election: true
I20260709 13:01:06.169046  7853 tablet_bootstrap.cc:492] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2: Bootstrap starting.
I20260709 13:01:06.169093  7733 raft_consensus.cc:2468] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 0.
I20260709 13:01:06.170756  7856 tablet_bootstrap.cc:492] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24: Bootstrap starting.
I20260709 13:01:06.170686  7773 leader_election.cc:304] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:06.171687  7891 leader_election.cc:290] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:06.172271  7891 raft_consensus.cc:2804] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:06.166208  7855 ts_tablet_manager.cc:1403] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.006s
I20260709 13:01:06.172668  7891 raft_consensus.cc:493] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.173118  7891 raft_consensus.cc:3060] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.176570  7855 raft_consensus.cc:359] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.178433  7853 tablet_bootstrap.cc:654] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:06.182677  7855 raft_consensus.cc:385] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:06.183130  7855 raft_consensus.cc:740] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Initialized, Role: FOLLOWER
I20260709 13:01:06.178920  7558 catalog_manager.cc:5697] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de reported cstate change: term changed from 0 to 1, leader changed from <none> to 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129). New cstate: current_term: 1 leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:06.183976  7855 consensus_queue.cc:260] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.181393  7891 raft_consensus.cc:515] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.187090  7891 leader_election.cc:290] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:06.188943  7879 raft_consensus.cc:493] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.189363  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8df6cb823b42648888082885992fce" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:06.190104  7733 raft_consensus.cc:3060] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.189987  7879 raft_consensus.cc:515] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.191128  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8df6cb823b42648888082885992fce" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:06.191792  7661 raft_consensus.cc:3060] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.192582  7879 leader_election.cc:290] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:06.184554  7558 catalog_manager.cc:5697] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de reported cstate change: term changed from 0 to 1, leader changed from <none> to 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129). New cstate: current_term: 1 leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:06.194442  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd26df543a8b435ba0bc8ccdfac81287" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" is_pre_election: true
I20260709 13:01:06.194787  7856 tablet_bootstrap.cc:654] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:06.195361  7660 raft_consensus.cc:2393] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07fc46d8587b48f7b5015b4d639028a2 in current term 1: Already voted for candidate 0e62bb9587c142d9a58b1934bcea3a24 in this term.
I20260709 13:01:06.189275  7855 ts_tablet_manager.cc:1434] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:01:06.197167  7855 tablet_bootstrap.cc:492] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de: Bootstrap starting.
I20260709 13:01:06.197856  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd26df543a8b435ba0bc8ccdfac81287" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24" is_pre_election: true
I20260709 13:01:06.201118  7700 leader_election.cc:304] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2; no voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de
I20260709 13:01:06.202731  7879 raft_consensus.cc:3060] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.210999  7733 raft_consensus.cc:2468] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 1.
I20260709 13:01:06.212934  7661 raft_consensus.cc:2468] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 1.
I20260709 13:01:06.214695  7773 leader_election.cc:304] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:06.215827  7891 raft_consensus.cc:2804] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:06.216355  7891 raft_consensus.cc:697] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 LEADER]: Becoming Leader. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Running, Role: LEADER
I20260709 13:01:06.217374  7855 tablet_bootstrap.cc:654] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:06.213120  7879 raft_consensus.cc:2749] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:06.217291  7891 consensus_queue.cc:237] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.225543  7853 tablet_bootstrap.cc:492] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2: No bootstrap required, opened a new log
I20260709 13:01:06.226191  7853 ts_tablet_manager.cc:1403] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.005s
I20260709 13:01:06.262826  7856 tablet_bootstrap.cc:492] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24: No bootstrap required, opened a new log
I20260709 13:01:06.280578  7888 raft_consensus.cc:493] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.281255  7888 raft_consensus.cc:515] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.283757  7888 leader_election.cc:290] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:06.280578  7853 raft_consensus.cc:359] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.286249  7853 raft_consensus.cc:385] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:06.286707  7853 raft_consensus.cc:740] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Initialized, Role: FOLLOWER
I20260709 13:01:06.286864  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08769b2911b946138991f109d8c538ff" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24" is_pre_election: true
I20260709 13:01:06.287532  7853 consensus_queue.cc:260] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.287667  7809 raft_consensus.cc:2468] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 0.
I20260709 13:01:06.285943  7856 ts_tablet_manager.cc:1403] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24: Time spent bootstrapping tablet: real 0.118s	user 0.018s	sys 0.022s
I20260709 13:01:06.289872  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08769b2911b946138991f109d8c538ff" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2" is_pre_election: true
I20260709 13:01:06.290582  7733 raft_consensus.cc:2468] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 0.
I20260709 13:01:06.292013  7625 leader_election.cc:304] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:06.293028  7888 raft_consensus.cc:2804] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:06.293403  7888 raft_consensus.cc:493] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.293889  7888 raft_consensus.cc:3060] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.293423  7856 raft_consensus.cc:359] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.294327  7856 raft_consensus.cc:385] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:06.294672  7856 raft_consensus.cc:740] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Initialized, Role: FOLLOWER
I20260709 13:01:06.295480  7856 consensus_queue.cc:260] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.296854  7853 ts_tablet_manager.cc:1434] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.001s
I20260709 13:01:06.297911  7856 ts_tablet_manager.cc:1434] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:01:06.299017  7856 tablet_bootstrap.cc:492] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24: Bootstrap starting.
I20260709 13:01:06.297927  7853 tablet_bootstrap.cc:492] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2: Bootstrap starting.
I20260709 13:01:06.303994  7855 tablet_bootstrap.cc:492] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de: No bootstrap required, opened a new log
I20260709 13:01:06.304431  7855 ts_tablet_manager.cc:1403] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent bootstrapping tablet: real 0.108s	user 0.021s	sys 0.044s
I20260709 13:01:06.305923  7888 raft_consensus.cc:515] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.308161  7857 consensus_queue.cc:1048] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:06.310650  7855 raft_consensus.cc:359] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.311429  7855 raft_consensus.cc:385] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:06.310504  7557 catalog_manager.cc:5697] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "0e62bb9587c142d9a58b1934bcea3a24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:06.313213  7888 leader_election.cc:290] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 election: Requested vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:06.311688  7855 raft_consensus.cc:740] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Initialized, Role: FOLLOWER
I20260709 13:01:06.314261  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08769b2911b946138991f109d8c538ff" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:06.314674  7810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08769b2911b946138991f109d8c538ff" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:06.314342  7855 consensus_queue.cc:260] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.315006  7733 raft_consensus.cc:3060] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.315236  7810 raft_consensus.cc:3060] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.316838  7855 ts_tablet_manager.cc:1434] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:01:06.322793  7857 raft_consensus.cc:493] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.323776  7857 raft_consensus.cc:515] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.326236  7857 leader_election.cc:290] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:06.327379  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3dcf4a6e49144739a6e8dd5d0038911" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" is_pre_election: true
I20260709 13:01:06.331952  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3dcf4a6e49144739a6e8dd5d0038911" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24" is_pre_election: true
I20260709 13:01:06.333901  7808 raft_consensus.cc:2393] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07fc46d8587b48f7b5015b4d639028a2 in current term 1: Already voted for candidate 41df2eb1354a45e8a59e1e8ba3a508de in this term.
I20260709 13:01:06.337771  7700 leader_election.cc:304] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2; no voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de
I20260709 13:01:06.343518  7857 raft_consensus.cc:3060] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.346894  7856 tablet_bootstrap.cc:654] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:06.355275  7857 raft_consensus.cc:2749] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:06.347033  7733 raft_consensus.cc:2468] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 1.
I20260709 13:01:06.342921  7810 raft_consensus.cc:2468] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 1.
I20260709 13:01:06.361646  7625 leader_election.cc:304] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:06.361848  7879 consensus_queue.cc:1048] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:06.362635  7884 raft_consensus.cc:2804] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:06.351660  7853 tablet_bootstrap.cc:654] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:06.363229  7884 raft_consensus.cc:697] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Becoming Leader. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:06.364986  7884 consensus_queue.cc:237] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.370127  7859 consensus_queue.cc:1048] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:06.371625  7810 raft_consensus.cc:3060] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.394891  7859 consensus_queue.cc:1048] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:06.406158  7856 tablet_bootstrap.cc:492] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24: No bootstrap required, opened a new log
I20260709 13:01:06.406811  7856 ts_tablet_manager.cc:1403] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24: Time spent bootstrapping tablet: real 0.108s	user 0.015s	sys 0.013s
I20260709 13:01:06.414072  7853 tablet_bootstrap.cc:492] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2: No bootstrap required, opened a new log
I20260709 13:01:06.415575  7853 ts_tablet_manager.cc:1403] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2: Time spent bootstrapping tablet: real 0.118s	user 0.011s	sys 0.006s
I20260709 13:01:06.431344  7877 raft_consensus.cc:493] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.432224  7877 raft_consensus.cc:515] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.452083  7877 leader_election.cc:290] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:06.453251  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4243864b8434ee4bb2f34ae8383fab1" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" is_pre_election: true
I20260709 13:01:06.454372  7660 raft_consensus.cc:2468] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 0.
I20260709 13:01:06.458603  7856 raft_consensus.cc:359] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.461958  7856 raft_consensus.cc:385] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:06.463409  7856 raft_consensus.cc:740] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Initialized, Role: FOLLOWER
I20260709 13:01:06.463737  7557 catalog_manager.cc:5697] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de reported cstate change: term changed from 0 to 1, leader changed from <none> to 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129). New cstate: current_term: 1 leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:06.453251  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4243864b8434ee4bb2f34ae8383fab1" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24" is_pre_election: true
I20260709 13:01:06.465818  7699 leader_election.cc:304] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:06.468055  7808 raft_consensus.cc:2468] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 0.
I20260709 13:01:06.466187  7856 consensus_queue.cc:260] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.477240  7853 raft_consensus.cc:359] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.478363  7853 raft_consensus.cc:385] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:06.476962  7877 raft_consensus.cc:2804] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:06.478983  7853 raft_consensus.cc:740] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Initialized, Role: FOLLOWER
I20260709 13:01:06.479250  7884 consensus_queue.cc:1048] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:06.480471  7853 consensus_queue.cc:260] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.481263  7877 raft_consensus.cc:493] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.481863  7877 raft_consensus.cc:3060] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.507404  7853 ts_tablet_manager.cc:1434] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2: Time spent starting tablet: real 0.091s	user 0.007s	sys 0.002s
I20260709 13:01:06.508522  7856 ts_tablet_manager.cc:1434] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24: Time spent starting tablet: real 0.101s	user 0.013s	sys 0.000s
I20260709 13:01:06.509637  7856 tablet_bootstrap.cc:492] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24: Bootstrap starting.
I20260709 13:01:06.518869  7808 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.536099  7859 consensus_queue.cc:1048] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:06.540535  7889 consensus_queue.cc:1048] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:06.547446  7863 raft_consensus.cc:493] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.548341  7863 raft_consensus.cc:515] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.541749  7877 raft_consensus.cc:515] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.561249  7863 leader_election.cc:290] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:06.562991  7888 consensus_queue.cc:1048] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:06.563339  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" is_pre_election: true
I20260709 13:01:06.560245  7856 tablet_bootstrap.cc:654] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:06.636502  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4243864b8434ee4bb2f34ae8383fab1" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:06.637296  7661 raft_consensus.cc:3060] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.630995  7888 raft_consensus.cc:493] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.637894  7888 raft_consensus.cc:515] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.636894  7877 leader_election.cc:290] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:06.644281  7863 consensus_queue.cc:1048] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:01:06.646245  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4243864b8434ee4bb2f34ae8383fab1" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:06.646973  7809 raft_consensus.cc:3060] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.644284  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b496f2cfc1b4f69ac86961166c72bdc" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2" is_pre_election: true
I20260709 13:01:06.651312  7736 raft_consensus.cc:2468] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 0.
I20260709 13:01:06.640910  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2" is_pre_election: true
I20260709 13:01:06.654924  7888 leader_election.cc:290] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:06.657025  7626 leader_election.cc:304] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:06.657881  7806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b496f2cfc1b4f69ac86961166c72bdc" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24" is_pre_election: true
I20260709 13:01:06.660852  7888 raft_consensus.cc:2804] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:01:06.661262  7625 leader_election.cc:343] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:06.661325  7888 raft_consensus.cc:493] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.668216  7888 raft_consensus.cc:3060] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.657496  7734 raft_consensus.cc:2468] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 1.
I20260709 13:01:06.672179  7775 leader_election.cc:304] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 0e62bb9587c142d9a58b1934bcea3a24; no voters: 41df2eb1354a45e8a59e1e8ba3a508de
I20260709 13:01:06.683391  7889 consensus_queue.cc:1048] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:06.694010  7891 raft_consensus.cc:2804] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:06.698761  7891 raft_consensus.cc:493] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41df2eb1354a45e8a59e1e8ba3a508de)
I20260709 13:01:06.698809  7661 raft_consensus.cc:2468] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 1.
I20260709 13:01:06.699343  7891 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:06.700299  7699 leader_election.cc:304] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:06.700554  7888 raft_consensus.cc:515] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.701268  7877 raft_consensus.cc:2804] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:06.701704  7877 raft_consensus.cc:697] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 LEADER]: Becoming Leader. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:06.702421  7877 consensus_queue.cc:237] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.703383  7888 leader_election.cc:290] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 election: Requested vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:06.705061  7809 raft_consensus.cc:2468] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 1.
I20260709 13:01:06.711014  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b496f2cfc1b4f69ac86961166c72bdc" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:06.711752  7733 raft_consensus.cc:3060] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:06.712131  7888 consensus_queue.cc:1048] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:06.711396  7856 tablet_bootstrap.cc:492] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24: No bootstrap required, opened a new log
I20260709 13:01:06.713323  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b496f2cfc1b4f69ac86961166c72bdc" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
W20260709 13:01:06.716708  7625 leader_election.cc:343] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:06.720296  7856 ts_tablet_manager.cc:1403] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24: Time spent bootstrapping tablet: real 0.211s	user 0.037s	sys 0.053s
I20260709 13:01:06.724396  7891 raft_consensus.cc:515] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.732154  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:06.734483  7856 raft_consensus.cc:359] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.735898  7808 raft_consensus.cc:1240] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Rejecting Update request from peer 41df2eb1354a45e8a59e1e8ba3a508de for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:06.745851  7856 raft_consensus.cc:385] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:06.746317  7856 raft_consensus.cc:740] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Initialized, Role: FOLLOWER
I20260709 13:01:06.733762  7733 raft_consensus.cc:2468] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 1.
I20260709 13:01:06.736287  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:06.747545  7888 consensus_queue.cc:1059] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:06.749425  7626 leader_election.cc:304] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 0e62bb9587c142d9a58b1934bcea3a24
I20260709 13:01:06.751291  7886 raft_consensus.cc:2804] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:06.751838  7886 raft_consensus.cc:697] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Becoming Leader. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:06.752801  7886 consensus_queue.cc:237] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.748658  7775 leader_election.cc:304] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24; no voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de
I20260709 13:01:06.737738  7891 leader_election.cc:290] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 2 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:06.755254  7856 consensus_queue.cc:260] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.747273  7557 catalog_manager.cc:5697] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "07fc46d8587b48f7b5015b4d639028a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:06.760010  7859 raft_consensus.cc:3055] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:06.760540  7859 raft_consensus.cc:740] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:06.758875  7889 raft_consensus.cc:2749] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:06.764961  7856 ts_tablet_manager.cc:1434] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:01:06.766004  7859 consensus_queue.cc:260] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:06.767148  7859 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:06.784868  7889 consensus_queue.cc:1048] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:06.809082  7556 catalog_manager.cc:5697] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de reported cstate change: term changed from 0 to 1, leader changed from <none> to 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129). New cstate: current_term: 1 leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:06.831118  7886 consensus_queue.cc:1048] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:06.844406  7884 consensus_queue.cc:1048] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:06.985008  7808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f8df6cb823b42648888082885992fce"
dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
mode: GRACEFUL
new_leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
 from {username='slave'} at 127.0.0.1:52388
I20260709 13:01:06.985857  7808 raft_consensus.cc:606] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 LEADER]: Received request to transfer leadership to TS 41df2eb1354a45e8a59e1e8ba3a508de
I20260709 13:01:07.009819  7734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "514f802d953f454aa82f02440a5b0830"
dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
mode: GRACEFUL
new_leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
 from {username='slave'} at 127.0.0.1:53530
I20260709 13:01:07.010504  7734 raft_consensus.cc:606] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 LEADER]: Received request to transfer leadership to TS 41df2eb1354a45e8a59e1e8ba3a508de
I20260709 13:01:07.027870  7659 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3dcf4a6e49144739a6e8dd5d0038911"
dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
mode: GRACEFUL
new_leader_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:55648
I20260709 13:01:07.028585  7659 raft_consensus.cc:606] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Received request to transfer leadership to TS 07fc46d8587b48f7b5015b4d639028a2
I20260709 13:01:07.035215  7808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd26df543a8b435ba0bc8ccdfac81287"
dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
mode: GRACEFUL
new_leader_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:52388
I20260709 13:01:07.036010  7808 raft_consensus.cc:606] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 LEADER]: Received request to transfer leadership to TS 07fc46d8587b48f7b5015b4d639028a2
I20260709 13:01:07.042995  7659 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d"
dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
mode: GRACEFUL
new_leader_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:55648
I20260709 13:01:07.043653  7659 raft_consensus.cc:606] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Received request to transfer leadership to TS 07fc46d8587b48f7b5015b4d639028a2
I20260709 13:01:07.044005  7659 raft_consensus.cc:614] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:07.053391  7659 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b496f2cfc1b4f69ac86961166c72bdc"
dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
mode: GRACEFUL
new_leader_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
 from {username='slave'} at 127.0.0.1:55648
I20260709 13:01:07.054062  7659 raft_consensus.cc:606] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Received request to transfer leadership to TS 0e62bb9587c142d9a58b1934bcea3a24
I20260709 13:01:07.083045  7931 raft_consensus.cc:493] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.083814  7931 raft_consensus.cc:515] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.087018  7931 leader_election.cc:290] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:07.087642  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b496f2cfc1b4f69ac86961166c72bdc" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" is_pre_election: true
I20260709 13:01:07.087642  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b496f2cfc1b4f69ac86961166c72bdc" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2" is_pre_election: true
I20260709 13:01:07.088794  7734 raft_consensus.cc:2393] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e62bb9587c142d9a58b1934bcea3a24 in current term 1: Already voted for candidate 41df2eb1354a45e8a59e1e8ba3a508de in this term.
I20260709 13:01:07.089920  7775 leader_election.cc:304] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24; no voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de
I20260709 13:01:07.090691  7931 raft_consensus.cc:3060] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:07.099450  7931 raft_consensus.cc:2749] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:07.139734  7877 consensus_queue.cc:1048] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:07.153789  7879 consensus_queue.cc:1048] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:07.226794  7863 raft_consensus.cc:993] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24: : Instructing follower 07fc46d8587b48f7b5015b4d639028a2 to start an election
I20260709 13:01:07.227329  7931 raft_consensus.cc:1081] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 LEADER]: Signalling peer 07fc46d8587b48f7b5015b4d639028a2 to start an election
I20260709 13:01:07.229317  7734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd26df543a8b435ba0bc8ccdfac81287"
dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:53516
I20260709 13:01:07.229905  7734 raft_consensus.cc:493] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:07.230234  7734 raft_consensus.cc:3060] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.236071  7734 raft_consensus.cc:515] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.238319  7734 leader_election.cc:290] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 2 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:07.239504  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd26df543a8b435ba0bc8ccdfac81287" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:07.240046  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd26df543a8b435ba0bc8ccdfac81287" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:07.240305  7659 raft_consensus.cc:3060] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.240706  7808 raft_consensus.cc:3055] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:07.241104  7808 raft_consensus.cc:740] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Running, Role: LEADER
I20260709 13:01:07.242072  7808 consensus_queue.cc:260] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.243535  7808 raft_consensus.cc:3060] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.246706  7659 raft_consensus.cc:2468] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 2.
I20260709 13:01:07.248023  7699 leader_election.cc:304] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:07.248821  7942 raft_consensus.cc:2804] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:07.249315  7942 raft_consensus.cc:697] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 LEADER]: Becoming Leader. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:07.250243  7942 consensus_queue.cc:237] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.251204  7808 raft_consensus.cc:2468] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 2.
I20260709 13:01:07.257051  7884 consensus_queue.cc:1048] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:07.259464  7558 catalog_manager.cc:5697] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 reported cstate change: term changed from 1 to 2, leader changed from 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131) to 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "07fc46d8587b48f7b5015b4d639028a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:07.269428  7884 raft_consensus.cc:993] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de: : Instructing follower 0e62bb9587c142d9a58b1934bcea3a24 to start an election
I20260709 13:01:07.269958  7884 raft_consensus.cc:1081] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Signalling peer 0e62bb9587c142d9a58b1934bcea3a24 to start an election
I20260709 13:01:07.276362  7809 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b496f2cfc1b4f69ac86961166c72bdc"
dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
 from {username='slave'} at 127.0.0.1:52382
I20260709 13:01:07.277189  7809 raft_consensus.cc:493] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:07.277619  7809 raft_consensus.cc:3060] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.278275  7886 consensus_queue.cc:1048] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:07.287490  7809 raft_consensus.cc:515] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.288915  7808 raft_consensus.cc:1240] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Rejecting Update request from peer 41df2eb1354a45e8a59e1e8ba3a508de for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:07.291857  7809 leader_election.cc:290] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 2 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:07.293129  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b496f2cfc1b4f69ac86961166c72bdc" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:07.295945  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b496f2cfc1b4f69ac86961166c72bdc" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:07.296660  7886 consensus_queue.cc:1059] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:07.299535  7659 raft_consensus.cc:3055] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:07.300100  7659 raft_consensus.cc:740] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:07.301141  7659 consensus_queue.cc:260] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.303468  7942 raft_consensus.cc:993] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2: : Instructing follower 41df2eb1354a45e8a59e1e8ba3a508de to start an election
I20260709 13:01:07.303735  7659 raft_consensus.cc:3060] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.304041  7942 raft_consensus.cc:1081] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 LEADER]: Signalling peer 41df2eb1354a45e8a59e1e8ba3a508de to start an election
I20260709 13:01:07.306452  7660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "514f802d953f454aa82f02440a5b0830"
dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
 from {username='slave'} at 127.0.0.1:55640
I20260709 13:01:07.307165  7660 raft_consensus.cc:493] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:07.307574  7660 raft_consensus.cc:3060] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.311295  7659 raft_consensus.cc:2468] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 2.
W20260709 13:01:07.311849  7886 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:01:07.312657  7773 leader_election.cc:304] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 07fc46d8587b48f7b5015b4d639028a2
I20260709 13:01:07.313665  7931 raft_consensus.cc:2804] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:07.314234  7931 raft_consensus.cc:697] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 LEADER]: Becoming Leader. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Running, Role: LEADER
I20260709 13:01:07.315241  7931 consensus_queue.cc:237] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.315941  7660 raft_consensus.cc:515] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.319695  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514f802d953f454aa82f02440a5b0830" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:07.320348  7809 raft_consensus.cc:3060] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.321746  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514f802d953f454aa82f02440a5b0830" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:07.322423  7734 raft_consensus.cc:3055] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:07.322774  7734 raft_consensus.cc:740] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:07.323634  7734 consensus_queue.cc:260] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.324869  7734 raft_consensus.cc:3060] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.325500  7558 catalog_manager.cc:5697] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 reported cstate change: term changed from 1 to 2, leader changed from 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129) to 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "0e62bb9587c142d9a58b1934bcea3a24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:07.328938  7660 leader_election.cc:290] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 2 election: Requested vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:07.328585  7809 raft_consensus.cc:2468] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 2.
I20260709 13:01:07.331974  7625 leader_election.cc:304] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:07.334472  7734 raft_consensus.cc:2468] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 2.
I20260709 13:01:07.334853  7886 raft_consensus.cc:2804] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:07.335397  7886 raft_consensus.cc:697] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 LEADER]: Becoming Leader. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:07.336328  7886 consensus_queue.cc:237] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.337561  7863 raft_consensus.cc:993] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24: : Instructing follower 41df2eb1354a45e8a59e1e8ba3a508de to start an election
I20260709 13:01:07.338080  7931 raft_consensus.cc:1081] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 LEADER]: Signalling peer 41df2eb1354a45e8a59e1e8ba3a508de to start an election
I20260709 13:01:07.340214  7660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f8df6cb823b42648888082885992fce"
dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
 from {username='slave'} at 127.0.0.1:55646
I20260709 13:01:07.340986  7660 raft_consensus.cc:493] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:07.341399  7660 raft_consensus.cc:3060] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.347271  7558 catalog_manager.cc:5697] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de reported cstate change: term changed from 1 to 2, leader changed from 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130) to 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129). New cstate: current_term: 2 leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:07.350957  7660 raft_consensus.cc:515] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.353945  7660 leader_election.cc:290] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 2 election: Requested vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:07.356756  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8df6cb823b42648888082885992fce" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:07.357452  7809 raft_consensus.cc:3055] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:07.357815  7809 raft_consensus.cc:740] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Running, Role: LEADER
I20260709 13:01:07.358760  7809 consensus_queue.cc:260] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.360133  7809 raft_consensus.cc:3060] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.363193  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8df6cb823b42648888082885992fce" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:07.364002  7734 raft_consensus.cc:3060] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.371953  7734 raft_consensus.cc:2468] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 2.
I20260709 13:01:07.373492  7626 leader_election.cc:304] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:07.374575  7886 raft_consensus.cc:2804] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:07.375274  7886 raft_consensus.cc:697] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 LEADER]: Becoming Leader. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:07.376273  7886 consensus_queue.cc:237] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.378870  7809 raft_consensus.cc:2468] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 2.
I20260709 13:01:07.385144  7558 catalog_manager.cc:5697] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de reported cstate change: term changed from 1 to 2, leader changed from 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131) to 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129). New cstate: current_term: 2 leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:07.464767  7888 raft_consensus.cc:993] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de: : Instructing follower 07fc46d8587b48f7b5015b4d639028a2 to start an election
I20260709 13:01:07.465466  7884 raft_consensus.cc:1081] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Signalling peer 07fc46d8587b48f7b5015b4d639028a2 to start an election
I20260709 13:01:07.472973  7734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3dcf4a6e49144739a6e8dd5d0038911"
dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:53506
I20260709 13:01:07.473732  7734 raft_consensus.cc:493] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:07.474170  7734 raft_consensus.cc:3060] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.480116  7734 raft_consensus.cc:515] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.482270  7734 leader_election.cc:290] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 2 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:07.483146  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3dcf4a6e49144739a6e8dd5d0038911" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:07.483897  7660 raft_consensus.cc:3055] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:07.483839  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3dcf4a6e49144739a6e8dd5d0038911" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:07.484511  7660 raft_consensus.cc:740] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:07.484819  7809 raft_consensus.cc:3060] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.485476  7660 consensus_queue.cc:260] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.486568  7660 raft_consensus.cc:3060] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.492657  7660 raft_consensus.cc:2468] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 2.
I20260709 13:01:07.492807  7809 raft_consensus.cc:2468] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 2.
I20260709 13:01:07.494145  7699 leader_election.cc:304] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:07.495049  7877 raft_consensus.cc:2804] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:07.495607  7877 raft_consensus.cc:697] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 LEADER]: Becoming Leader. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:07.496488  7877 consensus_queue.cc:237] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:07.504881  7558 catalog_manager.cc:5697] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 reported cstate change: term changed from 1 to 2, leader changed from 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129) to 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "07fc46d8587b48f7b5015b4d639028a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:07.688433  7809 raft_consensus.cc:1275] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Refusing update from remote peer 07fc46d8587b48f7b5015b4d639028a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:07.690212  7877 consensus_queue.cc:1048] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:07.706243  7660 raft_consensus.cc:1275] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Refusing update from remote peer 07fc46d8587b48f7b5015b4d639028a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:07.708442  7942 consensus_queue.cc:1048] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:07.771780  7809 raft_consensus.cc:1275] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Refusing update from remote peer 41df2eb1354a45e8a59e1e8ba3a508de: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:07.773311  7734 raft_consensus.cc:3060] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.773316  7886 consensus_queue.cc:1048] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:07.781175  7734 raft_consensus.cc:1275] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Refusing update from remote peer 0e62bb9587c142d9a58b1934bcea3a24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:07.789361  7931 consensus_queue.cc:1048] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:07.793710  7734 raft_consensus.cc:1275] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Refusing update from remote peer 41df2eb1354a45e8a59e1e8ba3a508de: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:07.796113  7886 consensus_queue.cc:1048] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:07.808653  7660 raft_consensus.cc:1275] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Refusing update from remote peer 0e62bb9587c142d9a58b1934bcea3a24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:07.810367  7863 consensus_queue.cc:1048] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:07.848249  7733 raft_consensus.cc:1275] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Refusing update from remote peer 41df2eb1354a45e8a59e1e8ba3a508de: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:07.849611  7886 consensus_queue.cc:1048] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:07.855387  7809 raft_consensus.cc:1275] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Refusing update from remote peer 41df2eb1354a45e8a59e1e8ba3a508de: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:07.856932  7888 consensus_queue.cc:1048] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:07.922021  7660 raft_consensus.cc:1275] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Refusing update from remote peer 07fc46d8587b48f7b5015b4d639028a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:07.923547  7950 consensus_queue.cc:1048] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:07.935360  7809 raft_consensus.cc:1275] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Refusing update from remote peer 07fc46d8587b48f7b5015b4d639028a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:07.937202  7877 consensus_queue.cc:1048] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:08.219102  7877 raft_consensus.cc:493] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41df2eb1354a45e8a59e1e8ba3a508de)
I20260709 13:01:08.219784  7877 raft_consensus.cc:515] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:08.221854  7877 leader_election.cc:290] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:08.222836  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" is_pre_election: true
I20260709 13:01:08.223028  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24" is_pre_election: true
I20260709 13:01:08.223816  7809 raft_consensus.cc:2393] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07fc46d8587b48f7b5015b4d639028a2 in current term 2: Already voted for candidate 0e62bb9587c142d9a58b1934bcea3a24 in this term.
I20260709 13:01:08.223559  7660 raft_consensus.cc:2468] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 2.
I20260709 13:01:08.225205  7699 leader_election.cc:304] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 0e62bb9587c142d9a58b1934bcea3a24
I20260709 13:01:08.226017  7877 raft_consensus.cc:2804] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:08.226420  7877 raft_consensus.cc:493] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41df2eb1354a45e8a59e1e8ba3a508de)
I20260709 13:01:08.226915  7877 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:08.236020  7877 raft_consensus.cc:515] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:08.237844  7877 leader_election.cc:290] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 2 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:08.238919  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:08.238919  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:08.239691  7809 raft_consensus.cc:2393] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 07fc46d8587b48f7b5015b4d639028a2 in current term 2: Already voted for candidate 0e62bb9587c142d9a58b1934bcea3a24 in this term.
I20260709 13:01:08.245078  7660 raft_consensus.cc:2468] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 2.
I20260709 13:01:08.246526  7699 leader_election.cc:304] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 0e62bb9587c142d9a58b1934bcea3a24
I20260709 13:01:08.247694  7877 raft_consensus.cc:2804] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:08.248416  7877 raft_consensus.cc:697] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 2 LEADER]: Becoming Leader. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:08.249167  7877 consensus_queue.cc:237] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:08.257298  7556 catalog_manager.cc:5697] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 reported cstate change: term changed from 1 to 2, leader changed from 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129) to 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "07fc46d8587b48f7b5015b4d639028a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:08.402731  7863 raft_consensus.cc:493] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.403355  7863 raft_consensus.cc:515] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:08.405289  7863 leader_election.cc:290] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:08.406047  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" is_pre_election: true
I20260709 13:01:08.406297  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2" is_pre_election: true
I20260709 13:01:08.406774  7660 raft_consensus.cc:2468] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 2.
I20260709 13:01:08.407845  7773 leader_election.cc:304] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:08.408604  7863 raft_consensus.cc:2804] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:08.408978  7863 raft_consensus.cc:493] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.409227  7863 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.414516  7863 raft_consensus.cc:515] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:08.416883  7863 leader_election.cc:290] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 3 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:08.417797  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:08.418008  7733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:08.418382  7660 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.424072  7660 raft_consensus.cc:2468] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 3.
I20260709 13:01:08.425397  7773 leader_election.cc:304] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 07fc46d8587b48f7b5015b4d639028a2
I20260709 13:01:08.426365  7863 raft_consensus.cc:2804] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:08.426970  7863 raft_consensus.cc:697] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 LEADER]: Becoming Leader. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Running, Role: LEADER
I20260709 13:01:08.427868  7863 consensus_queue.cc:237] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:08.435441  7557 catalog_manager.cc:5697] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 reported cstate change: term changed from 2 to 3, leader changed from 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130) to 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "0e62bb9587c142d9a58b1934bcea3a24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:08.661994  7660 raft_consensus.cc:1240] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Rejecting Update request from peer 07fc46d8587b48f7b5015b4d639028a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:08.663535  7877 consensus_queue.cc:1059] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:08.664279  7879 raft_consensus.cc:3055] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:08.664628  7879 raft_consensus.cc:740] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:08.665508  7879 consensus_queue.cc:260] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:08.666700  7879 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:08.816576  7660 raft_consensus.cc:1275] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Refusing update from remote peer 0e62bb9587c142d9a58b1934bcea3a24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:08.818194  7863 consensus_queue.cc:1048] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:08.829936  7733 raft_consensus.cc:1275] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Refusing update from remote peer 0e62bb9587c142d9a58b1934bcea3a24: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:01:08.830466  7733 pending_rounds.cc:85] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2: Aborting all ops after (but not including) 1
I20260709 13:01:08.830885  7733 pending_rounds.cc:107] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:01:08.831194  7733 raft_consensus.cc:2889] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:08.833094  7946 consensus_queue.cc:1048] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:10.087970  7526 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:01:10.088920  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:10.115293  7660 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c"
dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
mode: GRACEFUL
new_leader_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
 from {username='slave'} at 127.0.0.1:55658
I20260709 13:01:10.115921  7660 raft_consensus.cc:606] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Received request to transfer leadership to TS 0e62bb9587c142d9a58b1934bcea3a24
I20260709 13:01:10.117244  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:01:10.117767  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:10.167213  7886 raft_consensus.cc:993] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de: : Instructing follower 0e62bb9587c142d9a58b1934bcea3a24 to start an election
I20260709 13:01:10.168061  7969 raft_consensus.cc:1081] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Signalling peer 0e62bb9587c142d9a58b1934bcea3a24 to start an election
I20260709 13:01:10.170084  7809 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c"
dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
 from {username='slave'} at 127.0.0.1:52382
I20260709 13:01:10.170732  7809 raft_consensus.cc:493] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:10.171062  7809 raft_consensus.cc:3060] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:10.176924  7809 raft_consensus.cc:515] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:10.179111  7809 leader_election.cc:290] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 2 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:10.180122  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:10.180462  7736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:10.180923  7660 raft_consensus.cc:3055] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:10.181317  7736 raft_consensus.cc:3060] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:10.181387  7660 raft_consensus.cc:740] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:10.182359  7660 consensus_queue.cc:260] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:10.183849  7660 raft_consensus.cc:3060] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:10.188117  7736 raft_consensus.cc:2468] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 2.
I20260709 13:01:10.190001  7660 raft_consensus.cc:2468] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 2.
I20260709 13:01:10.190153  7775 leader_election.cc:304] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 0e62bb9587c142d9a58b1934bcea3a24; no voters: 
I20260709 13:01:10.191195  7968 raft_consensus.cc:2804] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:10.192857  7968 raft_consensus.cc:697] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 LEADER]: Becoming Leader. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Running, Role: LEADER
I20260709 13:01:10.193848  7968 consensus_queue.cc:237] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:10.206794  7556 catalog_manager.cc:5697] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 reported cstate change: term changed from 1 to 2, leader changed from 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129) to 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "0e62bb9587c142d9a58b1934bcea3a24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:10.719589  7660 raft_consensus.cc:1275] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Refusing update from remote peer 0e62bb9587c142d9a58b1934bcea3a24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:10.721408  7968 consensus_queue.cc:1048] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:10.734069  7736 raft_consensus.cc:1275] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Refusing update from remote peer 0e62bb9587c142d9a58b1934bcea3a24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:10.739893  7979 consensus_queue.cc:1048] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:11.118683  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:11.130460  7660 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f8df6cb823b42648888082885992fce"
dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
mode: GRACEFUL
new_leader_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:55648
I20260709 13:01:11.131191  7660 raft_consensus.cc:606] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 LEADER]: Received request to transfer leadership to TS 07fc46d8587b48f7b5015b4d639028a2
I20260709 13:01:11.137568  7809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c"
dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
mode: GRACEFUL
new_leader_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:52388
I20260709 13:01:11.138247  7809 raft_consensus.cc:606] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 LEADER]: Received request to transfer leadership to TS 07fc46d8587b48f7b5015b4d639028a2
I20260709 13:01:11.144484  7660 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08769b2911b946138991f109d8c538ff"
dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
mode: GRACEFUL
new_leader_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:55648
I20260709 13:01:11.145237  7660 raft_consensus.cc:606] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Received request to transfer leadership to TS 07fc46d8587b48f7b5015b4d639028a2
I20260709 13:01:11.151234  7660 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "514f802d953f454aa82f02440a5b0830"
dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
mode: GRACEFUL
new_leader_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:55648
I20260709 13:01:11.151878  7660 raft_consensus.cc:606] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 LEADER]: Received request to transfer leadership to TS 07fc46d8587b48f7b5015b4d639028a2
I20260709 13:01:11.165036  7809 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d"
dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
mode: GRACEFUL
new_leader_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:52388
I20260709 13:01:11.165771  7809 raft_consensus.cc:606] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 LEADER]: Received request to transfer leadership to TS 07fc46d8587b48f7b5015b4d639028a2
I20260709 13:01:11.197927  7969 raft_consensus.cc:993] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de: : Instructing follower 07fc46d8587b48f7b5015b4d639028a2 to start an election
I20260709 13:01:11.198462  7987 raft_consensus.cc:1081] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 LEADER]: Signalling peer 07fc46d8587b48f7b5015b4d639028a2 to start an election
I20260709 13:01:11.200109  7736 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f8df6cb823b42648888082885992fce"
dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:53506
I20260709 13:01:11.200655  7736 raft_consensus.cc:493] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:11.200991  7736 raft_consensus.cc:3060] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.206645  7736 raft_consensus.cc:515] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.208875  7736 leader_election.cc:290] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 3 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:11.210000  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8df6cb823b42648888082885992fce" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:11.210031  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8df6cb823b42648888082885992fce" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:11.210887  7809 raft_consensus.cc:3060] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.210988  7660 raft_consensus.cc:3055] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:11.211413  7660 raft_consensus.cc:740] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:11.212136  7660 consensus_queue.cc:260] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.213276  7660 raft_consensus.cc:3060] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.216212  7809 raft_consensus.cc:2468] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 3.
I20260709 13:01:11.217577  7700 leader_election.cc:304] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 0e62bb9587c142d9a58b1934bcea3a24; no voters: 
I20260709 13:01:11.218482  7986 raft_consensus.cc:2804] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:11.219122  7986 raft_consensus.cc:697] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 3 LEADER]: Becoming Leader. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:11.220140  7660 raft_consensus.cc:2468] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 3.
I20260709 13:01:11.220086  7986 consensus_queue.cc:237] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.229421  7557 catalog_manager.cc:5697] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 reported cstate change: term changed from 2 to 3, leader changed from 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129) to 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130). New cstate: current_term: 3 leader_uuid: "07fc46d8587b48f7b5015b4d639028a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.243528  7987 raft_consensus.cc:993] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de: : Instructing follower 07fc46d8587b48f7b5015b4d639028a2 to start an election
I20260709 13:01:11.244164  7969 raft_consensus.cc:1081] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 LEADER]: Signalling peer 07fc46d8587b48f7b5015b4d639028a2 to start an election
I20260709 13:01:11.245824  7734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "514f802d953f454aa82f02440a5b0830"
dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:53506
I20260709 13:01:11.246486  7734 raft_consensus.cc:493] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:11.246929  7734 raft_consensus.cc:3060] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.252948  7734 raft_consensus.cc:515] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.255470  7734 leader_election.cc:290] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 3 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:11.256428  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514f802d953f454aa82f02440a5b0830" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:11.256677  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514f802d953f454aa82f02440a5b0830" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:11.257195  7660 raft_consensus.cc:3055] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:11.257375  7809 raft_consensus.cc:3060] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.257582  7660 raft_consensus.cc:740] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:11.258416  7660 consensus_queue.cc:260] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.259560  7660 raft_consensus.cc:3060] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.262671  7809 raft_consensus.cc:2468] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 3.
I20260709 13:01:11.264032  7700 leader_election.cc:304] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 0e62bb9587c142d9a58b1934bcea3a24; no voters: 
I20260709 13:01:11.265038  7986 raft_consensus.cc:2804] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:11.265605  7986 raft_consensus.cc:697] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 3 LEADER]: Becoming Leader. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:11.265630  7660 raft_consensus.cc:2468] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 3.
I20260709 13:01:11.266731  7986 consensus_queue.cc:237] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.275836  7557 catalog_manager.cc:5697] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 reported cstate change: term changed from 2 to 3, leader changed from 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129) to 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130). New cstate: current_term: 3 leader_uuid: "07fc46d8587b48f7b5015b4d639028a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.328199  7994 raft_consensus.cc:993] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24: : Instructing follower 07fc46d8587b48f7b5015b4d639028a2 to start an election
I20260709 13:01:11.328778  7994 raft_consensus.cc:1081] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 LEADER]: Signalling peer 07fc46d8587b48f7b5015b4d639028a2 to start an election
I20260709 13:01:11.330813  7736 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c"
dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:53516
I20260709 13:01:11.331492  7736 raft_consensus.cc:493] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:11.331914  7736 raft_consensus.cc:3060] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.332825  7987 raft_consensus.cc:993] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de: : Instructing follower 07fc46d8587b48f7b5015b4d639028a2 to start an election
I20260709 13:01:11.333321  7969 raft_consensus.cc:1081] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Signalling peer 07fc46d8587b48f7b5015b4d639028a2 to start an election
I20260709 13:01:11.335155  7734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08769b2911b946138991f109d8c538ff"
dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:53506
I20260709 13:01:11.335741  7734 raft_consensus.cc:493] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:11.336112  7734 raft_consensus.cc:3060] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.338135  7736 raft_consensus.cc:515] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.340473  7736 leader_election.cc:290] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 3 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:11.341470  7660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:11.341854  7809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:11.342136  7660 raft_consensus.cc:3060] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.342512  7809 raft_consensus.cc:3055] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:11.342928  7809 raft_consensus.cc:740] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Running, Role: LEADER
I20260709 13:01:11.342765  7734 raft_consensus.cc:515] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.343817  7809 consensus_queue.cc:260] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.345319  7734 leader_election.cc:290] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 2 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:11.346642  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08769b2911b946138991f109d8c538ff" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:11.345175  7809 raft_consensus.cc:3060] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.346673  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08769b2911b946138991f109d8c538ff" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:11.347323  7659 raft_consensus.cc:3055] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:11.347632  7808 raft_consensus.cc:3060] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.347697  7659 raft_consensus.cc:740] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:11.348644  7659 consensus_queue.cc:260] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.349936  7659 raft_consensus.cc:3060] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.350718  7660 raft_consensus.cc:2468] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 3.
I20260709 13:01:11.351979  7699 leader_election.cc:304] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:11.353067  7986 raft_consensus.cc:2804] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:11.353583  7986 raft_consensus.cc:697] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 3 LEADER]: Becoming Leader. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:11.355300  7809 raft_consensus.cc:2468] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 3.
I20260709 13:01:11.354470  7986 consensus_queue.cc:237] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.358318  7808 raft_consensus.cc:2468] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 2.
I20260709 13:01:11.358706  7659 raft_consensus.cc:2468] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 2.
I20260709 13:01:11.359724  7700 leader_election.cc:304] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 0e62bb9587c142d9a58b1934bcea3a24; no voters: 
I20260709 13:01:11.360666  7986 raft_consensus.cc:2804] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:11.361158  7986 raft_consensus.cc:697] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [term 2 LEADER]: Becoming Leader. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:11.361990  7986 consensus_queue.cc:237] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.364892  7557 catalog_manager.cc:5697] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 reported cstate change: term changed from 2 to 3, leader changed from 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131) to 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130). New cstate: current_term: 3 leader_uuid: "07fc46d8587b48f7b5015b4d639028a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.378458  7557 catalog_manager.cc:5697] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 reported cstate change: term changed from 1 to 2, leader changed from 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129) to 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "07fc46d8587b48f7b5015b4d639028a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.445811  7995 raft_consensus.cc:993] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24: : Instructing follower 07fc46d8587b48f7b5015b4d639028a2 to start an election
I20260709 13:01:11.446318  7995 raft_consensus.cc:1081] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 LEADER]: Signalling peer 07fc46d8587b48f7b5015b4d639028a2 to start an election
I20260709 13:01:11.448099  7734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d"
dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
 from {username='slave'} at 127.0.0.1:53516
I20260709 13:01:11.448671  7734 raft_consensus.cc:493] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:11.448987  7734 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.454694  7734 raft_consensus.cc:515] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.456755  7734 leader_election.cc:290] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 4 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:11.457839  7659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:11.458025  7808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "07fc46d8587b48f7b5015b4d639028a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:11.458487  7659 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.458678  7808 raft_consensus.cc:3055] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:11.459105  7808 raft_consensus.cc:740] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Running, Role: LEADER
I20260709 13:01:11.459910  7808 consensus_queue.cc:260] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.461057  7808 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:11.464807  7659 raft_consensus.cc:2468] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 4.
I20260709 13:01:11.466320  7699 leader_election.cc:304] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fc46d8587b48f7b5015b4d639028a2, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:11.467330  7986 raft_consensus.cc:2804] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:11.467437  7808 raft_consensus.cc:2468] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fc46d8587b48f7b5015b4d639028a2 in term 4.
I20260709 13:01:11.467850  7986 raft_consensus.cc:697] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 4 LEADER]: Becoming Leader. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:11.468832  7986 consensus_queue.cc:237] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:11.476652  7557 catalog_manager.cc:5697] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 reported cstate change: term changed from 3 to 4, leader changed from 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131) to 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130). New cstate: current_term: 4 leader_uuid: "07fc46d8587b48f7b5015b4d639028a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.663461  7659 raft_consensus.cc:1275] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Refusing update from remote peer 07fc46d8587b48f7b5015b4d639028a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:11.665113  7986 consensus_queue.cc:1048] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.677534  7808 raft_consensus.cc:1275] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Refusing update from remote peer 07fc46d8587b48f7b5015b4d639028a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:11.680385  7879 consensus_queue.cc:1048] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:11.764839  7808 raft_consensus.cc:1275] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Refusing update from remote peer 07fc46d8587b48f7b5015b4d639028a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:11.766414  7879 consensus_queue.cc:1048] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.776670  7660 raft_consensus.cc:1275] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Refusing update from remote peer 07fc46d8587b48f7b5015b4d639028a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:11.780443  7986 consensus_queue.cc:1048] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:11.921499  7659 raft_consensus.cc:1275] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Refusing update from remote peer 07fc46d8587b48f7b5015b4d639028a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:11.922935  7986 consensus_queue.cc:1048] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.934110  7808 raft_consensus.cc:1275] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Refusing update from remote peer 07fc46d8587b48f7b5015b4d639028a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:11.936004  8005 consensus_queue.cc:1048] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:11.956241  7659 raft_consensus.cc:1275] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Refusing update from remote peer 07fc46d8587b48f7b5015b4d639028a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:11.957674  7659 raft_consensus.cc:1275] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 4 FOLLOWER]: Refusing update from remote peer 07fc46d8587b48f7b5015b4d639028a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:11.957751  8005 consensus_queue.cc:1048] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:11.959200  8000 consensus_queue.cc:1048] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:11.970672  7809 raft_consensus.cc:1275] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Refusing update from remote peer 07fc46d8587b48f7b5015b4d639028a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:11.971129  7808 raft_consensus.cc:1275] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 4 FOLLOWER]: Refusing update from remote peer 07fc46d8587b48f7b5015b4d639028a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:11.973112  7986 consensus_queue.cc:1048] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:11.974124  8011 consensus_queue.cc:1048] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:14.231101  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:14.477383  7734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d"
dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
mode: GRACEFUL
new_leader_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
 from {username='slave'} at 127.0.0.1:34488
I20260709 13:01:14.478195  7734 raft_consensus.cc:606] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 4 LEADER]: Received request to transfer leadership to TS 0e62bb9587c142d9a58b1934bcea3a24
I20260709 13:01:14.482509  7734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c"
dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
mode: GRACEFUL
new_leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
 from {username='slave'} at 127.0.0.1:34488
I20260709 13:01:14.483275  7734 raft_consensus.cc:606] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 3 LEADER]: Received request to transfer leadership to TS 41df2eb1354a45e8a59e1e8ba3a508de
I20260709 13:01:14.486274  7734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f8df6cb823b42648888082885992fce"
dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
mode: GRACEFUL
new_leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
 from {username='slave'} at 127.0.0.1:34488
I20260709 13:01:14.487107  7734 raft_consensus.cc:606] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 3 LEADER]: Received request to transfer leadership to TS 41df2eb1354a45e8a59e1e8ba3a508de
I20260709 13:01:14.489886  7734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4243864b8434ee4bb2f34ae8383fab1"
dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
mode: GRACEFUL
new_leader_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
 from {username='slave'} at 127.0.0.1:34488
I20260709 13:01:14.490438  7734 raft_consensus.cc:606] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 LEADER]: Received request to transfer leadership to TS 0e62bb9587c142d9a58b1934bcea3a24
I20260709 13:01:14.493695  7734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd26df543a8b435ba0bc8ccdfac81287"
dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
mode: GRACEFUL
new_leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
 from {username='slave'} at 127.0.0.1:34488
I20260709 13:01:14.494338  7734 raft_consensus.cc:606] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 LEADER]: Received request to transfer leadership to TS 41df2eb1354a45e8a59e1e8ba3a508de
I20260709 13:01:14.496207  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:14.496865  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:01:14.814797  7589 debug-util.cc:398] Leaking SignalData structure 0x7b0800062060 after lost signal to thread 7529
W20260709 13:01:14.815717  7589 debug-util.cc:398] Leaking SignalData structure 0x7b0800062740 after lost signal to thread 7592
W20260709 13:01:14.816660  7589 debug-util.cc:398] Leaking SignalData structure 0x7b08001336c0 after lost signal to thread 7685
W20260709 13:01:14.818647  7589 debug-util.cc:398] Leaking SignalData structure 0x7b08000f02a0 after lost signal to thread 7834
I20260709 13:01:14.868592  8019 raft_consensus.cc:993] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2: : Instructing follower 41df2eb1354a45e8a59e1e8ba3a508de to start an election
I20260709 13:01:14.869371  8015 raft_consensus.cc:1081] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 3 LEADER]: Signalling peer 41df2eb1354a45e8a59e1e8ba3a508de to start an election
I20260709 13:01:14.871703  7661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f8df6cb823b42648888082885992fce"
dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
 from {username='slave'} at 127.0.0.1:55640
I20260709 13:01:14.872373  7661 raft_consensus.cc:493] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:14.872792  7661 raft_consensus.cc:3060] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.879956  7661 raft_consensus.cc:515] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:14.883214  7661 leader_election.cc:290] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 4 election: Requested vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:14.884407  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8df6cb823b42648888082885992fce" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:14.884196  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8df6cb823b42648888082885992fce" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:14.885175  7734 raft_consensus.cc:3055] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:14.885226  7805 raft_consensus.cc:3060] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.885671  7734 raft_consensus.cc:740] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:14.886832  7734 consensus_queue.cc:260] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:14.888392  7734 raft_consensus.cc:3060] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:14.894277  7805 raft_consensus.cc:2468] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 4.
I20260709 13:01:14.896308  7625 leader_election.cc:304] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:14.897554  7734 raft_consensus.cc:2468] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 4.
I20260709 13:01:14.900527  8023 raft_consensus.cc:2804] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:14.902134  8023 raft_consensus.cc:697] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 4 LEADER]: Becoming Leader. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:14.903260  8023 consensus_queue.cc:237] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:14.915725  7557 catalog_manager.cc:5697] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de reported cstate change: term changed from 3 to 4, leader changed from 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130) to 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129). New cstate: current_term: 4 leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:14.917066  8019 raft_consensus.cc:993] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2: : Instructing follower 0e62bb9587c142d9a58b1934bcea3a24 to start an election
I20260709 13:01:14.917672  8015 raft_consensus.cc:1081] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 LEADER]: Signalling peer 0e62bb9587c142d9a58b1934bcea3a24 to start an election
I20260709 13:01:14.920195  7805 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4243864b8434ee4bb2f34ae8383fab1"
dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
 from {username='slave'} at 127.0.0.1:52368
I20260709 13:01:14.920985  7805 raft_consensus.cc:493] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:14.921473  7805 raft_consensus.cc:3060] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.932013  7805 raft_consensus.cc:515] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:14.936496  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4243864b8434ee4bb2f34ae8383fab1" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:14.937368  7734 raft_consensus.cc:3055] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:14.936537  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4243864b8434ee4bb2f34ae8383fab1" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:14.937810  7734 raft_consensus.cc:740] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:14.938337  7661 raft_consensus.cc:3060] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.938951  7734 consensus_queue.cc:260] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:14.940647  7734 raft_consensus.cc:3060] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.943989  7805 leader_election.cc:290] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 2 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:14.947034  7661 raft_consensus.cc:2468] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 2.
I20260709 13:01:14.948602  7773 leader_election.cc:304] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:14.949905  8022 raft_consensus.cc:2804] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:14.950445  8022 raft_consensus.cc:697] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 LEADER]: Becoming Leader. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Running, Role: LEADER
I20260709 13:01:14.951427  8022 consensus_queue.cc:237] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:14.952772  7734 raft_consensus.cc:2468] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 2.
I20260709 13:01:14.962553  7557 catalog_manager.cc:5697] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 reported cstate change: term changed from 1 to 2, leader changed from 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130) to 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "0e62bb9587c142d9a58b1934bcea3a24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:14.976295  8019 raft_consensus.cc:993] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2: : Instructing follower 0e62bb9587c142d9a58b1934bcea3a24 to start an election
I20260709 13:01:14.977080  8019 raft_consensus.cc:1081] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 4 LEADER]: Signalling peer 0e62bb9587c142d9a58b1934bcea3a24 to start an election
I20260709 13:01:14.979187  7805 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d"
dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
 from {username='slave'} at 127.0.0.1:52368
I20260709 13:01:14.979749  7805 raft_consensus.cc:493] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:14.980101  7805 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:14.986796  7805 raft_consensus.cc:515] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:14.989290  7805 leader_election.cc:290] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 5 election: Requested vote from peers 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899), 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993)
I20260709 13:01:14.990113  7661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
I20260709 13:01:14.990859  7661 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:14.991379  8015 raft_consensus.cc:993] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2: : Instructing follower 41df2eb1354a45e8a59e1e8ba3a508de to start an election
I20260709 13:01:14.992050  8015 raft_consensus.cc:1081] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 3 LEADER]: Signalling peer 41df2eb1354a45e8a59e1e8ba3a508de to start an election
I20260709 13:01:14.992415  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c42151fcb7c494abc2aa68118789a8d" candidate_uuid: "0e62bb9587c142d9a58b1934bcea3a24" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:14.993129  7734 raft_consensus.cc:3055] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:14.993468  7734 raft_consensus.cc:740] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:14.994349  7734 consensus_queue.cc:260] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:14.995787  7734 raft_consensus.cc:3060] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:14.997149  7661 raft_consensus.cc:2468] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 5.
I20260709 13:01:14.998771  7773 leader_election.cc:304] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:14.999845  7661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c"
dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
 from {username='slave'} at 127.0.0.1:55640
I20260709 13:01:15.000034  8022 raft_consensus.cc:2804] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:15.000689  7661 raft_consensus.cc:493] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:15.000692  8022 raft_consensus.cc:697] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 5 LEADER]: Becoming Leader. State: Replica: 0e62bb9587c142d9a58b1934bcea3a24, State: Running, Role: LEADER
I20260709 13:01:15.001425  7661 raft_consensus.cc:3060] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:15.002067  8022 consensus_queue.cc:237] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:15.005012  7734 raft_consensus.cc:2468] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e62bb9587c142d9a58b1934bcea3a24 in term 5.
I20260709 13:01:15.010089  7661 raft_consensus.cc:515] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:15.013270  7661 leader_election.cc:290] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 4 election: Requested vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:15.013543  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:15.014269  7734 raft_consensus.cc:3055] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:15.014664  7734 raft_consensus.cc:740] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:15.013924  7557 catalog_manager.cc:5697] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 reported cstate change: term changed from 4 to 5, leader changed from 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130) to 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131). New cstate: current_term: 5 leader_uuid: "0e62bb9587c142d9a58b1934bcea3a24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:15.015666  7734 consensus_queue.cc:260] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:15.017010  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68993f87452b4c7c9920f2898da0a86c" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:15.017227  7734 raft_consensus.cc:3060] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:15.017824  7805 raft_consensus.cc:3060] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:15.026695  7805 raft_consensus.cc:2468] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 4.
I20260709 13:01:15.028067  7625 leader_election.cc:304] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:15.028404  7734 raft_consensus.cc:2468] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 4.
I20260709 13:01:15.029078  8023 raft_consensus.cc:2804] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:15.029597  8023 raft_consensus.cc:697] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 4 LEADER]: Becoming Leader. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:15.030546  8023 consensus_queue.cc:237] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:15.039721  7557 catalog_manager.cc:5697] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de reported cstate change: term changed from 3 to 4, leader changed from 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130) to 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129). New cstate: current_term: 4 leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:15.043720  8015 raft_consensus.cc:993] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2: : Instructing follower 41df2eb1354a45e8a59e1e8ba3a508de to start an election
I20260709 13:01:15.044260  8019 raft_consensus.cc:1081] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 LEADER]: Signalling peer 41df2eb1354a45e8a59e1e8ba3a508de to start an election
I20260709 13:01:15.045959  7661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd26df543a8b435ba0bc8ccdfac81287"
dest_uuid: "41df2eb1354a45e8a59e1e8ba3a508de"
 from {username='slave'} at 127.0.0.1:55640
I20260709 13:01:15.046582  7661 raft_consensus.cc:493] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:15.047031  7661 raft_consensus.cc:3060] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:15.053395  7661 raft_consensus.cc:515] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:15.056591  7734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd26df543a8b435ba0bc8ccdfac81287" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "07fc46d8587b48f7b5015b4d639028a2"
I20260709 13:01:15.056874  7661 leader_election.cc:290] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 3 election: Requested vote from peers 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993), 0e62bb9587c142d9a58b1934bcea3a24 (127.7.89.131:43031)
I20260709 13:01:15.057514  7734 raft_consensus.cc:3055] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:15.057569  7805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd26df543a8b435ba0bc8ccdfac81287" candidate_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e62bb9587c142d9a58b1934bcea3a24"
I20260709 13:01:15.058032  7734 raft_consensus.cc:740] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 07fc46d8587b48f7b5015b4d639028a2, State: Running, Role: LEADER
I20260709 13:01:15.058314  7805 raft_consensus.cc:3060] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:15.059060  7734 consensus_queue.cc:260] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:15.060498  7734 raft_consensus.cc:3060] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:15.064673  7805 raft_consensus.cc:2468] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 3.
I20260709 13:01:15.066188  7625 leader_election.cc:304] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e62bb9587c142d9a58b1934bcea3a24, 41df2eb1354a45e8a59e1e8ba3a508de; no voters: 
I20260709 13:01:15.067036  8023 raft_consensus.cc:2804] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:15.067545  8023 raft_consensus.cc:697] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 LEADER]: Becoming Leader. State: Replica: 41df2eb1354a45e8a59e1e8ba3a508de, State: Running, Role: LEADER
I20260709 13:01:15.068766  7734 raft_consensus.cc:2468] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41df2eb1354a45e8a59e1e8ba3a508de in term 3.
I20260709 13:01:15.068871  8023 consensus_queue.cc:237] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } }
I20260709 13:01:15.079042  7557 catalog_manager.cc:5697] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de reported cstate change: term changed from 2 to 3, leader changed from 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130) to 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129). New cstate: current_term: 3 leader_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:15.390036  7661 raft_consensus.cc:1275] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Refusing update from remote peer 0e62bb9587c142d9a58b1934bcea3a24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:15.391948  8022 consensus_queue.cc:1048] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:15.403822  7734 raft_consensus.cc:1275] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 4 FOLLOWER]: Refusing update from remote peer 41df2eb1354a45e8a59e1e8ba3a508de: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:15.405638  8023 consensus_queue.cc:1048] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:15.416839  7733 raft_consensus.cc:1275] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Refusing update from remote peer 0e62bb9587c142d9a58b1934bcea3a24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:15.425418  8037 consensus_queue.cc:1048] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:15.428460  7659 raft_consensus.cc:1275] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 5 FOLLOWER]: Refusing update from remote peer 0e62bb9587c142d9a58b1934bcea3a24: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:15.430521  8022 consensus_queue.cc:1048] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41df2eb1354a45e8a59e1e8ba3a508de" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:15.450531  7805 raft_consensus.cc:1275] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 4 FOLLOWER]: Refusing update from remote peer 41df2eb1354a45e8a59e1e8ba3a508de: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:15.452840  8023 consensus_queue.cc:1048] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:15.470305  7733 raft_consensus.cc:1275] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Refusing update from remote peer 41df2eb1354a45e8a59e1e8ba3a508de: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:15.472422  8023 consensus_queue.cc:1048] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:15.488492  7733 raft_consensus.cc:1275] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 5 FOLLOWER]: Refusing update from remote peer 0e62bb9587c142d9a58b1934bcea3a24: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:15.492431  8037 consensus_queue.cc:1048] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:15.494146  7733 raft_consensus.cc:1275] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 4 FOLLOWER]: Refusing update from remote peer 41df2eb1354a45e8a59e1e8ba3a508de: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:15.494462  7807 raft_consensus.cc:1275] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 4 FOLLOWER]: Refusing update from remote peer 41df2eb1354a45e8a59e1e8ba3a508de: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:15.496268  8024 consensus_queue.cc:1048] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "07fc46d8587b48f7b5015b4d639028a2" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:15.497867  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:15.498317  8023 consensus_queue.cc:1048] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:15.515251  7807 raft_consensus.cc:1275] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Refusing update from remote peer 41df2eb1354a45e8a59e1e8ba3a508de: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:15.518239  8023 consensus_queue.cc:1048] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e62bb9587c142d9a58b1934bcea3a24" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:15.506345  7526 tablet_server.cc:179] TabletServer@127.7.89.129:0 shutting down...
I20260709 13:01:15.624064  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:15.625020  7526 tablet_replica.cc:333] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de: stopping tablet replica
I20260709 13:01:15.625895  7526 raft_consensus.cc:2243] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.626516  7526 raft_consensus.cc:2272] T e4243864b8434ee4bb2f34ae8383fab1 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.630542  7526 tablet_replica.cc:333] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de: stopping tablet replica
I20260709 13:01:15.631296  7526 raft_consensus.cc:2243] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.631896  7526 raft_consensus.cc:2272] T 4b496f2cfc1b4f69ac86961166c72bdc P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.634016  7526 tablet_replica.cc:333] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de: stopping tablet replica
I20260709 13:01:15.634690  7526 raft_consensus.cc:2243] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.635227  7526 raft_consensus.cc:2272] T 08769b2911b946138991f109d8c538ff P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.637579  7526 tablet_replica.cc:333] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de: stopping tablet replica
I20260709 13:01:15.638296  7526 raft_consensus.cc:2243] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:01:15.639274  7526 raft_consensus.cc:2272] T 8f8df6cb823b42648888082885992fce P 41df2eb1354a45e8a59e1e8ba3a508de [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.641680  7526 tablet_replica.cc:333] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de: stopping tablet replica
I20260709 13:01:15.642443  7526 raft_consensus.cc:2243] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:15.643435  7526 raft_consensus.cc:2272] T fd26df543a8b435ba0bc8ccdfac81287 P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.645881  7526 tablet_replica.cc:333] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de: stopping tablet replica
I20260709 13:01:15.646476  7526 raft_consensus.cc:2243] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.647044  7526 raft_consensus.cc:2272] T 514f802d953f454aa82f02440a5b0830 P 41df2eb1354a45e8a59e1e8ba3a508de [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.649238  7526 tablet_replica.cc:333] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de: stopping tablet replica
I20260709 13:01:15.650033  7526 raft_consensus.cc:2243] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.650712  7526 raft_consensus.cc:2272] T 2c42151fcb7c494abc2aa68118789a8d P 41df2eb1354a45e8a59e1e8ba3a508de [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.653043  7526 tablet_replica.cc:333] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de: stopping tablet replica
I20260709 13:01:15.653604  7526 raft_consensus.cc:2243] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.654150  7526 raft_consensus.cc:2272] T a3dcf4a6e49144739a6e8dd5d0038911 P 41df2eb1354a45e8a59e1e8ba3a508de [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.656189  7526 tablet_replica.cc:333] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de: stopping tablet replica
I20260709 13:01:15.656697  7526 raft_consensus.cc:2243] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:01:15.657527  7526 raft_consensus.cc:2272] T 68993f87452b4c7c9920f2898da0a86c P 41df2eb1354a45e8a59e1e8ba3a508de [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:01:15.821339  7699 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.89.129:32899: connect: Connection refused (error 111)
I20260709 13:01:15.823539  7526 tablet_server.cc:196] TabletServer@127.7.89.129:0 shutdown complete.
W20260709 13:01:15.826017  7699 consensus_peers.cc:597] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 -> Peer 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899): Couldn't send request to peer 41df2eb1354a45e8a59e1e8ba3a508de. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:32899: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:15.826530  7699 consensus_peers.cc:597] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 -> Peer 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899): Couldn't send request to peer 41df2eb1354a45e8a59e1e8ba3a508de. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:32899: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:15.835278  7699 consensus_peers.cc:597] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 -> Peer 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899): Couldn't send request to peer 41df2eb1354a45e8a59e1e8ba3a508de. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:32899: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:15.844509  7773 consensus_peers.cc:597] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 -> Peer 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899): Couldn't send request to peer 41df2eb1354a45e8a59e1e8ba3a508de. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:32899: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:15.854730  7526 tablet_server.cc:179] TabletServer@127.7.89.130:0 shutting down...
I20260709 13:01:15.885272  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:15.886214  7526 tablet_replica.cc:333] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2: stopping tablet replica
I20260709 13:01:15.887200  7526 raft_consensus.cc:2243] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.888494  7526 raft_consensus.cc:2272] T 4b496f2cfc1b4f69ac86961166c72bdc P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.891594  7526 tablet_replica.cc:333] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2: stopping tablet replica
I20260709 13:01:15.892370  7526 raft_consensus.cc:2243] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.892943  7526 raft_consensus.cc:2272] T e4243864b8434ee4bb2f34ae8383fab1 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.895630  7526 tablet_replica.cc:333] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2: stopping tablet replica
I20260709 13:01:15.896237  7526 raft_consensus.cc:2243] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:15.897173  7526 raft_consensus.cc:2272] T a3dcf4a6e49144739a6e8dd5d0038911 P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.899858  7526 tablet_replica.cc:333] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2: stopping tablet replica
I20260709 13:01:15.900477  7526 raft_consensus.cc:2243] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.900985  7526 raft_consensus.cc:2272] T fd26df543a8b435ba0bc8ccdfac81287 P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.903505  7526 tablet_replica.cc:333] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2: stopping tablet replica
I20260709 13:01:15.904057  7526 raft_consensus.cc:2243] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:15.904852  7526 raft_consensus.cc:2272] T 08769b2911b946138991f109d8c538ff P 07fc46d8587b48f7b5015b4d639028a2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.906801  7526 tablet_replica.cc:333] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2: stopping tablet replica
I20260709 13:01:15.907359  7526 raft_consensus.cc:2243] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.907863  7526 raft_consensus.cc:2272] T 8f8df6cb823b42648888082885992fce P 07fc46d8587b48f7b5015b4d639028a2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.910431  7526 tablet_replica.cc:333] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2: stopping tablet replica
I20260709 13:01:15.911069  7526 raft_consensus.cc:2243] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:15.911962  7526 raft_consensus.cc:2272] T 514f802d953f454aa82f02440a5b0830 P 07fc46d8587b48f7b5015b4d639028a2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.914144  7526 tablet_replica.cc:333] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2: stopping tablet replica
I20260709 13:01:15.914765  7526 raft_consensus.cc:2243] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.915251  7526 raft_consensus.cc:2272] T 2c42151fcb7c494abc2aa68118789a8d P 07fc46d8587b48f7b5015b4d639028a2 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.918855  7526 tablet_replica.cc:333] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2: stopping tablet replica
I20260709 13:01:15.919826  7526 raft_consensus.cc:2243] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.920454  7526 raft_consensus.cc:2272] T 68993f87452b4c7c9920f2898da0a86c P 07fc46d8587b48f7b5015b4d639028a2 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:01:15.921842  7775 consensus_peers.cc:597] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 -> Peer 07fc46d8587b48f7b5015b4d639028a2 (127.7.89.130:43993): Couldn't send request to peer 07fc46d8587b48f7b5015b4d639028a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:15.954946  7526 tablet_server.cc:196] TabletServer@127.7.89.130:0 shutdown complete.
W20260709 13:01:15.983109  7773 consensus_peers.cc:597] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 -> Peer 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899): Couldn't send request to peer 41df2eb1354a45e8a59e1e8ba3a508de. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:32899: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:15.987529  7526 tablet_server.cc:179] TabletServer@127.7.89.131:0 shutting down...
I20260709 13:01:16.025285  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:16.026362  7526 tablet_replica.cc:333] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24: stopping tablet replica
I20260709 13:01:16.027401  7526 raft_consensus.cc:2243] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:16.028628  7526 raft_consensus.cc:2272] T 4b496f2cfc1b4f69ac86961166c72bdc P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:16.031733  7526 tablet_replica.cc:333] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24: stopping tablet replica
I20260709 13:01:16.032967  7526 raft_consensus.cc:2243] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:16.034303  7526 raft_consensus.cc:2272] T e4243864b8434ee4bb2f34ae8383fab1 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:16.038326  7526 tablet_replica.cc:333] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24: stopping tablet replica
I20260709 13:01:16.039319  7526 raft_consensus.cc:2243] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:16.039974  7526 raft_consensus.cc:2272] T a3dcf4a6e49144739a6e8dd5d0038911 P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:16.043241  7526 tablet_replica.cc:333] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24: stopping tablet replica
I20260709 13:01:16.044260  7526 raft_consensus.cc:2243] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:16.044986  7526 raft_consensus.cc:2272] T 68993f87452b4c7c9920f2898da0a86c P 0e62bb9587c142d9a58b1934bcea3a24 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:16.048050  7526 tablet_replica.cc:333] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24: stopping tablet replica
I20260709 13:01:16.048748  7526 raft_consensus.cc:2243] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 5 LEADER]: Raft consensus shutting down.
W20260709 13:01:16.048928  7773 consensus_peers.cc:597] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 -> Peer 41df2eb1354a45e8a59e1e8ba3a508de (127.7.89.129:32899): Couldn't send request to peer 41df2eb1354a45e8a59e1e8ba3a508de. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:32899: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:16.050029  7526 raft_consensus.cc:2272] T 2c42151fcb7c494abc2aa68118789a8d P 0e62bb9587c142d9a58b1934bcea3a24 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:16.052611  7526 tablet_replica.cc:333] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24: stopping tablet replica
I20260709 13:01:16.053244  7526 raft_consensus.cc:2243] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:16.053795  7526 raft_consensus.cc:2272] T 514f802d953f454aa82f02440a5b0830 P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:16.056013  7526 tablet_replica.cc:333] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24: stopping tablet replica
I20260709 13:01:16.056578  7526 raft_consensus.cc:2243] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:16.057086  7526 raft_consensus.cc:2272] T 8f8df6cb823b42648888082885992fce P 0e62bb9587c142d9a58b1934bcea3a24 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:16.059667  7526 tablet_replica.cc:333] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24: stopping tablet replica
I20260709 13:01:16.060263  7526 raft_consensus.cc:2243] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:16.060765  7526 raft_consensus.cc:2272] T 08769b2911b946138991f109d8c538ff P 0e62bb9587c142d9a58b1934bcea3a24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:16.063185  7526 tablet_replica.cc:333] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24: stopping tablet replica
I20260709 13:01:16.063778  7526 raft_consensus.cc:2243] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:16.064316  7526 raft_consensus.cc:2272] T fd26df543a8b435ba0bc8ccdfac81287 P 0e62bb9587c142d9a58b1934bcea3a24 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:16.095870  7526 tablet_server.cc:196] TabletServer@127.7.89.131:0 shutdown complete.
I20260709 13:01:16.127272  7526 master.cc:562] Master@127.7.89.190:44281 shutting down...
I20260709 13:01:16.358741  7613 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 13:01:16.383690  7526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:01:16.384447  7526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:16.384887  7526 tablet_replica.cc:333] T 00000000000000000000000000000000 P e46f255b825443eeba2f9ff5dd6cd4e9: stopping tablet replica
W20260709 13:01:17.403434  7526 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 7589)
I20260709 13:01:17.618348  7526 master.cc:584] Master@127.7.89.190:44281 shutdown complete.
07fc46d8587b48f7b5015b4d639028a2  3
0e62bb9587c142d9a58b1934bcea3a24  2
41df2eb1354a45e8a59e1e8ba3a508de  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (15484 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:01:17.665541  7526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.89.190:39063
I20260709 13:01:17.666966  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:17.673090  8057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:17.673216  8058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:17.676009  8060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:17.676652  7526 server_base.cc:1061] running on GCE node
I20260709 13:01:17.677685  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:17.677896  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:17.678083  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602077678065 us; error 0 us; skew 500 ppm
I20260709 13:01:17.678777  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:17.681612  7526 webserver.cc:533] Webserver started at http://127.7.89.190:38963/ using document root <none> and password file <none>
I20260709 13:01:17.682216  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:17.682441  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:17.682786  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:17.684101  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/master-0-root/instance:
uuid: "635b60e76903487ba0bb2d0cfb7ba020"
format_stamp: "Formatted at 2026-07-09 13:01:17 on dist-test-slave-h0rz"
I20260709 13:01:17.689335  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 13:01:17.693313  8065 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:17.694242  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 13:01:17.694734  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/master-0-root
uuid: "635b60e76903487ba0bb2d0cfb7ba020"
format_stamp: "Formatted at 2026-07-09 13:01:17 on dist-test-slave-h0rz"
I20260709 13:01:17.695125  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:17.705107  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:17.706686  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:17.756294  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.190:39063
I20260709 13:01:17.756417  8116 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.190:39063 every 8 connection(s)
I20260709 13:01:17.760993  8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:17.773042  8117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020: Bootstrap starting.
I20260709 13:01:17.778549  8117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:17.783946  8117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020: No bootstrap required, opened a new log
I20260709 13:01:17.786470  8117 raft_consensus.cc:359] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635b60e76903487ba0bb2d0cfb7ba020" member_type: VOTER }
I20260709 13:01:17.787052  8117 raft_consensus.cc:385] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:17.787248  8117 raft_consensus.cc:740] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 635b60e76903487ba0bb2d0cfb7ba020, State: Initialized, Role: FOLLOWER
I20260709 13:01:17.787890  8117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635b60e76903487ba0bb2d0cfb7ba020" member_type: VOTER }
I20260709 13:01:17.788370  8117 raft_consensus.cc:399] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:01:17.788563  8117 raft_consensus.cc:493] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:01:17.788787  8117 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:17.793571  8117 raft_consensus.cc:515] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635b60e76903487ba0bb2d0cfb7ba020" member_type: VOTER }
I20260709 13:01:17.794178  8117 leader_election.cc:304] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 635b60e76903487ba0bb2d0cfb7ba020; no voters: 
I20260709 13:01:17.795518  8117 leader_election.cc:290] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:01:17.796062  8120 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:17.797606  8120 raft_consensus.cc:697] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [term 1 LEADER]: Becoming Leader. State: Replica: 635b60e76903487ba0bb2d0cfb7ba020, State: Running, Role: LEADER
I20260709 13:01:17.798395  8120 consensus_queue.cc:237] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635b60e76903487ba0bb2d0cfb7ba020" member_type: VOTER }
I20260709 13:01:17.799032  8117 sys_catalog.cc:565] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:01:17.803494  8121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "635b60e76903487ba0bb2d0cfb7ba020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635b60e76903487ba0bb2d0cfb7ba020" member_type: VOTER } }
I20260709 13:01:17.803646  8122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 635b60e76903487ba0bb2d0cfb7ba020. Latest consensus state: current_term: 1 leader_uuid: "635b60e76903487ba0bb2d0cfb7ba020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635b60e76903487ba0bb2d0cfb7ba020" member_type: VOTER } }
I20260709 13:01:17.804363  8121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:17.804387  8122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:17.813337  8127 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:01:17.819468  8127 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:01:17.822773  7526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:01:17.829833  8127 catalog_manager.cc:1383] Generated new cluster ID: af7c4701ce124f708a975a1c5fa48cde
I20260709 13:01:17.830228  8127 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:01:17.853130  8127 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:01:17.854689  8127 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:01:17.874500  8127 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020: Generated new TSK 0
I20260709 13:01:17.875294  8127 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:01:17.889750  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:17.897639  8138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:17.899250  8139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:17.901331  8141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:17.901490  7526 server_base.cc:1061] running on GCE node
I20260709 13:01:17.902770  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:17.903003  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:17.903196  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602077903172 us; error 0 us; skew 500 ppm
I20260709 13:01:17.903824  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:17.906971  7526 webserver.cc:533] Webserver started at http://127.7.89.129:44383/ using document root <none> and password file <none>
I20260709 13:01:17.907562  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:17.907784  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:17.908094  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:17.909318  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-0-root/instance:
uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
format_stamp: "Formatted at 2026-07-09 13:01:17 on dist-test-slave-h0rz"
I20260709 13:01:17.914840  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.003s
I20260709 13:01:17.919433  8146 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:17.920447  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:01:17.920845  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-0-root
uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
format_stamp: "Formatted at 2026-07-09 13:01:17 on dist-test-slave-h0rz"
I20260709 13:01:17.921160  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:17.937191  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:17.938735  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:17.940461  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:17.943142  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:17.943406  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:17.943688  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:17.943889  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:17.992758  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.129:35953
I20260709 13:01:17.992887  8208 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.129:35953 every 8 connection(s)
I20260709 13:01:17.998593  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:18.006868  8213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:18.008389  8214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:18.012317  7526 server_base.cc:1061] running on GCE node
W20260709 13:01:18.012820  8216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:18.014134  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:18.014483  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:18.014541  8209 heartbeater.cc:344] Connected to a master server at 127.7.89.190:39063
I20260709 13:01:18.014724  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602078014701 us; error 0 us; skew 500 ppm
I20260709 13:01:18.015079  8209 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:18.015471  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:18.016103  8209 heartbeater.cc:507] Master 127.7.89.190:39063 requested a full tablet report, sending...
I20260709 13:01:18.018697  8082 ts_manager.cc:194] Registered new tserver with Master: 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953)
I20260709 13:01:18.018975  7526 webserver.cc:533] Webserver started at http://127.7.89.130:35323/ using document root <none> and password file <none>
I20260709 13:01:18.019621  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:18.019853  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:18.020164  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:18.020995  8082 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34224
I20260709 13:01:18.021595  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-1-root/instance:
uuid: "78dc985924c447f08ef73e4638a9d527"
format_stamp: "Formatted at 2026-07-09 13:01:18 on dist-test-slave-h0rz"
I20260709 13:01:18.027217  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.003s
I20260709 13:01:18.031212  8221 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:18.032199  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:01:18.032601  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-1-root
uuid: "78dc985924c447f08ef73e4638a9d527"
format_stamp: "Formatted at 2026-07-09 13:01:18 on dist-test-slave-h0rz"
I20260709 13:01:18.032949  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:18.044104  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:18.045588  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:18.047411  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:18.050082  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:18.050361  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:18.050699  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:18.050894  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:18.097705  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.130:42449
I20260709 13:01:18.097806  8283 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.130:42449 every 8 connection(s)
I20260709 13:01:18.103330  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:18.112811  8287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:18.114779  8288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:18.118384  8290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:18.118959  8284 heartbeater.cc:344] Connected to a master server at 127.7.89.190:39063
I20260709 13:01:18.119185  7526 server_base.cc:1061] running on GCE node
I20260709 13:01:18.119510  8284 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:18.120227  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:18.120452  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:18.120683  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602078120643 us; error 0 us; skew 500 ppm
I20260709 13:01:18.120688  8284 heartbeater.cc:507] Master 127.7.89.190:39063 requested a full tablet report, sending...
I20260709 13:01:18.121675  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:18.123687  8082 ts_manager.cc:194] Registered new tserver with Master: 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449)
I20260709 13:01:18.125170  7526 webserver.cc:533] Webserver started at http://127.7.89.131:37011/ using document root <none> and password file <none>
I20260709 13:01:18.125982  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:18.125861  8082 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34226
I20260709 13:01:18.126338  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:18.126751  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:18.128350  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-2-root/instance:
uuid: "6ed78a204d0b4fc5b898283bef2b4889"
format_stamp: "Formatted at 2026-07-09 13:01:18 on dist-test-slave-h0rz"
I20260709 13:01:18.133668  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 13:01:18.137663  8295 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:18.138600  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:01:18.139045  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-2-root
uuid: "6ed78a204d0b4fc5b898283bef2b4889"
format_stamp: "Formatted at 2026-07-09 13:01:18 on dist-test-slave-h0rz"
I20260709 13:01:18.139545  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602061986899-7526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:18.157066  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:18.158566  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:18.160391  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:18.163246  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:18.163552  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:18.163851  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:18.164041  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:18.212446  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.131:37523
I20260709 13:01:18.212519  8357 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.131:37523 every 8 connection(s)
I20260709 13:01:18.230217  8358 heartbeater.cc:344] Connected to a master server at 127.7.89.190:39063
I20260709 13:01:18.230894  8358 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:18.232210  8358 heartbeater.cc:507] Master 127.7.89.190:39063 requested a full tablet report, sending...
I20260709 13:01:18.235407  8082 ts_manager.cc:194] Registered new tserver with Master: 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:18.235533  7526 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019129438s
I20260709 13:01:18.237533  8082 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34240
I20260709 13:01:18.269603  8082 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34242:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:01:18.273012  8082 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:01:18.350188  8174 tablet_service.cc:1511] Processing CreateTablet for tablet e6ce814b173348d8ad6fbcbb7f0ceae8 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:01:18.352001  8174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6ce814b173348d8ad6fbcbb7f0ceae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.350071  8173 tablet_service.cc:1511] Processing CreateTablet for tablet c0ef3890cef64b66988b3243a5503e6b (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:01:18.353307  8173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0ef3890cef64b66988b3243a5503e6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.370193  8249 tablet_service.cc:1511] Processing CreateTablet for tablet e6ce814b173348d8ad6fbcbb7f0ceae8 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:01:18.371048  8248 tablet_service.cc:1511] Processing CreateTablet for tablet c0ef3890cef64b66988b3243a5503e6b (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:01:18.371877  8172 tablet_service.cc:1511] Processing CreateTablet for tablet 919af25e2da94f199867009579567a66 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:01:18.371910  8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6ce814b173348d8ad6fbcbb7f0ceae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.373478  8172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 919af25e2da94f199867009579567a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.375039  8248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0ef3890cef64b66988b3243a5503e6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.384320  8323 tablet_service.cc:1511] Processing CreateTablet for tablet e6ce814b173348d8ad6fbcbb7f0ceae8 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:01:18.384958  8247 tablet_service.cc:1511] Processing CreateTablet for tablet 919af25e2da94f199867009579567a66 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:01:18.386108  8323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6ce814b173348d8ad6fbcbb7f0ceae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.386471  8247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 919af25e2da94f199867009579567a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.388871  8171 tablet_service.cc:1511] Processing CreateTablet for tablet b2434e2b7fd54cc19302a5f1457391a6 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:01:18.390182  8246 tablet_service.cc:1511] Processing CreateTablet for tablet b2434e2b7fd54cc19302a5f1457391a6 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:01:18.391815  8246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2434e2b7fd54cc19302a5f1457391a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.395020  8170 tablet_service.cc:1511] Processing CreateTablet for tablet b99c6d9bc60d4dadb91d2ac9a4a09891 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:01:18.397521  8245 tablet_service.cc:1511] Processing CreateTablet for tablet b99c6d9bc60d4dadb91d2ac9a4a09891 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:01:18.399346  8245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99c6d9bc60d4dadb91d2ac9a4a09891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.401644  8169 tablet_service.cc:1511] Processing CreateTablet for tablet 7d8cefd485c946b4a33d75aade5c3668 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:01:18.404261  8244 tablet_service.cc:1511] Processing CreateTablet for tablet 7d8cefd485c946b4a33d75aade5c3668 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:01:18.406754  8244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d8cefd485c946b4a33d75aade5c3668. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.410508  8243 tablet_service.cc:1511] Processing CreateTablet for tablet 9e810a1ccf6947c2bcf984f4ec1e9dae (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:01:18.412235  8243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e810a1ccf6947c2bcf984f4ec1e9dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.415407  8168 tablet_service.cc:1511] Processing CreateTablet for tablet 9e810a1ccf6947c2bcf984f4ec1e9dae (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:01:18.417102  8167 tablet_service.cc:1511] Processing CreateTablet for tablet 7207cf9522d64ea49340bd7c2950431e (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:01:18.419646  8242 tablet_service.cc:1511] Processing CreateTablet for tablet 7207cf9522d64ea49340bd7c2950431e (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:01:18.421252  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7207cf9522d64ea49340bd7c2950431e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.424649  8166 tablet_service.cc:1511] Processing CreateTablet for tablet 456f553f05b141b2b43d3d710034fb7f (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:01:18.429546  8241 tablet_service.cc:1511] Processing CreateTablet for tablet 456f553f05b141b2b43d3d710034fb7f (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:01:18.432204  8374 tablet_bootstrap.cc:492] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527: Bootstrap starting.
I20260709 13:01:18.433912  8165 tablet_service.cc:1511] Processing CreateTablet for tablet f8a2b7f4f90441f78ede444a23726b3b (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:01:18.436219  8241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 456f553f05b141b2b43d3d710034fb7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.439218  8247 tablet_service.cc:1511] Processing CreateTablet for tablet f8a2b7f4f90441f78ede444a23726b3b (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:01:18.440970  8247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8a2b7f4f90441f78ede444a23726b3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.444089  8322 tablet_service.cc:1511] Processing CreateTablet for tablet c0ef3890cef64b66988b3243a5503e6b (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:01:18.445729  8322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0ef3890cef64b66988b3243a5503e6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.448675  8321 tablet_service.cc:1511] Processing CreateTablet for tablet 919af25e2da94f199867009579567a66 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:01:18.450464  8321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 919af25e2da94f199867009579567a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.456650  8320 tablet_service.cc:1511] Processing CreateTablet for tablet b2434e2b7fd54cc19302a5f1457391a6 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:01:18.458550  8320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2434e2b7fd54cc19302a5f1457391a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.466389  8319 tablet_service.cc:1511] Processing CreateTablet for tablet b99c6d9bc60d4dadb91d2ac9a4a09891 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:01:18.468734  8319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99c6d9bc60d4dadb91d2ac9a4a09891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.469098  8165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8a2b7f4f90441f78ede444a23726b3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.471558  8168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e810a1ccf6947c2bcf984f4ec1e9dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.474471  8167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7207cf9522d64ea49340bd7c2950431e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.475050  8317 tablet_service.cc:1511] Processing CreateTablet for tablet 9e810a1ccf6947c2bcf984f4ec1e9dae (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:01:18.476063  8171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2434e2b7fd54cc19302a5f1457391a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.478134  8317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e810a1ccf6947c2bcf984f4ec1e9dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.478235  8318 tablet_service.cc:1511] Processing CreateTablet for tablet 7d8cefd485c946b4a33d75aade5c3668 (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:01:18.480121  8318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d8cefd485c946b4a33d75aade5c3668. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.487237  8170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99c6d9bc60d4dadb91d2ac9a4a09891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.491127  8166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 456f553f05b141b2b43d3d710034fb7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.495098  8169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d8cefd485c946b4a33d75aade5c3668. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.513438  8316 tablet_service.cc:1511] Processing CreateTablet for tablet 7207cf9522d64ea49340bd7c2950431e (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:01:18.515317  8316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7207cf9522d64ea49340bd7c2950431e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.521857  8315 tablet_service.cc:1511] Processing CreateTablet for tablet 456f553f05b141b2b43d3d710034fb7f (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:01:18.523568  8315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 456f553f05b141b2b43d3d710034fb7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.529042  8314 tablet_service.cc:1511] Processing CreateTablet for tablet f8a2b7f4f90441f78ede444a23726b3b (DEFAULT_TABLE table=test-workload [id=271837a78d034954a3d395c73af726ea]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:01:18.533270  8314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8a2b7f4f90441f78ede444a23726b3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:18.574261  8374 tablet_bootstrap.cc:654] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.589073  8373 tablet_bootstrap.cc:492] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc: Bootstrap starting.
I20260709 13:01:18.621474  8374 tablet_bootstrap.cc:492] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527: No bootstrap required, opened a new log
I20260709 13:01:18.622072  8374 ts_tablet_manager.cc:1403] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527: Time spent bootstrapping tablet: real 0.192s	user 0.012s	sys 0.004s
I20260709 13:01:18.625344  8374 raft_consensus.cc:359] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.626137  8374 raft_consensus.cc:385] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.626451  8374 raft_consensus.cc:740] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.629091  8374 consensus_queue.cc:260] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.633181  8373 tablet_bootstrap.cc:654] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.634155  8376 tablet_bootstrap.cc:492] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889: Bootstrap starting.
I20260709 13:01:18.635953  8284 heartbeater.cc:499] Master 127.7.89.190:39063 was elected leader, sending a full tablet report...
I20260709 13:01:18.637157  8374 ts_tablet_manager.cc:1434] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.005s
I20260709 13:01:18.638334  8374 tablet_bootstrap.cc:492] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527: Bootstrap starting.
I20260709 13:01:18.646116  8376 tablet_bootstrap.cc:654] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.646116  8374 tablet_bootstrap.cc:654] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.673435  8377 raft_consensus.cc:493] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.674381  8377 raft_consensus.cc:515] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.681244  8377 leader_election.cc:290] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:18.697764  8373 tablet_bootstrap.cc:492] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc: No bootstrap required, opened a new log
I20260709 13:01:18.698510  8373 ts_tablet_manager.cc:1403] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent bootstrapping tablet: real 0.110s	user 0.033s	sys 0.039s
I20260709 13:01:18.701227  8374 tablet_bootstrap.cc:492] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527: No bootstrap required, opened a new log
I20260709 13:01:18.701874  8374 ts_tablet_manager.cc:1403] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527: Time spent bootstrapping tablet: real 0.064s	user 0.022s	sys 0.020s
I20260709 13:01:18.702432  8373 raft_consensus.cc:359] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.703331  8373 raft_consensus.cc:385] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.703738  8373 raft_consensus.cc:740] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.704881  8376 tablet_bootstrap.cc:492] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889: No bootstrap required, opened a new log
I20260709 13:01:18.704733  8373 consensus_queue.cc:260] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.705442  8376 ts_tablet_manager.cc:1403] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889: Time spent bootstrapping tablet: real 0.072s	user 0.003s	sys 0.037s
I20260709 13:01:18.707784  8374 raft_consensus.cc:359] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.708617  8374 raft_consensus.cc:385] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.709029  8374 raft_consensus.cc:740] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.708725  8376 raft_consensus.cc:359] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.709545  8376 raft_consensus.cc:385] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.709894  8376 raft_consensus.cc:740] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.710031  8374 consensus_queue.cc:260] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.712894  8374 ts_tablet_manager.cc:1434] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.005s
I20260709 13:01:18.714190  8374 tablet_bootstrap.cc:492] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527: Bootstrap starting.
I20260709 13:01:18.714709  8373 ts_tablet_manager.cc:1434] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.007s
I20260709 13:01:18.715415  8209 heartbeater.cc:499] Master 127.7.89.190:39063 was elected leader, sending a full tablet report...
I20260709 13:01:18.721230  8373 tablet_bootstrap.cc:492] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc: Bootstrap starting.
I20260709 13:01:18.722071  8374 tablet_bootstrap.cc:654] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.729460  8373 tablet_bootstrap.cc:654] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.735286  8374 tablet_bootstrap.cc:492] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527: No bootstrap required, opened a new log
I20260709 13:01:18.735904  8374 ts_tablet_manager.cc:1403] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.007s
I20260709 13:01:18.738255  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919af25e2da94f199867009579567a66" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" is_pre_election: true
I20260709 13:01:18.739971  8374 raft_consensus.cc:359] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
W20260709 13:01:18.740626  8223 leader_election.cc:343] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:18.740798  8374 raft_consensus.cc:385] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.741412  8374 raft_consensus.cc:740] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.742511  8374 consensus_queue.cc:260] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.745187  8333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919af25e2da94f199867009579567a66" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889" is_pre_election: true
W20260709 13:01:18.747215  8224 leader_election.cc:343] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:18.747674  8373 tablet_bootstrap.cc:492] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc: No bootstrap required, opened a new log
I20260709 13:01:18.747753  8224 leader_election.cc:304] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78dc985924c447f08ef73e4638a9d527; no voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889
I20260709 13:01:18.748495  8373 ts_tablet_manager.cc:1403] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.006s
I20260709 13:01:18.749181  8377 raft_consensus.cc:2749] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:18.751420  8374 ts_tablet_manager.cc:1434] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.001s
I20260709 13:01:18.752645  8374 tablet_bootstrap.cc:492] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527: Bootstrap starting.
I20260709 13:01:18.752271  8373 raft_consensus.cc:359] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.753198  8373 raft_consensus.cc:385] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.753585  8373 raft_consensus.cc:740] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.754415  8373 consensus_queue.cc:260] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.757090  8373 ts_tablet_manager.cc:1434] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:01:18.759281  8376 consensus_queue.cc:260] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.760064  8373 tablet_bootstrap.cc:492] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc: Bootstrap starting.
I20260709 13:01:18.760771  8386 raft_consensus.cc:493] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.761297  8386 raft_consensus.cc:515] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.764925  8358 heartbeater.cc:499] Master 127.7.89.190:39063 was elected leader, sending a full tablet report...
I20260709 13:01:18.765507  8376 ts_tablet_manager.cc:1434] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889: Time spent starting tablet: real 0.060s	user 0.013s	sys 0.000s
I20260709 13:01:18.766726  8376 tablet_bootstrap.cc:492] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889: Bootstrap starting.
I20260709 13:01:18.765524  8374 tablet_bootstrap.cc:654] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.774051  8386 leader_election.cc:290] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449)
I20260709 13:01:18.770807  8373 tablet_bootstrap.cc:654] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.783419  8376 tablet_bootstrap.cc:654] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.790213  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a2b7f4f90441f78ede444a23726b3b" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" is_pre_election: true
W20260709 13:01:18.792397  8297 leader_election.cc:343] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:18.796940  8373 tablet_bootstrap.cc:492] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc: No bootstrap required, opened a new log
I20260709 13:01:18.797678  8373 ts_tablet_manager.cc:1403] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent bootstrapping tablet: real 0.040s	user 0.011s	sys 0.010s
I20260709 13:01:18.801167  8373 raft_consensus.cc:359] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.802325  8373 raft_consensus.cc:385] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.802812  8373 raft_consensus.cc:740] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.803606  8373 consensus_queue.cc:260] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.805931  8373 ts_tablet_manager.cc:1434] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:01:18.807029  8373 tablet_bootstrap.cc:492] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc: Bootstrap starting.
I20260709 13:01:18.815122  8373 tablet_bootstrap.cc:654] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:18.817772  8130 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:18.824338  8376 tablet_bootstrap.cc:492] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889: No bootstrap required, opened a new log
I20260709 13:01:18.825017  8376 ts_tablet_manager.cc:1403] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889: Time spent bootstrapping tablet: real 0.059s	user 0.022s	sys 0.016s
I20260709 13:01:18.830020  8376 raft_consensus.cc:359] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.831194  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a2b7f4f90441f78ede444a23726b3b" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527" is_pre_election: true
I20260709 13:01:18.831205  8376 raft_consensus.cc:385] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.831938  8376 raft_consensus.cc:740] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Initialized, Role: FOLLOWER
W20260709 13:01:18.833283  8296 leader_election.cc:343] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:18.832932  8376 consensus_queue.cc:260] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.833847  8296 leader_election.cc:304] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889; no voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527
I20260709 13:01:18.834975  8374 tablet_bootstrap.cc:492] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527: No bootstrap required, opened a new log
I20260709 13:01:18.835282  8387 raft_consensus.cc:2749] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:18.835449  8373 tablet_bootstrap.cc:492] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc: No bootstrap required, opened a new log
I20260709 13:01:18.835508  8374 ts_tablet_manager.cc:1403] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527: Time spent bootstrapping tablet: real 0.083s	user 0.031s	sys 0.021s
I20260709 13:01:18.835987  8373 ts_tablet_manager.cc:1403] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.011s
I20260709 13:01:18.838950  8374 raft_consensus.cc:359] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.839108  8373 raft_consensus.cc:359] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.839890  8374 raft_consensus.cc:385] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.840095  8387 raft_consensus.cc:493] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.840283  8373 raft_consensus.cc:385] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.840416  8374 raft_consensus.cc:740] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.840683  8373 raft_consensus.cc:740] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.840775  8387 raft_consensus.cc:515] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.841446  8374 consensus_queue.cc:260] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.841818  8373 consensus_queue.cc:260] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.844174  8374 ts_tablet_manager.cc:1434] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:01:18.844497  8373 ts_tablet_manager.cc:1434] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:01:18.845271  8374 tablet_bootstrap.cc:492] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527: Bootstrap starting.
I20260709 13:01:18.845563  8373 tablet_bootstrap.cc:492] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc: Bootstrap starting.
I20260709 13:01:18.848780  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919af25e2da94f199867009579567a66" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" is_pre_election: true
I20260709 13:01:18.849864  8184 raft_consensus.cc:2468] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 0.
I20260709 13:01:18.851862  8297 leader_election.cc:304] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889; no voters: 
I20260709 13:01:18.853258  8373 tablet_bootstrap.cc:654] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.853446  8387 leader_election.cc:290] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449)
I20260709 13:01:18.854872  8386 raft_consensus.cc:2804] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:18.855311  8386 raft_consensus.cc:493] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.855226  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919af25e2da94f199867009579567a66" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527" is_pre_election: true
I20260709 13:01:18.855762  8386 raft_consensus.cc:3060] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:18.856017  8259 raft_consensus.cc:2468] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 0.
I20260709 13:01:18.864622  8386 raft_consensus.cc:515] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.866101  8376 ts_tablet_manager.cc:1434] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.004s
I20260709 13:01:18.867905  8376 tablet_bootstrap.cc:492] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889: Bootstrap starting.
I20260709 13:01:18.868798  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919af25e2da94f199867009579567a66" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
I20260709 13:01:18.869558  8184 raft_consensus.cc:3060] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:18.877498  8373 tablet_bootstrap.cc:492] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc: No bootstrap required, opened a new log
I20260709 13:01:18.877976  8184 raft_consensus.cc:2468] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 1.
I20260709 13:01:18.878183  8373 ts_tablet_manager.cc:1403] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.015s
I20260709 13:01:18.879400  8374 tablet_bootstrap.cc:654] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.879598  8297 leader_election.cc:304] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889; no voters: 
I20260709 13:01:18.881088  8387 raft_consensus.cc:2804] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:18.882200  8373 raft_consensus.cc:359] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.882987  8373 raft_consensus.cc:385] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.888211  8373 raft_consensus.cc:740] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.887686  8387 raft_consensus.cc:697] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 LEADER]: Becoming Leader. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Running, Role: LEADER
I20260709 13:01:18.889343  8387 consensus_queue.cc:237] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.889259  8373 consensus_queue.cc:260] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.890175  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919af25e2da94f199867009579567a66" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:18.891053  8376 tablet_bootstrap.cc:654] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.891126  8259 raft_consensus.cc:3060] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:18.894801  8373 ts_tablet_manager.cc:1434] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent starting tablet: real 0.016s	user 0.011s	sys 0.001s
I20260709 13:01:18.896529  8373 tablet_bootstrap.cc:492] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc: Bootstrap starting.
I20260709 13:01:18.901417  8386 leader_election.cc:290] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449)
I20260709 13:01:18.902274  8259 raft_consensus.cc:2468] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 1.
I20260709 13:01:18.911605  8373 tablet_bootstrap.cc:654] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.916281  8374 tablet_bootstrap.cc:492] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527: No bootstrap required, opened a new log
I20260709 13:01:18.916546  8376 tablet_bootstrap.cc:492] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889: No bootstrap required, opened a new log
I20260709 13:01:18.916972  8374 ts_tablet_manager.cc:1403] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527: Time spent bootstrapping tablet: real 0.072s	user 0.018s	sys 0.004s
I20260709 13:01:18.917156  8376 ts_tablet_manager.cc:1403] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889: Time spent bootstrapping tablet: real 0.050s	user 0.009s	sys 0.007s
I20260709 13:01:18.917609  8082 catalog_manager.cc:5697] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "6ed78a204d0b4fc5b898283bef2b4889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:18.920609  8374 raft_consensus.cc:359] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.921496  8374 raft_consensus.cc:385] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.921837  8374 raft_consensus.cc:740] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.922365  8376 raft_consensus.cc:359] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.923319  8373 tablet_bootstrap.cc:492] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc: No bootstrap required, opened a new log
I20260709 13:01:18.922744  8374 consensus_queue.cc:260] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.923529  8376 raft_consensus.cc:385] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.924007  8376 raft_consensus.cc:740] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.924109  8373 ts_tablet_manager.cc:1403] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.007s
I20260709 13:01:18.924989  8376 consensus_queue.cc:260] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.927752  8376 ts_tablet_manager.cc:1434] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:01:18.928987  8376 tablet_bootstrap.cc:492] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889: Bootstrap starting.
I20260709 13:01:18.928741  8373 raft_consensus.cc:359] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.929842  8373 raft_consensus.cc:385] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.930280  8373 raft_consensus.cc:740] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.931571  8373 consensus_queue.cc:260] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.925554  8374 ts_tablet_manager.cc:1434] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:01:18.937678  8376 tablet_bootstrap.cc:654] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.939069  8373 ts_tablet_manager.cc:1434] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 13:01:18.940691  8373 tablet_bootstrap.cc:492] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc: Bootstrap starting.
I20260709 13:01:18.947916  8374 tablet_bootstrap.cc:492] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527: Bootstrap starting.
I20260709 13:01:18.951877  8373 tablet_bootstrap.cc:654] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.969257  8373 tablet_bootstrap.cc:492] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc: No bootstrap required, opened a new log
I20260709 13:01:18.970109  8373 ts_tablet_manager.cc:1403] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.008s
I20260709 13:01:18.971208  8374 tablet_bootstrap.cc:654] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.973803  8373 raft_consensus.cc:359] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.976181  8373 raft_consensus.cc:385] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.976539  8373 raft_consensus.cc:740] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.977694  8383 raft_consensus.cc:493] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.978044  8376 tablet_bootstrap.cc:492] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889: No bootstrap required, opened a new log
I20260709 13:01:18.978641  8376 ts_tablet_manager.cc:1403] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889: Time spent bootstrapping tablet: real 0.050s	user 0.019s	sys 0.023s
I20260709 13:01:18.978335  8383 raft_consensus.cc:515] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.979512  8374 tablet_bootstrap.cc:492] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527: No bootstrap required, opened a new log
I20260709 13:01:18.980052  8374 ts_tablet_manager.cc:1403] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.004s
I20260709 13:01:18.982095  8376 raft_consensus.cc:359] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.982986  8376 raft_consensus.cc:385] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.983320  8376 raft_consensus.cc:740] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.983232  8374 raft_consensus.cc:359] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.984076  8374 raft_consensus.cc:385] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:18.984400  8374 raft_consensus.cc:740] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Initialized, Role: FOLLOWER
I20260709 13:01:18.984369  8376 consensus_queue.cc:260] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.985522  8374 consensus_queue.cc:260] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:18.987248  8376 ts_tablet_manager.cc:1434] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:01:18.988483  8376 tablet_bootstrap.cc:492] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889: Bootstrap starting.
I20260709 13:01:18.992770  8374 ts_tablet_manager.cc:1434] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:01:18.994199  8374 tablet_bootstrap.cc:492] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527: Bootstrap starting.
I20260709 13:01:18.995906  8376 tablet_bootstrap.cc:654] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:18.996536  8383 leader_election.cc:290] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.004134  8377 raft_consensus.cc:493] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.004948  8377 raft_consensus.cc:515] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.006040  8373 consensus_queue.cc:260] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.009546  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6ce814b173348d8ad6fbcbb7f0ceae8" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" is_pre_election: true
I20260709 13:01:19.010601  8184 raft_consensus.cc:2468] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 0.
I20260709 13:01:19.012638  8223 leader_election.cc:304] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:19.015251  8377 leader_election.cc:290] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.016101  8377 raft_consensus.cc:493] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.016736  8377 raft_consensus.cc:515] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.018138  8383 raft_consensus.cc:493] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.018908  8383 raft_consensus.cc:515] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.021441  8409 raft_consensus.cc:2804] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:19.022135  8383 leader_election.cc:290] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.021443  8377 leader_election.cc:290] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.023897  8333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6ce814b173348d8ad6fbcbb7f0ceae8" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889" is_pre_election: true
I20260709 13:01:19.025601  8373 ts_tablet_manager.cc:1434] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent starting tablet: real 0.055s	user 0.004s	sys 0.004s
I20260709 13:01:19.026932  8333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d8cefd485c946b4a33d75aade5c3668" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889" is_pre_election: true
I20260709 13:01:19.027913  8333 raft_consensus.cc:2468] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 0.
W20260709 13:01:19.028368  8224 leader_election.cc:343] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.025620  8374 tablet_bootstrap.cc:654] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:19.029214  8224 leader_election.cc:304] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:19.030184  8377 raft_consensus.cc:2804] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:19.030689  8377 raft_consensus.cc:493] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.031097  8377 raft_consensus.cc:3060] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.037520  8373 tablet_bootstrap.cc:492] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc: Bootstrap starting.
I20260709 13:01:19.037973  8376 tablet_bootstrap.cc:492] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889: No bootstrap required, opened a new log
I20260709 13:01:19.038126  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d8cefd485c946b4a33d75aade5c3668" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" is_pre_election: true
I20260709 13:01:19.038578  8376 ts_tablet_manager.cc:1403] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.008s
I20260709 13:01:19.038933  8184 raft_consensus.cc:2468] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 0.
I20260709 13:01:19.041674  8387 raft_consensus.cc:493] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.042346  8387 raft_consensus.cc:515] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.042779  8376 raft_consensus.cc:359] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.043690  8376 raft_consensus.cc:385] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.044025  8376 raft_consensus.cc:740] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.045025  8376 consensus_queue.cc:260] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.048322  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527" is_pre_election: true
I20260709 13:01:19.049127  8376 ts_tablet_manager.cc:1434] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:01:19.049099  8259 raft_consensus.cc:2468] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 0.
I20260709 13:01:19.051131  8296 leader_election.cc:304] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:19.052209  8386 raft_consensus.cc:2804] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:19.052644  8386 raft_consensus.cc:493] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.052999  8386 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.052999  8376 tablet_bootstrap.cc:492] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889: Bootstrap starting.
I20260709 13:01:19.058926  8387 leader_election.cc:290] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449)
I20260709 13:01:19.022169  8409 raft_consensus.cc:493] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.046361  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" is_pre_election: true
I20260709 13:01:19.061018  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527" is_pre_election: true
W20260709 13:01:19.061251  8297 leader_election.cc:343] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.062402  8409 raft_consensus.cc:3060] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:19.065208  8147 leader_election.cc:343] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.066380  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527" is_pre_election: true
W20260709 13:01:19.068271  8147 leader_election.cc:343] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.070266  8376 tablet_bootstrap.cc:654] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:19.071309  8373 tablet_bootstrap.cc:654] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:19.071178  8386 raft_consensus.cc:515] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.075058  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
W20260709 13:01:19.076853  8297 leader_election.cc:343] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.082003  8377 raft_consensus.cc:515] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.084836  8376 tablet_bootstrap.cc:492] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889: No bootstrap required, opened a new log
I20260709 13:01:19.085687  8376 ts_tablet_manager.cc:1403] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.009s
I20260709 13:01:19.085651  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d8cefd485c946b4a33d75aade5c3668" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
I20260709 13:01:19.086668  8184 raft_consensus.cc:3060] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.090385  8377 leader_election.cc:290] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.094231  8373 tablet_bootstrap.cc:492] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc: No bootstrap required, opened a new log
I20260709 13:01:19.094874  8373 ts_tablet_manager.cc:1403] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent bootstrapping tablet: real 0.058s	user 0.009s	sys 0.007s
I20260709 13:01:19.098722  8373 raft_consensus.cc:359] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.099524  8373 raft_consensus.cc:385] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.099840  8373 raft_consensus.cc:740] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.100785  8373 consensus_queue.cc:260] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.103523  8373 ts_tablet_manager.cc:1434] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:01:19.104883  8373 tablet_bootstrap.cc:492] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc: Bootstrap starting.
I20260709 13:01:19.106374  8333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d8cefd485c946b4a33d75aade5c3668" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:19.107110  8333 raft_consensus.cc:3060] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.110525  8383 raft_consensus.cc:493] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.111379  8383 raft_consensus.cc:515] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.111975  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889" is_pre_election: true
I20260709 13:01:19.112753  8373 tablet_bootstrap.cc:654] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:19.114138  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889" is_pre_election: true
I20260709 13:01:19.116266  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a2b7f4f90441f78ede444a23726b3b" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527" is_pre_election: true
I20260709 13:01:19.118000  8376 raft_consensus.cc:359] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.117359  8259 raft_consensus.cc:2468] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 0.
I20260709 13:01:19.119668  8376 raft_consensus.cc:385] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:01:19.118705  8149 leader_election.cc:343] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.121913  8376 raft_consensus.cc:740] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.122054  8149 leader_election.cc:304] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc; no voters: 6ed78a204d0b4fc5b898283bef2b4889, 78dc985924c447f08ef73e4638a9d527
I20260709 13:01:19.121059  8147 leader_election.cc:304] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:19.082250  8409 raft_consensus.cc:515] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.132663  8376 consensus_queue.cc:260] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.136543  8332 raft_consensus.cc:2468] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 0.
I20260709 13:01:19.139951  8376 ts_tablet_manager.cc:1434] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889: Time spent starting tablet: real 0.041s	user 0.009s	sys 0.001s
I20260709 13:01:19.140378  8409 leader_election.cc:290] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.098585  8184 raft_consensus.cc:2468] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 1.
I20260709 13:01:19.141124  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6ce814b173348d8ad6fbcbb7f0ceae8" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:19.143926  8223 leader_election.cc:304] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:19.140888  8183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6ce814b173348d8ad6fbcbb7f0ceae8" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
I20260709 13:01:19.144820  8183 raft_consensus.cc:3060] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.145074  8409 raft_consensus.cc:2804] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:19.146118  8386 leader_election.cc:290] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449)
I20260709 13:01:19.145717  8409 raft_consensus.cc:697] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Becoming Leader. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Running, Role: LEADER
I20260709 13:01:19.147753  8409 consensus_queue.cc:237] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.148311  8383 leader_election.cc:290] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.148909  8383 raft_consensus.cc:2749] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:19.149546  8383 raft_consensus.cc:2804] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:19.149893  8383 raft_consensus.cc:493] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.150219  8383 raft_consensus.cc:3060] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.142784  8331 raft_consensus.cc:3060] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.152674  8333 raft_consensus.cc:2468] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 1.
I20260709 13:01:19.155961  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:19.156715  8259 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.165753  8259 raft_consensus.cc:2468] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 1.
I20260709 13:01:19.167237  8296 leader_election.cc:304] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889, 78dc985924c447f08ef73e4638a9d527; no voters: 60db2e9279404d9dab6b5c19f8d7cafc
I20260709 13:01:19.167615  8376 tablet_bootstrap.cc:492] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889: Bootstrap starting.
I20260709 13:01:19.172044  8331 raft_consensus.cc:2468] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 1.
I20260709 13:01:19.174140  8183 raft_consensus.cc:2468] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 1.
I20260709 13:01:19.175081  8224 leader_election.cc:304] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:19.175885  8377 raft_consensus.cc:2804] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:19.176370  8377 raft_consensus.cc:697] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Becoming Leader. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Running, Role: LEADER
I20260709 13:01:19.177269  8377 consensus_queue.cc:237] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.179189  8376 tablet_bootstrap.cc:654] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:19.182168  8383 raft_consensus.cc:515] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.183787  8081 catalog_manager.cc:5697] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78dc985924c447f08ef73e4638a9d527 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "78dc985924c447f08ef73e4638a9d527" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.184888  8373 tablet_bootstrap.cc:492] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc: No bootstrap required, opened a new log
I20260709 13:01:19.186409  8373 ts_tablet_manager.cc:1403] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent bootstrapping tablet: real 0.082s	user 0.020s	sys 0.009s
I20260709 13:01:19.189539  8383 leader_election.cc:290] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 election: Requested vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
W20260709 13:01:19.170564  8149 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37523, user_credentials={real_user=slave}} blocked reactor thread for 52131.7us
I20260709 13:01:19.170915  8386 raft_consensus.cc:2804] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:19.191220  8386 raft_consensus.cc:697] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 LEADER]: Becoming Leader. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Running, Role: LEADER
I20260709 13:01:19.192271  8386 consensus_queue.cc:237] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.192508  8149 leader_election.cc:304] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889; no voters: 78dc985924c447f08ef73e4638a9d527
I20260709 13:01:19.193464  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a2b7f4f90441f78ede444a23726b3b" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:19.194124  8332 raft_consensus.cc:3060] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.171878  8374 tablet_bootstrap.cc:492] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527: No bootstrap required, opened a new log
I20260709 13:01:19.194669  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a2b7f4f90441f78ede444a23726b3b" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:19.195047  8374 ts_tablet_manager.cc:1403] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527: Time spent bootstrapping tablet: real 0.201s	user 0.043s	sys 0.057s
I20260709 13:01:19.197500  8409 raft_consensus.cc:493] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.198298  8409 raft_consensus.cc:515] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.199612  8374 raft_consensus.cc:359] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.202406  8374 raft_consensus.cc:385] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.197721  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a2b7f4f90441f78ede444a23726b3b" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889" is_pre_election: true
I20260709 13:01:19.202571  8383 raft_consensus.cc:2804] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:19.203711  8333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99c6d9bc60d4dadb91d2ac9a4a09891" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889" is_pre_election: true
I20260709 13:01:19.203868  8383 raft_consensus.cc:493] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:19.205497  8224 leader_election.cc:343] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.205739  8383 raft_consensus.cc:3060] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.209654  8332 raft_consensus.cc:2468] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 1.
I20260709 13:01:19.202943  8374 raft_consensus.cc:740] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.211017  8183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99c6d9bc60d4dadb91d2ac9a4a09891" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" is_pre_election: true
I20260709 13:01:19.212170  8374 consensus_queue.cc:260] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.195546  8259 raft_consensus.cc:3060] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:19.212703  8223 leader_election.cc:343] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.213678  8223 leader_election.cc:304] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78dc985924c447f08ef73e4638a9d527; no voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889
I20260709 13:01:19.201503  8409 leader_election.cc:290] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.217409  8383 raft_consensus.cc:515] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.218505  8373 raft_consensus.cc:359] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.219300  8409 raft_consensus.cc:2749] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:19.220927  8376 tablet_bootstrap.cc:492] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889: No bootstrap required, opened a new log
I20260709 13:01:19.220927  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:19.221633  8376 ts_tablet_manager.cc:1403] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889: Time spent bootstrapping tablet: real 0.080s	user 0.017s	sys 0.023s
I20260709 13:01:19.221805  8149 leader_election.cc:304] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889; no voters: 
W20260709 13:01:19.222844  8147 leader_election.cc:343] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.219262  8373 raft_consensus.cc:385] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.224291  8373 raft_consensus.cc:740] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.225068  8373 consensus_queue.cc:260] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.226536  8374 ts_tablet_manager.cc:1434] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.002s
I20260709 13:01:19.225962  8376 raft_consensus.cc:359] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.227090  8376 raft_consensus.cc:385] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.227484  8376 raft_consensus.cc:740] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.227732  8374 tablet_bootstrap.cc:492] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527: Bootstrap starting.
I20260709 13:01:19.228410  8376 consensus_queue.cc:260] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.229172  8383 leader_election.cc:290] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 election: Requested vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.230015  8414 raft_consensus.cc:2804] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:19.230576  8414 raft_consensus.cc:697] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Becoming Leader. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:19.231861  8376 ts_tablet_manager.cc:1434] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:01:19.227761  8373 ts_tablet_manager.cc:1434] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 13:01:19.231632  8414 consensus_queue.cc:237] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.233579  8376 tablet_bootstrap.cc:492] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889: Bootstrap starting.
I20260709 13:01:19.237555  8374 tablet_bootstrap.cc:654] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:19.233574  8373 tablet_bootstrap.cc:492] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc: Bootstrap starting.
I20260709 13:01:19.239944  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:19.240658  8332 raft_consensus.cc:3060] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.240175  8081 catalog_manager.cc:5697] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "6ed78a204d0b4fc5b898283bef2b4889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:19.248435  8080 catalog_manager.cc:5697] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78dc985924c447f08ef73e4638a9d527 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "78dc985924c447f08ef73e4638a9d527" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.257866  8373 tablet_bootstrap.cc:654] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:19.259768  8376 tablet_bootstrap.cc:654] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:19.267333  8259 raft_consensus.cc:2468] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 1.
I20260709 13:01:19.276206  8332 raft_consensus.cc:2468] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 1.
I20260709 13:01:19.277760  8149 leader_election.cc:304] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889; no voters: 78dc985924c447f08ef73e4638a9d527
I20260709 13:01:19.279132  8383 raft_consensus.cc:2804] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:19.279691  8383 raft_consensus.cc:697] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Becoming Leader. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:19.280669  8383 consensus_queue.cc:237] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.287048  8374 tablet_bootstrap.cc:492] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527: No bootstrap required, opened a new log
I20260709 13:01:19.287812  8376 tablet_bootstrap.cc:492] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889: No bootstrap required, opened a new log
I20260709 13:01:19.287946  8374 ts_tablet_manager.cc:1403] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527: Time spent bootstrapping tablet: real 0.060s	user 0.024s	sys 0.003s
I20260709 13:01:19.288658  8376 ts_tablet_manager.cc:1403] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.006s
I20260709 13:01:19.289776  8415 raft_consensus.cc:493] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.287868  8081 catalog_manager.cc:5697] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc reported cstate change: term changed from 0 to 1, leader changed from <none> to 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.290359  8415 raft_consensus.cc:515] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.291380  8373 tablet_bootstrap.cc:492] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc: No bootstrap required, opened a new log
I20260709 13:01:19.291941  8373 ts_tablet_manager.cc:1403] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.000s
I20260709 13:01:19.291970  8376 raft_consensus.cc:359] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.292881  8376 raft_consensus.cc:385] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.293187  8376 raft_consensus.cc:740] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.293965  8376 consensus_queue.cc:260] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.295467  8333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7207cf9522d64ea49340bd7c2950431e" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889" is_pre_election: true
I20260709 13:01:19.296335  8376 ts_tablet_manager.cc:1434] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:01:19.296303  8333 raft_consensus.cc:2468] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 0.
I20260709 13:01:19.297832  8376 tablet_bootstrap.cc:492] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889: Bootstrap starting.
I20260709 13:01:19.298153  8149 leader_election.cc:304] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889; no voters: 
I20260709 13:01:19.292897  8415 leader_election.cc:290] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.299667  8415 raft_consensus.cc:2804] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:19.300038  8415 raft_consensus.cc:493] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.300407  8415 raft_consensus.cc:3060] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.295537  8374 raft_consensus.cc:359] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.301266  8373 raft_consensus.cc:359] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.302865  8374 raft_consensus.cc:385] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.302992  8373 raft_consensus.cc:385] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.303236  8374 raft_consensus.cc:740] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.303421  8373 raft_consensus.cc:740] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.304098  8374 consensus_queue.cc:260] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.306376  8374 ts_tablet_manager.cc:1434] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.001s
I20260709 13:01:19.307529  8374 tablet_bootstrap.cc:492] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527: Bootstrap starting.
I20260709 13:01:19.311759  8376 tablet_bootstrap.cc:654] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:19.314653  8374 tablet_bootstrap.cc:654] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:19.323766  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7207cf9522d64ea49340bd7c2950431e" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527" is_pre_election: true
W20260709 13:01:19.326138  8147 leader_election.cc:343] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.327363  8374 tablet_bootstrap.cc:492] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527: No bootstrap required, opened a new log
I20260709 13:01:19.327955  8374 ts_tablet_manager.cc:1403] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.009s
I20260709 13:01:19.329877  8079 catalog_manager.cc:5697] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc reported cstate change: term changed from 0 to 1, leader changed from <none> to 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.331784  8374 raft_consensus.cc:359] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.332579  8374 raft_consensus.cc:385] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.332881  8374 raft_consensus.cc:740] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.304255  8373 consensus_queue.cc:260] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.334383  8374 consensus_queue.cc:260] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.336539  8373 ts_tablet_manager.cc:1434] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.001s
I20260709 13:01:19.340166  8374 ts_tablet_manager.cc:1434] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.006s
I20260709 13:01:19.342761  8374 tablet_bootstrap.cc:492] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527: Bootstrap starting.
I20260709 13:01:19.343540  8386 consensus_queue.cc:1048] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:19.350219  8374 tablet_bootstrap.cc:654] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:19.358582  8415 raft_consensus.cc:515] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.362212  8377 raft_consensus.cc:493] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.362874  8377 raft_consensus.cc:515] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.365705  8377 leader_election.cc:290] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.367004  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" is_pre_election: true
I20260709 13:01:19.367187  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7207cf9522d64ea49340bd7c2950431e" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:19.367841  8184 raft_consensus.cc:2468] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 0.
W20260709 13:01:19.368818  8147 leader_election.cc:343] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.369158  8374 tablet_bootstrap.cc:492] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527: No bootstrap required, opened a new log
I20260709 13:01:19.369508  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889" is_pre_election: true
I20260709 13:01:19.370101  8374 ts_tablet_manager.cc:1403] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.003s
I20260709 13:01:19.371356  8415 leader_election.cc:290] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 election: Requested vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.371282  8223 leader_election.cc:304] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:19.372445  8377 raft_consensus.cc:2804] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:19.372874  8377 raft_consensus.cc:493] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.373339  8377 raft_consensus.cc:3060] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.374361  8374 raft_consensus.cc:359] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.375290  8374 raft_consensus.cc:385] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.375803  8374 raft_consensus.cc:740] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Initialized, Role: FOLLOWER
W20260709 13:01:19.375212  8224 leader_election.cc:343] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.376047  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7207cf9522d64ea49340bd7c2950431e" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:19.376745  8332 raft_consensus.cc:3060] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.376871  8374 consensus_queue.cc:260] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.380923  8374 ts_tablet_manager.cc:1434] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:01:19.389942  8395 consensus_queue.cc:1048] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:19.394104  8332 raft_consensus.cc:2468] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 1.
I20260709 13:01:19.394997  8377 raft_consensus.cc:515] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.397665  8149 leader_election.cc:304] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889; no voters: 78dc985924c447f08ef73e4638a9d527
I20260709 13:01:19.398795  8377 leader_election.cc:290] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.399202  8414 raft_consensus.cc:2804] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:19.399698  8414 raft_consensus.cc:697] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Becoming Leader. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:19.400362  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:19.400609  8414 consensus_queue.cc:237] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
W20260709 13:01:19.403587  8224 leader_election.cc:343] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.403777  8376 tablet_bootstrap.cc:492] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889: No bootstrap required, opened a new log
I20260709 13:01:19.404392  8376 ts_tablet_manager.cc:1403] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889: Time spent bootstrapping tablet: real 0.107s	user 0.035s	sys 0.025s
I20260709 13:01:19.408105  8183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
I20260709 13:01:19.409202  8183 raft_consensus.cc:3060] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.411968  8376 raft_consensus.cc:359] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.412642  8376 raft_consensus.cc:385] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.412923  8376 raft_consensus.cc:740] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.413650  8376 consensus_queue.cc:260] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.415380  8377 raft_consensus.cc:493] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.416188  8377 raft_consensus.cc:515] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.420010  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "456f553f05b141b2b43d3d710034fb7f" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" is_pre_election: true
I20260709 13:01:19.420816  8377 leader_election.cc:290] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.420842  8184 raft_consensus.cc:2468] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 0.
I20260709 13:01:19.422878  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "456f553f05b141b2b43d3d710034fb7f" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889" is_pre_election: true
I20260709 13:01:19.423832  8223 leader_election.cc:304] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:19.424657  8377 raft_consensus.cc:2804] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:01:19.425323  8224 leader_election.cc:343] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.425256  8377 raft_consensus.cc:493] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.425946  8377 raft_consensus.cc:3060] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.431031  8183 raft_consensus.cc:2468] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 1.
I20260709 13:01:19.433018  8223 leader_election.cc:304] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527; no voters: 6ed78a204d0b4fc5b898283bef2b4889
I20260709 13:01:19.434985  8409 raft_consensus.cc:2804] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:19.435709  8409 raft_consensus.cc:697] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Becoming Leader. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Running, Role: LEADER
I20260709 13:01:19.435949  8377 raft_consensus.cc:515] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.437395  8376 ts_tablet_manager.cc:1434] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:01:19.437101  8409 consensus_queue.cc:237] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.439414  8377 leader_election.cc:290] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.440965  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "456f553f05b141b2b43d3d710034fb7f" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:19.441016  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "456f553f05b141b2b43d3d710034fb7f" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
I20260709 13:01:19.442088  8184 raft_consensus.cc:3060] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:19.443137  8224 leader_election.cc:343] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:19.445027  8376 tablet_bootstrap.cc:492] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889: Bootstrap starting.
I20260709 13:01:19.456538  8081 catalog_manager.cc:5697] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc reported cstate change: term changed from 0 to 1, leader changed from <none> to 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.459563  8376 tablet_bootstrap.cc:654] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:19.466220  8184 raft_consensus.cc:2468] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 1.
I20260709 13:01:19.466526  8376 tablet_bootstrap.cc:492] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889: No bootstrap required, opened a new log
I20260709 13:01:19.467113  8376 ts_tablet_manager.cc:1403] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.001s
I20260709 13:01:19.468108  8409 raft_consensus.cc:493] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.468664  8223 leader_election.cc:304] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527; no voters: 6ed78a204d0b4fc5b898283bef2b4889
I20260709 13:01:19.468640  8409 raft_consensus.cc:515] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.472131  8377 raft_consensus.cc:2804] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:19.472224  8376 raft_consensus.cc:359] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.472815  8377 raft_consensus.cc:697] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Becoming Leader. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Running, Role: LEADER
I20260709 13:01:19.473039  8376 raft_consensus.cc:385] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:19.473379  8376 raft_consensus.cc:740] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Initialized, Role: FOLLOWER
I20260709 13:01:19.473875  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7207cf9522d64ea49340bd7c2950431e" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" is_pre_election: true
I20260709 13:01:19.473804  8377 consensus_queue.cc:237] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.474287  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7207cf9522d64ea49340bd7c2950431e" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889" is_pre_election: true
I20260709 13:01:19.476680  8332 raft_consensus.cc:2393] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 78dc985924c447f08ef73e4638a9d527 in current term 1: Already voted for candidate 60db2e9279404d9dab6b5c19f8d7cafc in this term.
I20260709 13:01:19.474073  8376 consensus_queue.cc:260] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.478113  8224 leader_election.cc:304] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78dc985924c447f08ef73e4638a9d527; no voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889
I20260709 13:01:19.478940  8376 ts_tablet_manager.cc:1434] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:01:19.479357  8410 raft_consensus.cc:3060] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.474792  8409 leader_election.cc:290] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.488515  8410 raft_consensus.cc:2749] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:19.491267  8080 catalog_manager.cc:5697] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78dc985924c447f08ef73e4638a9d527 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "78dc985924c447f08ef73e4638a9d527" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.492851  8080 catalog_manager.cc:5697] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78dc985924c447f08ef73e4638a9d527 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "78dc985924c447f08ef73e4638a9d527" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.574863  8410 consensus_queue.cc:1048] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:19.588583  8410 consensus_queue.cc:1048] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:19.606935  8410 consensus_queue.cc:1048] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:19.611778  8414 raft_consensus.cc:493] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.612340  8414 raft_consensus.cc:515] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.614500  8414 leader_election.cc:290] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.616441  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99c6d9bc60d4dadb91d2ac9a4a09891" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527" is_pre_election: true
I20260709 13:01:19.617245  8259 raft_consensus.cc:2468] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 0.
I20260709 13:01:19.619315  8147 leader_election.cc:304] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:19.620328  8414 raft_consensus.cc:2804] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:19.620749  8414 raft_consensus.cc:493] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.621131  8414 raft_consensus.cc:3060] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.625815  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99c6d9bc60d4dadb91d2ac9a4a09891" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889" is_pre_election: true
I20260709 13:01:19.626689  8332 raft_consensus.cc:2468] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 0.
I20260709 13:01:19.628466  8377 consensus_queue.cc:1048] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:19.630528  8414 raft_consensus.cc:515] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.633773  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99c6d9bc60d4dadb91d2ac9a4a09891" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:19.634599  8259 raft_consensus.cc:3060] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.635129  8414 leader_election.cc:290] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 election: Requested vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.634974  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99c6d9bc60d4dadb91d2ac9a4a09891" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:19.636582  8332 raft_consensus.cc:3060] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.643584  8386 consensus_queue.cc:1048] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:19.645453  8332 raft_consensus.cc:2468] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 1.
I20260709 13:01:19.646850  8149 leader_election.cc:304] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889; no voters: 
I20260709 13:01:19.646853  8259 raft_consensus.cc:2468] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 1.
I20260709 13:01:19.648273  8414 raft_consensus.cc:2804] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:19.648837  8414 raft_consensus.cc:697] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Becoming Leader. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:19.649757  8414 consensus_queue.cc:237] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.661756  8184 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.661957  8080 catalog_manager.cc:5697] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc reported cstate change: term changed from 0 to 1, leader changed from <none> to 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.675810  8426 consensus_queue.cc:1048] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:19.676090  8415 raft_consensus.cc:493] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.676875  8415 raft_consensus.cc:515] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.677381  8395 consensus_queue.cc:1048] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:19.681880  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527" is_pre_election: true
I20260709 13:01:19.684535  8333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889" is_pre_election: true
I20260709 13:01:19.686287  8149 leader_election.cc:304] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc; no voters: 6ed78a204d0b4fc5b898283bef2b4889, 78dc985924c447f08ef73e4638a9d527
I20260709 13:01:19.682698  8415 leader_election.cc:290] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.694312  8415 consensus_queue.cc:1048] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:19.698846  8426 raft_consensus.cc:2749] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:19.748466  8426 consensus_queue.cc:1048] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:19.759399  8258 raft_consensus.cc:3060] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.770958  8383 consensus_queue.cc:1048] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:19.798171  8258 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e6ce814b173348d8ad6fbcbb7f0ceae8"
dest_uuid: "78dc985924c447f08ef73e4638a9d527"
mode: GRACEFUL
new_leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:47206
I20260709 13:01:19.798965  8258 raft_consensus.cc:606] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Received request to transfer leadership to TS 60db2e9279404d9dab6b5c19f8d7cafc
I20260709 13:01:19.804994  8258 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b"
dest_uuid: "78dc985924c447f08ef73e4638a9d527"
mode: GRACEFUL
new_leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:47206
I20260709 13:01:19.805598  8258 raft_consensus.cc:606] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Received request to transfer leadership to TS 60db2e9279404d9dab6b5c19f8d7cafc
I20260709 13:01:19.821957  8332 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "919af25e2da94f199867009579567a66"
dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
mode: GRACEFUL
new_leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:45614
I20260709 13:01:19.822666  8332 raft_consensus.cc:606] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 LEADER]: Received request to transfer leadership to TS 60db2e9279404d9dab6b5c19f8d7cafc
I20260709 13:01:19.829272  8332 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6"
dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
mode: GRACEFUL
new_leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:45614
I20260709 13:01:19.829984  8332 raft_consensus.cc:606] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 LEADER]: Received request to transfer leadership to TS 60db2e9279404d9dab6b5c19f8d7cafc
I20260709 13:01:19.853077  8445 raft_consensus.cc:493] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.853680  8445 raft_consensus.cc:515] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.854029  8184 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
mode: GRACEFUL
new_leader_uuid: "78dc985924c447f08ef73e4638a9d527"
 from {username='slave'} at 127.0.0.1:44120
I20260709 13:01:19.854830  8184 raft_consensus.cc:606] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Received request to transfer leadership to TS 78dc985924c447f08ef73e4638a9d527
I20260709 13:01:19.856057  8445 leader_election.cc:290] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449)
I20260709 13:01:19.856914  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" is_pre_election: true
I20260709 13:01:19.857820  8184 raft_consensus.cc:2393] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ed78a204d0b4fc5b898283bef2b4889 in current term 1: Already voted for candidate 78dc985924c447f08ef73e4638a9d527 in this term.
I20260709 13:01:19.857817  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527" is_pre_election: true
I20260709 13:01:19.859740  8296 leader_election.cc:304] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889; no voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527
I20260709 13:01:19.860558  8445 raft_consensus.cc:3060] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.862164  8184 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7207cf9522d64ea49340bd7c2950431e"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
mode: GRACEFUL
new_leader_uuid: "78dc985924c447f08ef73e4638a9d527"
 from {username='slave'} at 127.0.0.1:44120
I20260709 13:01:19.862944  8184 raft_consensus.cc:606] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Received request to transfer leadership to TS 78dc985924c447f08ef73e4638a9d527
I20260709 13:01:19.862855  8383 consensus_queue.cc:1048] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:19.869657  8445 raft_consensus.cc:2749] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:19.876494  8415 consensus_queue.cc:1048] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:19.880589  8184 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f8a2b7f4f90441f78ede444a23726b3b"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
mode: GRACEFUL
new_leader_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
 from {username='slave'} at 127.0.0.1:44120
I20260709 13:01:19.881742  8184 raft_consensus.cc:606] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Received request to transfer leadership to TS 6ed78a204d0b4fc5b898283bef2b4889
I20260709 13:01:19.887897  8415 raft_consensus.cc:993] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc: : Instructing follower 78dc985924c447f08ef73e4638a9d527 to start an election
I20260709 13:01:19.888481  8383 raft_consensus.cc:1081] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Signalling peer 78dc985924c447f08ef73e4638a9d527 to start an election
I20260709 13:01:19.890246  8258 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7207cf9522d64ea49340bd7c2950431e"
dest_uuid: "78dc985924c447f08ef73e4638a9d527"
 from {username='slave'} at 127.0.0.1:47204
I20260709 13:01:19.890918  8258 raft_consensus.cc:493] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:19.891338  8258 raft_consensus.cc:3060] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.897548  8258 raft_consensus.cc:515] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.900023  8258 leader_election.cc:290] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 2 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.901074  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7207cf9522d64ea49340bd7c2950431e" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
I20260709 13:01:19.901270  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7207cf9522d64ea49340bd7c2950431e" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:19.901784  8184 raft_consensus.cc:3055] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:19.901903  8332 raft_consensus.cc:3060] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.902253  8184 raft_consensus.cc:740] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:19.903314  8184 consensus_queue.cc:260] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.904369  8184 raft_consensus.cc:3060] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.909487  8332 raft_consensus.cc:2468] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 2.
I20260709 13:01:19.910192  8184 raft_consensus.cc:2468] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 2.
I20260709 13:01:19.911072  8224 leader_election.cc:304] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:19.912026  8409 raft_consensus.cc:2804] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:19.912562  8409 raft_consensus.cc:697] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 2 LEADER]: Becoming Leader. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Running, Role: LEADER
I20260709 13:01:19.913270  8409 consensus_queue.cc:237] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.920284  8080 catalog_manager.cc:5697] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 reported cstate change: term changed from 1 to 2, leader changed from 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129) to 78dc985924c447f08ef73e4638a9d527 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "78dc985924c447f08ef73e4638a9d527" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.938141  8409 consensus_queue.cc:1048] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:19.946676  8377 raft_consensus.cc:993] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527: : Instructing follower 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:19.947326  8377 raft_consensus.cc:1081] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Signalling peer 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:19.949256  8183 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:44090
I20260709 13:01:19.950270  8183 raft_consensus.cc:493] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:19.950780  8183 raft_consensus.cc:3060] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.952390  8445 raft_consensus.cc:493] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.953204  8445 raft_consensus.cc:515] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.959494  8445 leader_election.cc:290] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449)
I20260709 13:01:19.960064  8183 raft_consensus.cc:515] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.960971  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "456f553f05b141b2b43d3d710034fb7f" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78dc985924c447f08ef73e4638a9d527" is_pre_election: true
I20260709 13:01:19.965222  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:19.965976  8259 raft_consensus.cc:3055] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:19.966058  8333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:19.966413  8259 raft_consensus.cc:740] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Running, Role: LEADER
I20260709 13:01:19.967041  8333 raft_consensus.cc:3060] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.967605  8259 consensus_queue.cc:260] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.969266  8183 leader_election.cc:290] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 2 election: Requested vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:19.962881  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "456f553f05b141b2b43d3d710034fb7f" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" is_pre_election: true
I20260709 13:01:19.971760  8184 raft_consensus.cc:2393] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ed78a204d0b4fc5b898283bef2b4889 in current term 1: Already voted for candidate 78dc985924c447f08ef73e4638a9d527 in this term.
I20260709 13:01:19.973081  8297 leader_election.cc:304] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889; no voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527
I20260709 13:01:19.974501  8445 raft_consensus.cc:3060] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:19.975848  8259 raft_consensus.cc:3060] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.977054  8377 consensus_queue.cc:1048] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:19.978498  8333 raft_consensus.cc:2468] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 2.
I20260709 13:01:19.979370  8332 raft_consensus.cc:1240] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Rejecting Update request from peer 78dc985924c447f08ef73e4638a9d527 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:19.984633  8149 leader_election.cc:304] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889; no voters: 
I20260709 13:01:19.986510  8383 raft_consensus.cc:2804] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:19.987097  8383 raft_consensus.cc:697] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Becoming Leader. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:19.988073  8259 raft_consensus.cc:2468] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 2.
I20260709 13:01:19.988023  8383 consensus_queue.cc:237] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:19.992687  8445 raft_consensus.cc:2749] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:19.998715  8459 consensus_queue.cc:1048] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:20.003906  8079 catalog_manager.cc:5697] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc reported cstate change: term changed from 1 to 2, leader changed from 78dc985924c447f08ef73e4638a9d527 (127.7.89.130) to 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129). New cstate: current_term: 2 leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:20.066586  8383 consensus_queue.cc:1048] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:20.077785  8415 consensus_queue.cc:1048] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:20.117022  8409 raft_consensus.cc:993] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527: : Instructing follower 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:20.117645  8459 raft_consensus.cc:1081] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Signalling peer 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:20.119459  8184 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e6ce814b173348d8ad6fbcbb7f0ceae8"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:44090
I20260709 13:01:20.120025  8184 raft_consensus.cc:493] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:20.120375  8184 raft_consensus.cc:3060] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.126060  8184 raft_consensus.cc:515] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.127959  8184 leader_election.cc:290] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 2 election: Requested vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:20.129073  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6ce814b173348d8ad6fbcbb7f0ceae8" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:20.129256  8333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6ce814b173348d8ad6fbcbb7f0ceae8" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:20.129781  8259 raft_consensus.cc:3055] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:20.129861  8333 raft_consensus.cc:3060] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.130164  8259 raft_consensus.cc:740] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Running, Role: LEADER
I20260709 13:01:20.131101  8259 consensus_queue.cc:260] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.133473  8259 raft_consensus.cc:3060] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.136724  8333 raft_consensus.cc:2468] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 2.
I20260709 13:01:20.138279  8149 leader_election.cc:304] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889; no voters: 
I20260709 13:01:20.139163  8415 raft_consensus.cc:2804] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:20.139626  8415 raft_consensus.cc:697] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Becoming Leader. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:20.140417  8415 consensus_queue.cc:237] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.140919  8259 raft_consensus.cc:2468] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 2.
I20260709 13:01:20.147981  8079 catalog_manager.cc:5697] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc reported cstate change: term changed from 1 to 2, leader changed from 78dc985924c447f08ef73e4638a9d527 (127.7.89.130) to 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129). New cstate: current_term: 2 leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:20.166121  8448 raft_consensus.cc:993] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc: : Instructing follower 6ed78a204d0b4fc5b898283bef2b4889 to start an election
I20260709 13:01:20.166721  8415 raft_consensus.cc:1081] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Signalling peer 6ed78a204d0b4fc5b898283bef2b4889 to start an election
I20260709 13:01:20.168473  8333 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f8a2b7f4f90441f78ede444a23726b3b"
dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
 from {username='slave'} at 127.0.0.1:45606
I20260709 13:01:20.168992  8333 raft_consensus.cc:493] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:20.169265  8333 raft_consensus.cc:3060] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.175014  8333 raft_consensus.cc:515] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.176951  8333 leader_election.cc:290] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 2 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449)
I20260709 13:01:20.178035  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a2b7f4f90441f78ede444a23726b3b" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
I20260709 13:01:20.178781  8184 raft_consensus.cc:3055] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:20.179184  8184 raft_consensus.cc:740] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:20.179015  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a2b7f4f90441f78ede444a23726b3b" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:20.179980  8259 raft_consensus.cc:3060] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.180186  8184 consensus_queue.cc:260] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.181612  8184 raft_consensus.cc:3060] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.186290  8259 raft_consensus.cc:2468] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 2.
I20260709 13:01:20.187642  8296 leader_election.cc:304] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:20.188513  8445 raft_consensus.cc:2804] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:20.189006  8445 raft_consensus.cc:697] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 LEADER]: Becoming Leader. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Running, Role: LEADER
I20260709 13:01:20.189527  8184 raft_consensus.cc:2468] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 2.
I20260709 13:01:20.189965  8445 consensus_queue.cc:237] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.197923  8079 catalog_manager.cc:5697] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 reported cstate change: term changed from 1 to 2, leader changed from 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129) to 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "6ed78a204d0b4fc5b898283bef2b4889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:20.229387  8448 raft_consensus.cc:993] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc: : Instructing follower 78dc985924c447f08ef73e4638a9d527 to start an election
I20260709 13:01:20.230067  8448 raft_consensus.cc:1081] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Signalling peer 78dc985924c447f08ef73e4638a9d527 to start an election
I20260709 13:01:20.232333  8259 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae"
dest_uuid: "78dc985924c447f08ef73e4638a9d527"
 from {username='slave'} at 127.0.0.1:47204
I20260709 13:01:20.233508  8259 raft_consensus.cc:493] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:20.234951  8259 raft_consensus.cc:3060] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.236770  8386 raft_consensus.cc:993] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889: : Instructing follower 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:20.237303  8386 raft_consensus.cc:1081] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 LEADER]: Signalling peer 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:20.240532  8183 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "919af25e2da94f199867009579567a66"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:44106
I20260709 13:01:20.241308  8183 raft_consensus.cc:493] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:20.241777  8183 raft_consensus.cc:3060] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.245699  8395 raft_consensus.cc:993] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889: : Instructing follower 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:20.246289  8395 raft_consensus.cc:1081] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 LEADER]: Signalling peer 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:20.248519  8184 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:44106
I20260709 13:01:20.249199  8184 raft_consensus.cc:493] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:20.249080  8259 raft_consensus.cc:515] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.249686  8184 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.250687  8183 raft_consensus.cc:515] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.251827  8259 leader_election.cc:290] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 2 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:20.252988  8182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
I20260709 13:01:20.253010  8333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:20.253741  8182 raft_consensus.cc:3055] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:20.254127  8182 raft_consensus.cc:740] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:20.254317  8183 leader_election.cc:290] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 2 election: Requested vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:20.255374  8259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919af25e2da94f199867009579567a66" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:20.255017  8182 consensus_queue.cc:260] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.256067  8259 raft_consensus.cc:3060] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.253758  8333 raft_consensus.cc:3060] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.257130  8182 raft_consensus.cc:3060] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.259999  8184 raft_consensus.cc:515] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.256035  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "919af25e2da94f199867009579567a66" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:20.262400  8332 raft_consensus.cc:3055] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:20.263259  8332 raft_consensus.cc:740] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Running, Role: LEADER
I20260709 13:01:20.265343  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:20.264256  8332 consensus_queue.cc:260] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.266077  8258 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.266111  8182 raft_consensus.cc:2468] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 2.
I20260709 13:01:20.267156  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:20.267812  8331 raft_consensus.cc:3055] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:20.267867  8223 leader_election.cc:304] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:20.268258  8331 raft_consensus.cc:740] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Running, Role: LEADER
I20260709 13:01:20.269311  8409 raft_consensus.cc:2804] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:20.269971  8409 raft_consensus.cc:697] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 2 LEADER]: Becoming Leader. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Running, Role: LEADER
I20260709 13:01:20.269217  8331 consensus_queue.cc:260] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.270992  8409 consensus_queue.cc:237] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.271750  8331 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.267372  8332 raft_consensus.cc:3060] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.274720  8184 leader_election.cc:290] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 2 election: Requested vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:20.276647  8333 raft_consensus.cc:2468] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 2.
I20260709 13:01:20.280251  8258 raft_consensus.cc:2468] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 2.
I20260709 13:01:20.266100  8259 raft_consensus.cc:2468] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 2.
I20260709 13:01:20.281230  8332 raft_consensus.cc:2468] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 2.
I20260709 13:01:20.283179  8147 leader_election.cc:304] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:20.283299  8331 raft_consensus.cc:2468] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 2.
I20260709 13:01:20.282773  8080 catalog_manager.cc:5697] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 reported cstate change: term changed from 1 to 2, leader changed from 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129) to 78dc985924c447f08ef73e4638a9d527 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "78dc985924c447f08ef73e4638a9d527" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:20.283933  8149 leader_election.cc:304] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889; no voters: 
I20260709 13:01:20.284580  8415 raft_consensus.cc:2804] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:20.284925  8448 raft_consensus.cc:2804] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:20.285082  8415 raft_consensus.cc:697] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Becoming Leader. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:20.285470  8448 raft_consensus.cc:697] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Becoming Leader. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:20.286077  8415 consensus_queue.cc:237] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.286404  8448 consensus_queue.cc:237] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:20.295233  8079 catalog_manager.cc:5697] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc reported cstate change: term changed from 1 to 2, leader changed from 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131) to 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129). New cstate: current_term: 2 leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:20.310501  8079 catalog_manager.cc:5697] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc reported cstate change: term changed from 1 to 2, leader changed from 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131) to 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129). New cstate: current_term: 2 leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:20.369096  8331 raft_consensus.cc:1275] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Refusing update from remote peer 78dc985924c447f08ef73e4638a9d527: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.370838  8409 consensus_queue.cc:1048] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:20.380492  8184 raft_consensus.cc:1275] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Refusing update from remote peer 78dc985924c447f08ef73e4638a9d527: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.383052  8459 consensus_queue.cc:1048] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:20.497066  8331 raft_consensus.cc:1275] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.498662  8448 consensus_queue.cc:1048] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:20.510440  8259 raft_consensus.cc:1275] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.512311  8448 consensus_queue.cc:1048] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:20.654237  8331 raft_consensus.cc:1275] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Refusing update from remote peer 78dc985924c447f08ef73e4638a9d527: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.656082  8409 consensus_queue.cc:1048] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:20.660923  8332 raft_consensus.cc:1275] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.662657  8448 consensus_queue.cc:1048] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:20.666361  8184 raft_consensus.cc:1275] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Refusing update from remote peer 78dc985924c447f08ef73e4638a9d527: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.668313  8410 consensus_queue.cc:1048] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:20.680052  8259 raft_consensus.cc:1275] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.682015  8415 consensus_queue.cc:1048] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:20.705268  8259 raft_consensus.cc:1275] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.706753  8415 consensus_queue.cc:1048] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:20.715062  8259 raft_consensus.cc:1275] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Refusing update from remote peer 6ed78a204d0b4fc5b898283bef2b4889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.717840  8464 consensus_queue.cc:1048] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:20.720705  8331 raft_consensus.cc:1275] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.723658  8415 consensus_queue.cc:1048] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:20.734303  8184 raft_consensus.cc:1275] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Refusing update from remote peer 6ed78a204d0b4fc5b898283bef2b4889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.736145  8464 consensus_queue.cc:1048] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:20.766845  8331 raft_consensus.cc:1275] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.768414  8415 consensus_queue.cc:1048] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:20.781164  8258 raft_consensus.cc:1275] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.782944  8415 consensus_queue.cc:1048] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:22.922096  7526 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:01:22.923733  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:22.950443  8184 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
mode: GRACEFUL
new_leader_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
 from {username='slave'} at 127.0.0.1:44134
I20260709 13:01:22.951284  8184 raft_consensus.cc:606] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Received request to transfer leadership to TS 6ed78a204d0b4fc5b898283bef2b4889
I20260709 13:01:22.954301  8184 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b99c6d9bc60d4dadb91d2ac9a4a09891"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
mode: GRACEFUL
new_leader_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
 from {username='slave'} at 127.0.0.1:44134
I20260709 13:01:22.954938  8184 raft_consensus.cc:606] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Received request to transfer leadership to TS 6ed78a204d0b4fc5b898283bef2b4889
I20260709 13:01:22.956248  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:01:22.956789  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:23.219543  8484 raft_consensus.cc:993] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc: : Instructing follower 6ed78a204d0b4fc5b898283bef2b4889 to start an election
I20260709 13:01:23.220060  8484 raft_consensus.cc:1081] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Signalling peer 6ed78a204d0b4fc5b898283bef2b4889 to start an election
I20260709 13:01:23.222102  8331 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b99c6d9bc60d4dadb91d2ac9a4a09891"
dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
 from {username='slave'} at 127.0.0.1:45606
I20260709 13:01:23.222853  8331 raft_consensus.cc:493] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:23.223261  8331 raft_consensus.cc:3060] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.231334  8331 raft_consensus.cc:515] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:23.233520  8331 leader_election.cc:290] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 2 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449)
I20260709 13:01:23.234404  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99c6d9bc60d4dadb91d2ac9a4a09891" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
I20260709 13:01:23.234956  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99c6d9bc60d4dadb91d2ac9a4a09891" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:23.235360  8184 raft_consensus.cc:3055] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:23.235553  8258 raft_consensus.cc:3060] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.235734  8184 raft_consensus.cc:740] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:23.236799  8184 consensus_queue.cc:260] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:23.238266  8184 raft_consensus.cc:3060] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.244709  8258 raft_consensus.cc:2468] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 2.
I20260709 13:01:23.246526  8296 leader_election.cc:304] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:23.246868  8184 raft_consensus.cc:2468] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 2.
I20260709 13:01:23.247907  8479 raft_consensus.cc:2804] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:23.249046  8479 raft_consensus.cc:697] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 LEADER]: Becoming Leader. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Running, Role: LEADER
I20260709 13:01:23.250212  8479 consensus_queue.cc:237] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:23.265093  8079 catalog_manager.cc:5697] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 reported cstate change: term changed from 1 to 2, leader changed from 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129) to 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "6ed78a204d0b4fc5b898283bef2b4889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:23.351037  8480 raft_consensus.cc:993] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc: : Instructing follower 6ed78a204d0b4fc5b898283bef2b4889 to start an election
I20260709 13:01:23.351642  8484 raft_consensus.cc:1081] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Signalling peer 6ed78a204d0b4fc5b898283bef2b4889 to start an election
I20260709 13:01:23.353427  8331 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6"
dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
 from {username='slave'} at 127.0.0.1:45606
I20260709 13:01:23.354041  8331 raft_consensus.cc:493] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:23.354362  8331 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.360392  8331 raft_consensus.cc:515] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:23.362242  8331 leader_election.cc:290] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 3 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449)
I20260709 13:01:23.363521  8184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
I20260709 13:01:23.363610  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:23.364090  8184 raft_consensus.cc:3055] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:23.364270  8258 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.364387  8184 raft_consensus.cc:740] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:23.365321  8184 consensus_queue.cc:260] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:23.366400  8184 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.370366  8258 raft_consensus.cc:2468] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 3.
I20260709 13:01:23.371317  8184 raft_consensus.cc:2468] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 3.
I20260709 13:01:23.371948  8296 leader_election.cc:304] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:23.373152  8479 raft_consensus.cc:2804] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:23.373621  8479 raft_consensus.cc:697] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 LEADER]: Becoming Leader. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Running, Role: LEADER
I20260709 13:01:23.374574  8479 consensus_queue.cc:237] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:23.383484  8079 catalog_manager.cc:5697] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 reported cstate change: term changed from 2 to 3, leader changed from 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129) to 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "6ed78a204d0b4fc5b898283bef2b4889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:23.643290  8258 raft_consensus.cc:1275] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Refusing update from remote peer 6ed78a204d0b4fc5b898283bef2b4889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:23.644878  8479 consensus_queue.cc:1048] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:23.656555  8184 raft_consensus.cc:1275] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Refusing update from remote peer 6ed78a204d0b4fc5b898283bef2b4889: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:23.658263  8479 consensus_queue.cc:1048] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:23.916610  8258 raft_consensus.cc:1275] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Refusing update from remote peer 6ed78a204d0b4fc5b898283bef2b4889: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:23.918255  8479 consensus_queue.cc:1048] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:23.930176  8182 raft_consensus.cc:1275] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Refusing update from remote peer 6ed78a204d0b4fc5b898283bef2b4889: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:23.932085  8498 consensus_queue.cc:1048] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:23.957828  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:23.980458  8332 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6"
dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
mode: GRACEFUL
new_leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:45614
I20260709 13:01:23.981346  8332 raft_consensus.cc:606] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 LEADER]: Received request to transfer leadership to TS 60db2e9279404d9dab6b5c19f8d7cafc
I20260709 13:01:23.987243  8331 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b99c6d9bc60d4dadb91d2ac9a4a09891"
dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
mode: GRACEFUL
new_leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:45614
I20260709 13:01:23.987881  8331 raft_consensus.cc:606] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 LEADER]: Received request to transfer leadership to TS 60db2e9279404d9dab6b5c19f8d7cafc
I20260709 13:01:23.993556  8258 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d8cefd485c946b4a33d75aade5c3668"
dest_uuid: "78dc985924c447f08ef73e4638a9d527"
mode: GRACEFUL
new_leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:47206
I20260709 13:01:23.994150  8258 raft_consensus.cc:606] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Received request to transfer leadership to TS 60db2e9279404d9dab6b5c19f8d7cafc
I20260709 13:01:23.999946  8258 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae"
dest_uuid: "78dc985924c447f08ef73e4638a9d527"
mode: GRACEFUL
new_leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:47206
I20260709 13:01:24.000505  8258 raft_consensus.cc:606] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 2 LEADER]: Received request to transfer leadership to TS 60db2e9279404d9dab6b5c19f8d7cafc
I20260709 13:01:24.005716  8258 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7207cf9522d64ea49340bd7c2950431e"
dest_uuid: "78dc985924c447f08ef73e4638a9d527"
mode: GRACEFUL
new_leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:47206
I20260709 13:01:24.006268  8258 raft_consensus.cc:606] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 2 LEADER]: Received request to transfer leadership to TS 60db2e9279404d9dab6b5c19f8d7cafc
I20260709 13:01:24.110823  8459 raft_consensus.cc:993] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527: : Instructing follower 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:24.111460  8500 raft_consensus.cc:1081] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Signalling peer 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:24.113265  8182 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d8cefd485c946b4a33d75aade5c3668"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:44090
I20260709 13:01:24.113842  8182 raft_consensus.cc:493] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:24.114193  8182 raft_consensus.cc:3060] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:24.120551  8182 raft_consensus.cc:515] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.122764  8182 leader_election.cc:290] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 2 election: Requested vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:24.123884  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d8cefd485c946b4a33d75aade5c3668" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:24.123999  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d8cefd485c946b4a33d75aade5c3668" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:24.124640  8258 raft_consensus.cc:3055] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:24.124665  8331 raft_consensus.cc:3060] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:24.124994  8258 raft_consensus.cc:740] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Running, Role: LEADER
I20260709 13:01:24.126024  8258 consensus_queue.cc:260] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.127175  8258 raft_consensus.cc:3060] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:24.131371  8331 raft_consensus.cc:2468] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 2.
I20260709 13:01:24.133354  8149 leader_election.cc:304] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889; no voters: 
I20260709 13:01:24.133446  8258 raft_consensus.cc:2468] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 2.
I20260709 13:01:24.134419  8484 raft_consensus.cc:2804] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:24.135972  8484 raft_consensus.cc:697] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Becoming Leader. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:24.136981  8484 consensus_queue.cc:237] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.145393  8080 catalog_manager.cc:5697] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc reported cstate change: term changed from 1 to 2, leader changed from 78dc985924c447f08ef73e4638a9d527 (127.7.89.130) to 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129). New cstate: current_term: 2 leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:24.232046  8459 raft_consensus.cc:993] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527: : Instructing follower 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:24.232698  8500 raft_consensus.cc:1081] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 2 LEADER]: Signalling peer 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:24.234742  8182 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:44090
I20260709 13:01:24.235540  8182 raft_consensus.cc:493] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:24.235976  8182 raft_consensus.cc:3060] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.241851  8182 raft_consensus.cc:515] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.243953  8182 leader_election.cc:290] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 3 election: Requested vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:24.244874  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:24.245225  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:24.245560  8258 raft_consensus.cc:3055] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:24.245810  8331 raft_consensus.cc:3060] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.245990  8258 raft_consensus.cc:740] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Running, Role: LEADER
I20260709 13:01:24.246943  8258 consensus_queue.cc:260] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.248366  8258 raft_consensus.cc:3060] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.252772  8331 raft_consensus.cc:2468] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 3.
I20260709 13:01:24.253913  8149 leader_election.cc:304] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 6ed78a204d0b4fc5b898283bef2b4889; no voters: 
I20260709 13:01:24.254725  8484 raft_consensus.cc:2804] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:24.254648  8258 raft_consensus.cc:2468] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 3.
I20260709 13:01:24.255343  8484 raft_consensus.cc:697] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 LEADER]: Becoming Leader. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:24.256321  8484 consensus_queue.cc:237] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.263767  8080 catalog_manager.cc:5697] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc reported cstate change: term changed from 2 to 3, leader changed from 78dc985924c447f08ef73e4638a9d527 (127.7.89.130) to 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129). New cstate: current_term: 3 leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:24.276245  8479 raft_consensus.cc:993] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889: : Instructing follower 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:24.276767  8498 raft_consensus.cc:1081] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 LEADER]: Signalling peer 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:24.278761  8182 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b99c6d9bc60d4dadb91d2ac9a4a09891"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:44106
I20260709 13:01:24.279489  8182 raft_consensus.cc:493] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:24.279933  8182 raft_consensus.cc:3060] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.286902  8182 raft_consensus.cc:515] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.288914  8182 leader_election.cc:290] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 3 election: Requested vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:24.289902  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99c6d9bc60d4dadb91d2ac9a4a09891" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:24.290086  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99c6d9bc60d4dadb91d2ac9a4a09891" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:24.290526  8258 raft_consensus.cc:3060] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.290783  8331 raft_consensus.cc:3055] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:24.291185  8331 raft_consensus.cc:740] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Running, Role: LEADER
I20260709 13:01:24.291958  8331 consensus_queue.cc:260] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.293022  8331 raft_consensus.cc:3060] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.296849  8258 raft_consensus.cc:2468] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 3.
I20260709 13:01:24.298353  8147 leader_election.cc:304] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:24.299427  8484 raft_consensus.cc:2804] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:24.300037  8484 raft_consensus.cc:697] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 LEADER]: Becoming Leader. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:24.300076  8331 raft_consensus.cc:2468] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 3.
I20260709 13:01:24.301299  8484 consensus_queue.cc:237] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.309072  8080 catalog_manager.cc:5697] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc reported cstate change: term changed from 2 to 3, leader changed from 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131) to 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129). New cstate: current_term: 3 leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:24.318871  8459 raft_consensus.cc:993] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527: : Instructing follower 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:24.319521  8500 raft_consensus.cc:1081] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 2 LEADER]: Signalling peer 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:24.321195  8182 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7207cf9522d64ea49340bd7c2950431e"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:44090
I20260709 13:01:24.321780  8182 raft_consensus.cc:493] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:24.322149  8182 raft_consensus.cc:3060] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.329226  8182 raft_consensus.cc:515] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.331507  8182 leader_election.cc:290] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 3 election: Requested vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:24.332312  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7207cf9522d64ea49340bd7c2950431e" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:24.332670  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7207cf9522d64ea49340bd7c2950431e" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:24.332965  8258 raft_consensus.cc:3055] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:24.333227  8331 raft_consensus.cc:3060] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.333336  8258 raft_consensus.cc:740] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Running, Role: LEADER
I20260709 13:01:24.334182  8258 consensus_queue.cc:260] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.335429  8258 raft_consensus.cc:3060] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.341821  8258 raft_consensus.cc:2468] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 3.
I20260709 13:01:24.343024  8147 leader_election.cc:304] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:24.343148  8331 raft_consensus.cc:2468] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 3.
I20260709 13:01:24.343876  8484 raft_consensus.cc:2804] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:24.344372  8484 raft_consensus.cc:697] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 LEADER]: Becoming Leader. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:24.345216  8484 consensus_queue.cc:237] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.352304  8080 catalog_manager.cc:5697] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc reported cstate change: term changed from 2 to 3, leader changed from 78dc985924c447f08ef73e4638a9d527 (127.7.89.130) to 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129). New cstate: current_term: 3 leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:24.447510  8479 raft_consensus.cc:993] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889: : Instructing follower 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:24.448271  8485 raft_consensus.cc:1081] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 LEADER]: Signalling peer 60db2e9279404d9dab6b5c19f8d7cafc to start an election
I20260709 13:01:24.450150  8182 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
 from {username='slave'} at 127.0.0.1:44106
I20260709 13:01:24.450907  8182 raft_consensus.cc:493] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:24.451256  8182 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:24.458312  8182 raft_consensus.cc:515] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.460557  8182 leader_election.cc:290] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 4 election: Requested vote from peers 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:24.461440  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:24.461725  8331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6" candidate_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:24.462124  8258 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:24.462533  8331 raft_consensus.cc:3055] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:24.462913  8331 raft_consensus.cc:740] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Running, Role: LEADER
I20260709 13:01:24.463665  8331 consensus_queue.cc:260] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.464694  8331 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:24.468070  8258 raft_consensus.cc:2468] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 4.
I20260709 13:01:24.469604  8147 leader_election.cc:304] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60db2e9279404d9dab6b5c19f8d7cafc, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:24.470587  8484 raft_consensus.cc:2804] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:24.470674  8331 raft_consensus.cc:2468] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60db2e9279404d9dab6b5c19f8d7cafc in term 4.
I20260709 13:01:24.471213  8484 raft_consensus.cc:697] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 4 LEADER]: Becoming Leader. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:24.472095  8484 consensus_queue.cc:237] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:24.478747  8080 catalog_manager.cc:5697] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc reported cstate change: term changed from 3 to 4, leader changed from 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131) to 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129). New cstate: current_term: 4 leader_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:24.537922  8331 raft_consensus.cc:1275] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:24.539386  8484 consensus_queue.cc:1048] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:24.550351  8258 raft_consensus.cc:1275] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:24.551976  8484 consensus_queue.cc:1048] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:24.692377  8258 raft_consensus.cc:1275] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:24.693813  8484 consensus_queue.cc:1048] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:24.694316  8331 raft_consensus.cc:1275] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:24.695974  8484 consensus_queue.cc:1048] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:24.741323  8258 raft_consensus.cc:1275] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:24.743000  8501 consensus_queue.cc:1048] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:24.753770  8331 raft_consensus.cc:1275] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:24.755934  8515 consensus_queue.cc:1048] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:24.834311  8331 raft_consensus.cc:1275] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:24.835829  8501 consensus_queue.cc:1048] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:24.846823  8259 raft_consensus.cc:1275] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:24.848546  8515 consensus_queue.cc:1048] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:24.925376  8331 raft_consensus.cc:1275] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 4 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:24.926980  8515 consensus_queue.cc:1048] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:24.939956  8259 raft_consensus.cc:1275] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 4 FOLLOWER]: Refusing update from remote peer 60db2e9279404d9dab6b5c19f8d7cafc: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:24.942071  8521 consensus_queue.cc:1048] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:27.013999  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:27.022010  8182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
mode: GRACEFUL
new_leader_uuid: "78dc985924c447f08ef73e4638a9d527"
 from {username='slave'} at 127.0.0.1:44134
I20260709 13:01:27.022746  8182 raft_consensus.cc:606] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 LEADER]: Received request to transfer leadership to TS 78dc985924c447f08ef73e4638a9d527
I20260709 13:01:27.025854  8182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
mode: GRACEFUL
new_leader_uuid: "78dc985924c447f08ef73e4638a9d527"
 from {username='slave'} at 127.0.0.1:44134
I20260709 13:01:27.026518  8182 raft_consensus.cc:606] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 4 LEADER]: Received request to transfer leadership to TS 78dc985924c447f08ef73e4638a9d527
I20260709 13:01:27.029408  8182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
mode: GRACEFUL
new_leader_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
 from {username='slave'} at 127.0.0.1:44134
I20260709 13:01:27.030009  8182 raft_consensus.cc:606] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Received request to transfer leadership to TS 6ed78a204d0b4fc5b898283bef2b4889
I20260709 13:01:27.033061  8182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e6ce814b173348d8ad6fbcbb7f0ceae8"
dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
mode: GRACEFUL
new_leader_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
 from {username='slave'} at 127.0.0.1:44134
I20260709 13:01:27.033707  8182 raft_consensus.cc:606] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Received request to transfer leadership to TS 6ed78a204d0b4fc5b898283bef2b4889
I20260709 13:01:27.035320  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:01:27.036022  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:27.040279  8484 raft_consensus.cc:993] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc: : Instructing follower 6ed78a204d0b4fc5b898283bef2b4889 to start an election
I20260709 13:01:27.040817  8484 raft_consensus.cc:1081] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Signalling peer 6ed78a204d0b4fc5b898283bef2b4889 to start an election
I20260709 13:01:27.042762  8332 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b"
dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
 from {username='slave'} at 127.0.0.1:45606
I20260709 13:01:27.043347  8332 raft_consensus.cc:493] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:27.043694  8332 raft_consensus.cc:3060] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:27.050015  8332 raft_consensus.cc:515] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:27.052028  8332 leader_election.cc:290] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 3 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449)
I20260709 13:01:27.053089  8182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
I20260709 13:01:27.053916  8182 raft_consensus.cc:3055] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:27.053187  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0ef3890cef64b66988b3243a5503e6b" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:27.054291  8182 raft_consensus.cc:740] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:27.054549  8258 raft_consensus.cc:3060] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:27.055265  8182 consensus_queue.cc:260] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:27.056761  8182 raft_consensus.cc:3060] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:27.062700  8258 raft_consensus.cc:2468] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 3.
I20260709 13:01:27.064399  8296 leader_election.cc:304] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:27.064785  8182 raft_consensus.cc:2468] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 3.
I20260709 13:01:27.065452  8485 raft_consensus.cc:2804] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:27.066603  8485 raft_consensus.cc:697] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 LEADER]: Becoming Leader. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Running, Role: LEADER
I20260709 13:01:27.067565  8485 consensus_queue.cc:237] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:27.076171  8080 catalog_manager.cc:5697] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 reported cstate change: term changed from 2 to 3, leader changed from 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129) to 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "6ed78a204d0b4fc5b898283bef2b4889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:27.122064  8524 raft_consensus.cc:993] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc: : Instructing follower 78dc985924c447f08ef73e4638a9d527 to start an election
I20260709 13:01:27.122633  8484 raft_consensus.cc:1081] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 LEADER]: Signalling peer 78dc985924c447f08ef73e4638a9d527 to start an election
I20260709 13:01:27.124399  8258 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae"
dest_uuid: "78dc985924c447f08ef73e4638a9d527"
 from {username='slave'} at 127.0.0.1:47204
I20260709 13:01:27.124938  8258 raft_consensus.cc:493] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:27.125223  8258 raft_consensus.cc:3060] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.131018  8258 raft_consensus.cc:515] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:27.133186  8258 leader_election.cc:290] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 4 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:27.134449  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:27.134495  8182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e810a1ccf6947c2bcf984f4ec1e9dae" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
I20260709 13:01:27.135303  8332 raft_consensus.cc:3060] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.135396  8182 raft_consensus.cc:3055] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:27.135686  8182 raft_consensus.cc:740] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:27.136448  8182 consensus_queue.cc:260] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:27.137534  8182 raft_consensus.cc:3060] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.140782  8332 raft_consensus.cc:2468] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 4.
I20260709 13:01:27.142313  8224 leader_election.cc:304] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:27.143342  8523 raft_consensus.cc:2804] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:27.143960  8182 raft_consensus.cc:2468] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 4.
I20260709 13:01:27.144827  8523 raft_consensus.cc:697] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 4 LEADER]: Becoming Leader. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Running, Role: LEADER
I20260709 13:01:27.145793  8523 consensus_queue.cc:237] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:27.153764  8080 catalog_manager.cc:5697] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 reported cstate change: term changed from 3 to 4, leader changed from 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129) to 78dc985924c447f08ef73e4638a9d527 (127.7.89.130). New cstate: current_term: 4 leader_uuid: "78dc985924c447f08ef73e4638a9d527" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:27.353891  8524 raft_consensus.cc:993] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc: : Instructing follower 6ed78a204d0b4fc5b898283bef2b4889 to start an election
I20260709 13:01:27.354485  8524 raft_consensus.cc:1081] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Signalling peer 6ed78a204d0b4fc5b898283bef2b4889 to start an election
I20260709 13:01:27.356256  8332 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e6ce814b173348d8ad6fbcbb7f0ceae8"
dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
 from {username='slave'} at 127.0.0.1:45606
I20260709 13:01:27.356771  8332 raft_consensus.cc:493] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:27.357169  8332 raft_consensus.cc:3060] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:27.362818  8332 raft_consensus.cc:515] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:27.364696  8332 leader_election.cc:290] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 3 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 78dc985924c447f08ef73e4638a9d527 (127.7.89.130:42449)
I20260709 13:01:27.365907  8258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6ce814b173348d8ad6fbcbb7f0ceae8" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78dc985924c447f08ef73e4638a9d527"
I20260709 13:01:27.365909  8182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6ce814b173348d8ad6fbcbb7f0ceae8" candidate_uuid: "6ed78a204d0b4fc5b898283bef2b4889" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
I20260709 13:01:27.366828  8258 raft_consensus.cc:3060] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:27.366833  8182 raft_consensus.cc:3055] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:27.367339  8182 raft_consensus.cc:740] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:27.368208  8182 consensus_queue.cc:260] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:27.369556  8182 raft_consensus.cc:3060] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:27.374801  8258 raft_consensus.cc:2468] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 3.
I20260709 13:01:27.376179  8296 leader_election.cc:304] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:27.377017  8485 raft_consensus.cc:2804] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:27.377029  8182 raft_consensus.cc:2468] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ed78a204d0b4fc5b898283bef2b4889 in term 3.
I20260709 13:01:27.378053  8485 raft_consensus.cc:697] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 LEADER]: Becoming Leader. State: Replica: 6ed78a204d0b4fc5b898283bef2b4889, State: Running, Role: LEADER
I20260709 13:01:27.379040  8485 consensus_queue.cc:237] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:27.388628  8080 catalog_manager.cc:5697] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 reported cstate change: term changed from 2 to 3, leader changed from 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129) to 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "6ed78a204d0b4fc5b898283bef2b4889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:27.408792  8484 raft_consensus.cc:993] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc: : Instructing follower 78dc985924c447f08ef73e4638a9d527 to start an election
I20260709 13:01:27.409307  8484 raft_consensus.cc:1081] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 4 LEADER]: Signalling peer 78dc985924c447f08ef73e4638a9d527 to start an election
I20260709 13:01:27.410931  8258 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6"
dest_uuid: "78dc985924c447f08ef73e4638a9d527"
 from {username='slave'} at 127.0.0.1:47204
I20260709 13:01:27.411429  8258 raft_consensus.cc:493] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:27.411722  8258 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:27.416786  8258 raft_consensus.cc:515] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:27.418427  8258 leader_election.cc:290] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 5 election: Requested vote from peers 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953), 6ed78a204d0b4fc5b898283bef2b4889 (127.7.89.131:37523)
I20260709 13:01:27.419487  8332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "6ed78a204d0b4fc5b898283bef2b4889"
I20260709 13:01:27.419487  8182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2434e2b7fd54cc19302a5f1457391a6" candidate_uuid: "78dc985924c447f08ef73e4638a9d527" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "60db2e9279404d9dab6b5c19f8d7cafc"
I20260709 13:01:27.420140  8332 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:27.420214  8182 raft_consensus.cc:3055] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:27.420616  8182 raft_consensus.cc:740] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 60db2e9279404d9dab6b5c19f8d7cafc, State: Running, Role: LEADER
I20260709 13:01:27.421316  8182 consensus_queue.cc:260] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:27.422346  8182 raft_consensus.cc:3060] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:27.425685  8332 raft_consensus.cc:2468] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 5.
I20260709 13:01:27.426926  8224 leader_election.cc:304] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ed78a204d0b4fc5b898283bef2b4889, 78dc985924c447f08ef73e4638a9d527; no voters: 
I20260709 13:01:27.427881  8523 raft_consensus.cc:2804] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:27.428339  8182 raft_consensus.cc:2468] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78dc985924c447f08ef73e4638a9d527 in term 5.
I20260709 13:01:27.428335  8523 raft_consensus.cc:697] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 5 LEADER]: Becoming Leader. State: Replica: 78dc985924c447f08ef73e4638a9d527, State: Running, Role: LEADER
I20260709 13:01:27.430013  8523 consensus_queue.cc:237] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } }
I20260709 13:01:27.438314  8080 catalog_manager.cc:5697] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 reported cstate change: term changed from 4 to 5, leader changed from 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129) to 78dc985924c447f08ef73e4638a9d527 (127.7.89.130). New cstate: current_term: 5 leader_uuid: "78dc985924c447f08ef73e4638a9d527" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:27.509063  8258 raft_consensus.cc:1275] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Refusing update from remote peer 6ed78a204d0b4fc5b898283bef2b4889: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:27.510637  8485 consensus_queue.cc:1048] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:27.522794  8182 raft_consensus.cc:1275] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Refusing update from remote peer 6ed78a204d0b4fc5b898283bef2b4889: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:27.524876  8538 consensus_queue.cc:1048] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:27.721906  8182 raft_consensus.cc:1275] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 4 FOLLOWER]: Refusing update from remote peer 78dc985924c447f08ef73e4638a9d527: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:27.723553  8523 consensus_queue.cc:1048] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:27.733914  8332 raft_consensus.cc:1275] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 4 FOLLOWER]: Refusing update from remote peer 78dc985924c447f08ef73e4638a9d527: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:27.736042  8523 consensus_queue.cc:1048] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:27.808764  8182 raft_consensus.cc:1275] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Refusing update from remote peer 6ed78a204d0b4fc5b898283bef2b4889: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:27.810182  8538 consensus_queue.cc:1048] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:27.820938  8258 raft_consensus.cc:1275] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Refusing update from remote peer 6ed78a204d0b4fc5b898283bef2b4889: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:27.823462  8485 consensus_queue.cc:1048] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78dc985924c447f08ef73e4638a9d527" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:27.857338  8332 raft_consensus.cc:1275] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 5 FOLLOWER]: Refusing update from remote peer 78dc985924c447f08ef73e4638a9d527: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:27.858902  8545 consensus_queue.cc:1048] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ed78a204d0b4fc5b898283bef2b4889" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:27.869346  8182 raft_consensus.cc:1275] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 5 FOLLOWER]: Refusing update from remote peer 78dc985924c447f08ef73e4638a9d527: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:27.871270  8543 consensus_queue.cc:1048] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db2e9279404d9dab6b5c19f8d7cafc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:28.037226  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:28.044878  7526 tablet_server.cc:179] TabletServer@127.7.89.129:0 shutting down...
I20260709 13:01:28.073655  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:28.074595  7526 tablet_replica.cc:333] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc: stopping tablet replica
I20260709 13:01:28.075713  7526 raft_consensus.cc:2243] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:28.076855  7526 raft_consensus.cc:2272] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.078985  7526 tablet_replica.cc:333] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc: stopping tablet replica
I20260709 13:01:28.079699  7526 raft_consensus.cc:2243] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.080132  7526 raft_consensus.cc:2272] T 456f553f05b141b2b43d3d710034fb7f P 60db2e9279404d9dab6b5c19f8d7cafc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.082165  7526 tablet_replica.cc:333] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc: stopping tablet replica
I20260709 13:01:28.082809  7526 raft_consensus.cc:2243] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.083222  7526 raft_consensus.cc:2272] T b2434e2b7fd54cc19302a5f1457391a6 P 60db2e9279404d9dab6b5c19f8d7cafc [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.085577  7526 tablet_replica.cc:333] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc: stopping tablet replica
I20260709 13:01:28.086181  7526 raft_consensus.cc:2243] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:28.087086  7526 raft_consensus.cc:2272] T 7d8cefd485c946b4a33d75aade5c3668 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.089035  7526 tablet_replica.cc:333] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc: stopping tablet replica
I20260709 13:01:28.089535  7526 raft_consensus.cc:2243] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:28.090276  7526 raft_consensus.cc:2272] T 7207cf9522d64ea49340bd7c2950431e P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.092289  7526 tablet_replica.cc:333] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc: stopping tablet replica
I20260709 13:01:28.092871  7526 raft_consensus.cc:2243] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.093335  7526 raft_consensus.cc:2272] T c0ef3890cef64b66988b3243a5503e6b P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.096122  7526 tablet_replica.cc:333] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc: stopping tablet replica
I20260709 13:01:28.096738  7526 raft_consensus.cc:2243] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.097383  7526 raft_consensus.cc:2272] T f8a2b7f4f90441f78ede444a23726b3b P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.099584  7526 tablet_replica.cc:333] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc: stopping tablet replica
I20260709 13:01:28.100193  7526 raft_consensus.cc:2243] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.100708  7526 raft_consensus.cc:2272] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 60db2e9279404d9dab6b5c19f8d7cafc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.103519  7526 tablet_replica.cc:333] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc: stopping tablet replica
I20260709 13:01:28.104071  7526 raft_consensus.cc:2243] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.104487  7526 raft_consensus.cc:2272] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 60db2e9279404d9dab6b5c19f8d7cafc [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.107074  7526 tablet_replica.cc:333] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc: stopping tablet replica
I20260709 13:01:28.107641  7526 raft_consensus.cc:2243] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:28.108491  7526 raft_consensus.cc:2272] T 919af25e2da94f199867009579567a66 P 60db2e9279404d9dab6b5c19f8d7cafc [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:01:28.113440  8223 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
W20260709 13:01:28.117357  8223 consensus_peers.cc:597] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 -> Peer 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953): Couldn't send request to peer 60db2e9279404d9dab6b5c19f8d7cafc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:28.142807  7526 tablet_server.cc:196] TabletServer@127.7.89.129:0 shutdown complete.
I20260709 13:01:28.171478  7526 tablet_server.cc:179] TabletServer@127.7.89.130:0 shutting down...
I20260709 13:01:28.200042  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:28.201107  7526 tablet_replica.cc:333] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527: stopping tablet replica
I20260709 13:01:28.201953  7526 raft_consensus.cc:2243] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:01:28.203009  7526 raft_consensus.cc:2272] T 456f553f05b141b2b43d3d710034fb7f P 78dc985924c447f08ef73e4638a9d527 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.205214  7526 tablet_replica.cc:333] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527: stopping tablet replica
I20260709 13:01:28.205847  7526 raft_consensus.cc:2243] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:01:28.206801  7526 raft_consensus.cc:2272] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 78dc985924c447f08ef73e4638a9d527 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.209461  7526 tablet_replica.cc:333] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527: stopping tablet replica
I20260709 13:01:28.210124  7526 raft_consensus.cc:2243] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.210700  7526 raft_consensus.cc:2272] T 7207cf9522d64ea49340bd7c2950431e P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.212666  7526 tablet_replica.cc:333] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527: stopping tablet replica
I20260709 13:01:28.213318  7526 raft_consensus.cc:2243] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.213819  7526 raft_consensus.cc:2272] T c0ef3890cef64b66988b3243a5503e6b P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.216958  7526 tablet_replica.cc:333] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527: stopping tablet replica
I20260709 13:01:28.217592  7526 raft_consensus.cc:2243] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.218120  7526 raft_consensus.cc:2272] T f8a2b7f4f90441f78ede444a23726b3b P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.220049  7526 tablet_replica.cc:333] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527: stopping tablet replica
I20260709 13:01:28.220624  7526 raft_consensus.cc:2243] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:01:28.221494  7526 raft_consensus.cc:2272] T b2434e2b7fd54cc19302a5f1457391a6 P 78dc985924c447f08ef73e4638a9d527 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.224017  7526 tablet_replica.cc:333] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527: stopping tablet replica
I20260709 13:01:28.224599  7526 raft_consensus.cc:2243] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.225068  7526 raft_consensus.cc:2272] T 7d8cefd485c946b4a33d75aade5c3668 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.227046  7526 tablet_replica.cc:333] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527: stopping tablet replica
I20260709 13:01:28.227764  7526 raft_consensus.cc:2243] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.228466  7526 raft_consensus.cc:2272] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.231288  7526 tablet_replica.cc:333] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527: stopping tablet replica
I20260709 13:01:28.232043  7526 raft_consensus.cc:2243] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.232781  7526 raft_consensus.cc:2272] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 78dc985924c447f08ef73e4638a9d527 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.235713  7526 tablet_replica.cc:333] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527: stopping tablet replica
I20260709 13:01:28.236263  7526 raft_consensus.cc:2243] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.236778  7526 raft_consensus.cc:2272] T 919af25e2da94f199867009579567a66 P 78dc985924c447f08ef73e4638a9d527 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.268754  7526 tablet_server.cc:196] TabletServer@127.7.89.130:0 shutdown complete.
W20260709 13:01:28.281348  8297 consensus_peers.cc:597] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 -> Peer 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953): Couldn't send request to peer 60db2e9279404d9dab6b5c19f8d7cafc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:35953: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:28.293298  7526 tablet_server.cc:179] TabletServer@127.7.89.131:0 shutting down...
I20260709 13:01:28.321775  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:01:28.322691  7526 tablet_replica.cc:333] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889: stopping tablet replica
I20260709 13:01:28.323735  7526 raft_consensus.cc:2243] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.324419  7526 raft_consensus.cc:2272] T 456f553f05b141b2b43d3d710034fb7f P 6ed78a204d0b4fc5b898283bef2b4889 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.326577  7526 tablet_replica.cc:333] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889: stopping tablet replica
I20260709 13:01:28.327247  7526 raft_consensus.cc:2243] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.327662  7526 raft_consensus.cc:2272] T b99c6d9bc60d4dadb91d2ac9a4a09891 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.329612  7526 tablet_replica.cc:333] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889: stopping tablet replica
I20260709 13:01:28.330132  7526 raft_consensus.cc:2243] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.330514  7526 raft_consensus.cc:2272] T 9e810a1ccf6947c2bcf984f4ec1e9dae P 6ed78a204d0b4fc5b898283bef2b4889 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.332953  7526 tablet_replica.cc:333] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889: stopping tablet replica
I20260709 13:01:28.333556  7526 raft_consensus.cc:2243] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:28.334421  7526 raft_consensus.cc:2272] T e6ce814b173348d8ad6fbcbb7f0ceae8 P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.336869  7526 tablet_replica.cc:333] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889: stopping tablet replica
I20260709 13:01:28.337431  7526 raft_consensus.cc:2243] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.337869  7526 raft_consensus.cc:2272] T 919af25e2da94f199867009579567a66 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.340241  7526 tablet_replica.cc:333] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889: stopping tablet replica
I20260709 13:01:28.341065  7526 raft_consensus.cc:2243] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.341773  7526 raft_consensus.cc:2272] T 7d8cefd485c946b4a33d75aade5c3668 P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.344256  7526 tablet_replica.cc:333] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889: stopping tablet replica
W20260709 13:01:28.344914  8297 consensus_peers.cc:597] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 -> Peer 60db2e9279404d9dab6b5c19f8d7cafc (127.7.89.129:35953): Couldn't send request to peer 60db2e9279404d9dab6b5c19f8d7cafc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:35953: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:28.344960  7526 raft_consensus.cc:2243] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.345752  7526 raft_consensus.cc:2272] T b2434e2b7fd54cc19302a5f1457391a6 P 6ed78a204d0b4fc5b898283bef2b4889 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.348141  7526 tablet_replica.cc:333] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889: stopping tablet replica
I20260709 13:01:28.348711  7526 raft_consensus.cc:2243] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:01:28.349555  7526 raft_consensus.cc:2272] T c0ef3890cef64b66988b3243a5503e6b P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.351776  7526 tablet_replica.cc:333] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889: stopping tablet replica
I20260709 13:01:28.352291  7526 raft_consensus.cc:2243] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:28.352733  7526 raft_consensus.cc:2272] T 7207cf9522d64ea49340bd7c2950431e P 6ed78a204d0b4fc5b898283bef2b4889 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.354594  7526 tablet_replica.cc:333] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889: stopping tablet replica
I20260709 13:01:28.355142  7526 raft_consensus.cc:2243] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:01:28.355903  7526 raft_consensus.cc:2272] T f8a2b7f4f90441f78ede444a23726b3b P 6ed78a204d0b4fc5b898283bef2b4889 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.385856  7526 tablet_server.cc:196] TabletServer@127.7.89.131:0 shutdown complete.
I20260709 13:01:28.408977  7526 master.cc:562] Master@127.7.89.190:39063 shutting down...
I20260709 13:01:28.432987  7526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:01:28.433676  7526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:28.434092  7526 tablet_replica.cc:333] T 00000000000000000000000000000000 P 635b60e76903487ba0bb2d0cfb7ba020: stopping tablet replica
I20260709 13:01:28.457619  7526 master.cc:584] Master@127.7.89.190:39063 shutdown complete.
60db2e9279404d9dab6b5c19f8d7cafc  5
6ed78a204d0b4fc5b898283bef2b4889  1
78dc985924c447f08ef73e4638a9d527  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10844 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:01:28.510070  7526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.89.190:45099
I20260709 13:01:28.511420  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:28.518014  8554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:28.518009  8555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:28.519748  8557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:28.521660  7526 server_base.cc:1061] running on GCE node
I20260709 13:01:28.523099  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:28.523383  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:28.523607  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602088523581 us; error 0 us; skew 500 ppm
I20260709 13:01:28.524627  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:28.528571  7526 webserver.cc:533] Webserver started at http://127.7.89.190:40153/ using document root <none> and password file <none>
I20260709 13:01:28.529320  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:28.529587  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:28.530059  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:28.531970  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/master-0-root/instance:
uuid: "06d3808b553142ef8f7a4e57845757cd"
format_stamp: "Formatted at 2026-07-09 13:01:28 on dist-test-slave-h0rz"
I20260709 13:01:28.538021  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:01:28.542445  8562 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:28.543663  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.003s
I20260709 13:01:28.544051  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/master-0-root
uuid: "06d3808b553142ef8f7a4e57845757cd"
format_stamp: "Formatted at 2026-07-09 13:01:28 on dist-test-slave-h0rz"
I20260709 13:01:28.544411  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:28.562605  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:28.564214  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:28.609781  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.190:45099
I20260709 13:01:28.609889  8613 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.190:45099 every 8 connection(s)
I20260709 13:01:28.614408  8614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:28.627454  8614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Bootstrap starting.
I20260709 13:01:28.633406  8614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:28.639245  8614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: No bootstrap required, opened a new log
I20260709 13:01:28.641825  8614 raft_consensus.cc:359] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06d3808b553142ef8f7a4e57845757cd" member_type: VOTER }
I20260709 13:01:28.642285  8614 raft_consensus.cc:385] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:28.642582  8614 raft_consensus.cc:740] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d3808b553142ef8f7a4e57845757cd, State: Initialized, Role: FOLLOWER
I20260709 13:01:28.643225  8614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06d3808b553142ef8f7a4e57845757cd" member_type: VOTER }
I20260709 13:01:28.643703  8614 raft_consensus.cc:399] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:01:28.643934  8614 raft_consensus.cc:493] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:01:28.644214  8614 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:28.649361  8614 raft_consensus.cc:515] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06d3808b553142ef8f7a4e57845757cd" member_type: VOTER }
I20260709 13:01:28.649955  8614 leader_election.cc:304] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06d3808b553142ef8f7a4e57845757cd; no voters: 
I20260709 13:01:28.651345  8614 leader_election.cc:290] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:01:28.651736  8617 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:28.653442  8617 raft_consensus.cc:697] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [term 1 LEADER]: Becoming Leader. State: Replica: 06d3808b553142ef8f7a4e57845757cd, State: Running, Role: LEADER
I20260709 13:01:28.654327  8617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06d3808b553142ef8f7a4e57845757cd" member_type: VOTER }
I20260709 13:01:28.654600  8614 sys_catalog.cc:565] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:01:28.659759  8618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06d3808b553142ef8f7a4e57845757cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06d3808b553142ef8f7a4e57845757cd" member_type: VOTER } }
I20260709 13:01:28.660144  8619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06d3808b553142ef8f7a4e57845757cd. Latest consensus state: current_term: 1 leader_uuid: "06d3808b553142ef8f7a4e57845757cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06d3808b553142ef8f7a4e57845757cd" member_type: VOTER } }
I20260709 13:01:28.660691  8618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:28.661283  8619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [sys.catalog]: This master's current role is: LEADER
I20260709 13:01:28.665113  8624 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:01:28.673913  8624 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:01:28.675098  7526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:01:28.684279  8624 catalog_manager.cc:1383] Generated new cluster ID: cad1d2952161445e9ac3a2960adebc93
I20260709 13:01:28.684643  8624 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:01:28.722043  8624 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:01:28.723596  8624 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:01:28.747886  8624 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Generated new TSK 0
I20260709 13:01:28.748687  8624 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:01:28.806434  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:28.813387  8635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:28.814687  8636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:28.816114  8638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:28.816274  7526 server_base.cc:1061] running on GCE node
I20260709 13:01:28.817332  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:28.817554  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:28.817747  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602088817728 us; error 0 us; skew 500 ppm
I20260709 13:01:28.818307  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:28.820976  7526 webserver.cc:533] Webserver started at http://127.7.89.129:40487/ using document root <none> and password file <none>
I20260709 13:01:28.821583  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:28.821805  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:28.822134  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:28.823546  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-0-root/instance:
uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
format_stamp: "Formatted at 2026-07-09 13:01:28 on dist-test-slave-h0rz"
I20260709 13:01:28.828972  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.005s
I20260709 13:01:28.833297  8643 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:28.834131  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:01:28.834493  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-0-root
uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
format_stamp: "Formatted at 2026-07-09 13:01:28 on dist-test-slave-h0rz"
I20260709 13:01:28.834889  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:28.849764  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:28.851439  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:28.853228  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:28.855968  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:28.856237  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:28.856572  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:28.856775  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:28.903035  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.129:43221
I20260709 13:01:28.903151  8705 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.129:43221 every 8 connection(s)
I20260709 13:01:28.908849  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:28.917096  8710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:28.918550  8711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:28.920183  8713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:28.920526  7526 server_base.cc:1061] running on GCE node
I20260709 13:01:28.921823  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:28.922060  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:28.922228  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602088922208 us; error 0 us; skew 500 ppm
I20260709 13:01:28.922847  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:28.925700  7526 webserver.cc:533] Webserver started at http://127.7.89.130:45081/ using document root <none> and password file <none>
I20260709 13:01:28.926231  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:28.926455  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:28.926838  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:28.926802  8706 heartbeater.cc:344] Connected to a master server at 127.7.89.190:45099
I20260709 13:01:28.927321  8706 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:28.928117  8706 heartbeater.cc:507] Master 127.7.89.190:45099 requested a full tablet report, sending...
I20260709 13:01:28.928282  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-1-root/instance:
uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
format_stamp: "Formatted at 2026-07-09 13:01:28 on dist-test-slave-h0rz"
I20260709 13:01:28.930356  8579 ts_manager.cc:194] Registered new tserver with Master: 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221)
I20260709 13:01:28.932349  8579 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43462
I20260709 13:01:28.933737  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:01:28.937517  8718 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:28.938455  7526 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:01:28.938808  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-1-root
uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
format_stamp: "Formatted at 2026-07-09 13:01:28 on dist-test-slave-h0rz"
I20260709 13:01:28.939105  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:28.954442  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:28.955936  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:28.957741  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:28.960652  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:28.960902  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:28.961217  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:28.961377  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:29.007339  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.130:42321
I20260709 13:01:29.007477  8780 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.130:42321 every 8 connection(s)
I20260709 13:01:29.012764  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:29.021304  8784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:29.024403  8785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:29.028403  8781 heartbeater.cc:344] Connected to a master server at 127.7.89.190:45099
I20260709 13:01:29.029036  8781 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:29.029179  7526 server_base.cc:1061] running on GCE node
W20260709 13:01:29.029407  8788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:29.030318  8781 heartbeater.cc:507] Master 127.7.89.190:45099 requested a full tablet report, sending...
I20260709 13:01:29.030539  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:29.030900  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:29.031085  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602089031066 us; error 0 us; skew 500 ppm
I20260709 13:01:29.031910  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:29.033111  8579 ts_manager.cc:194] Registered new tserver with Master: 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:29.035028  8579 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43478
I20260709 13:01:29.035279  7526 webserver.cc:533] Webserver started at http://127.7.89.131:43501/ using document root <none> and password file <none>
I20260709 13:01:29.035977  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:29.036283  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:29.036620  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:29.038439  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root/instance:
uuid: "e814ed2013244fa6be42623ea5c0c7b9"
format_stamp: "Formatted at 2026-07-09 13:01:29 on dist-test-slave-h0rz"
I20260709 13:01:29.043509  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:01:29.047286  8792 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:29.048270  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:01:29.048609  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root
uuid: "e814ed2013244fa6be42623ea5c0c7b9"
format_stamp: "Formatted at 2026-07-09 13:01:29 on dist-test-slave-h0rz"
I20260709 13:01:29.048923  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:29.074676  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:29.076277  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:29.078038  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:29.080938  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:29.081169  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:29.081523  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:29.081689  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:29.131013  8854 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.131:37857 every 8 connection(s)
I20260709 13:01:29.131002  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.131:37857
I20260709 13:01:29.147466  8855 heartbeater.cc:344] Connected to a master server at 127.7.89.190:45099
I20260709 13:01:29.147919  8855 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:29.148779  8855 heartbeater.cc:507] Master 127.7.89.190:45099 requested a full tablet report, sending...
I20260709 13:01:29.151223  8579 ts_manager.cc:194] Registered new tserver with Master: e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:29.152233  7526 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017284433s
I20260709 13:01:29.152994  8579 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43482
I20260709 13:01:29.182719  8579 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43486:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:01:29.189545  8579 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:01:29.403741  8820 tablet_service.cc:1511] Processing CreateTablet for tablet 295d39c61dd3489e98d9393a2da9c062 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:01:29.404839  8817 tablet_service.cc:1511] Processing CreateTablet for tablet 1148edfb8a5a41fdaced6489dfd22b63 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:01:29.405500  8820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 295d39c61dd3489e98d9393a2da9c062. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.406414  8817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1148edfb8a5a41fdaced6489dfd22b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.407852  8819 tablet_service.cc:1511] Processing CreateTablet for tablet 9ca09750f1b44e018d62c21aeb12a929 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:01:29.409314  8819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ca09750f1b44e018d62c21aeb12a929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.411993  8818 tablet_service.cc:1511] Processing CreateTablet for tablet 9f81d7aa5ed04c68b603d272f24760e2 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:01:29.412518  8816 tablet_service.cc:1511] Processing CreateTablet for tablet 05559e602d4b4ea3986701fa621706a0 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:01:29.415448  8815 tablet_service.cc:1511] Processing CreateTablet for tablet 5ee1e688ccd74392befec90535c1efa4 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:01:29.416936  8815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ee1e688ccd74392befec90535c1efa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.418989  8816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05559e602d4b4ea3986701fa621706a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.420684  8818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f81d7aa5ed04c68b603d272f24760e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.420262  8814 tablet_service.cc:1511] Processing CreateTablet for tablet 2217efcaa5fe4b729f1beef2f96e4189 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:01:29.423494  8814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2217efcaa5fe4b729f1beef2f96e4189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.424605  8813 tablet_service.cc:1511] Processing CreateTablet for tablet b50b38d6e5314ca190c9a14a5e0f655d (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:01:29.426155  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b50b38d6e5314ca190c9a14a5e0f655d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.429754  8812 tablet_service.cc:1511] Processing CreateTablet for tablet 3524354c9db64c0c9b532bef2a0ceb6f (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:01:29.431329  8812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3524354c9db64c0c9b532bef2a0ceb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.434973  8811 tablet_service.cc:1511] Processing CreateTablet for tablet 10f975b4911743768a7512f2ad3793c9 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:01:29.436576  8811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10f975b4911743768a7512f2ad3793c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.486541  8812 tablet_service.cc:1511] Processing CreateTablet for tablet 7fb0366ec71a44feb8c7b426e851dcc6 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:01:29.488073  8812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fb0366ec71a44feb8c7b426e851dcc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.491336  8817 tablet_service.cc:1511] Processing CreateTablet for tablet 09c390d8b5bb47c9a7d7a841a4fe135e (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:01:29.492846  8817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09c390d8b5bb47c9a7d7a841a4fe135e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.548591  8870 tablet_bootstrap.cc:492] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:29.550513  8813 tablet_service.cc:1511] Processing CreateTablet for tablet 20fcd6d0c3304abda83f416721e807c5 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:01:29.552143  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20fcd6d0c3304abda83f416721e807c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.553988  8671 tablet_service.cc:1511] Processing CreateTablet for tablet 295d39c61dd3489e98d9393a2da9c062 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:01:29.555649  8671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 295d39c61dd3489e98d9393a2da9c062. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.568801  8670 tablet_service.cc:1511] Processing CreateTablet for tablet 9ca09750f1b44e018d62c21aeb12a929 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:01:29.570567  8670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ca09750f1b44e018d62c21aeb12a929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.572075  8664 tablet_service.cc:1511] Processing CreateTablet for tablet b50b38d6e5314ca190c9a14a5e0f655d (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:01:29.588331  8664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b50b38d6e5314ca190c9a14a5e0f655d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.573566  8669 tablet_service.cc:1511] Processing CreateTablet for tablet 9f81d7aa5ed04c68b603d272f24760e2 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:01:29.592290  8669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f81d7aa5ed04c68b603d272f24760e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.593004  8820 tablet_service.cc:1511] Processing CreateTablet for tablet 1f9ddea0a887446f910d38f8302e8172 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:01:29.594385  8820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f9ddea0a887446f910d38f8302e8172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.574915  8668 tablet_service.cc:1511] Processing CreateTablet for tablet 1148edfb8a5a41fdaced6489dfd22b63 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:01:29.596098  8668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1148edfb8a5a41fdaced6489dfd22b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.576121  8667 tablet_service.cc:1511] Processing CreateTablet for tablet 05559e602d4b4ea3986701fa621706a0 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:01:29.599009  8818 tablet_service.cc:1511] Processing CreateTablet for tablet 149c2129693643678d9da7765768609c (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:01:29.600189  8667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05559e602d4b4ea3986701fa621706a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.577430  8666 tablet_service.cc:1511] Processing CreateTablet for tablet 5ee1e688ccd74392befec90535c1efa4 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:01:29.604133  8666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ee1e688ccd74392befec90535c1efa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.604768  8815 tablet_service.cc:1511] Processing CreateTablet for tablet 8acd9840694e4f62803bfe9810425202 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:01:29.578680  8665 tablet_service.cc:1511] Processing CreateTablet for tablet 2217efcaa5fe4b729f1beef2f96e4189 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:01:29.606415  8665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2217efcaa5fe4b729f1beef2f96e4189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.607968  8870 tablet_bootstrap.cc:654] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.580848  8663 tablet_service.cc:1511] Processing CreateTablet for tablet 3524354c9db64c0c9b532bef2a0ceb6f (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:01:29.609614  8663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3524354c9db64c0c9b532bef2a0ceb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.582118  8662 tablet_service.cc:1511] Processing CreateTablet for tablet 10f975b4911743768a7512f2ad3793c9 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:01:29.611433  8662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10f975b4911743768a7512f2ad3793c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.619634  8816 tablet_service.cc:1511] Processing CreateTablet for tablet 142891c077a34ba289adf90a99aee612 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:01:29.630177  8814 tablet_service.cc:1511] Processing CreateTablet for tablet c01533e2a5204b1ebccac1133e495542 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:01:29.634286  8818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 149c2129693643678d9da7765768609c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.634761  8819 tablet_service.cc:1511] Processing CreateTablet for tablet 95fdf24d372f40b8877c2792102f34cf (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:01:29.637023  8819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95fdf24d372f40b8877c2792102f34cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.637810  8815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8acd9840694e4f62803bfe9810425202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.630177  8811 tablet_service.cc:1511] Processing CreateTablet for tablet 8e9173dea5bf4398a5e138cb273df07c (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:01:29.643393  8814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c01533e2a5204b1ebccac1133e495542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.658676  8667 tablet_service.cc:1511] Processing CreateTablet for tablet 7fb0366ec71a44feb8c7b426e851dcc6 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:01:29.660422  8667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fb0366ec71a44feb8c7b426e851dcc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.663437  8816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142891c077a34ba289adf90a99aee612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.664121  8817 tablet_service.cc:1511] Processing CreateTablet for tablet 55c31a55ecab4a17acb00373b55d6f1d (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:01:29.676107  8817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c31a55ecab4a17acb00373b55d6f1d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:29.669564  8627 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:29.671942  8662 tablet_service.cc:1511] Processing CreateTablet for tablet 09c390d8b5bb47c9a7d7a841a4fe135e (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:01:29.679733  8662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09c390d8b5bb47c9a7d7a841a4fe135e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.678539  8812 tablet_service.cc:1511] Processing CreateTablet for tablet 7d6f9069613f4a83a64061e85b6f97da (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:01:29.704291  8812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d6f9069613f4a83a64061e85b6f97da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.710778  8746 tablet_service.cc:1511] Processing CreateTablet for tablet 295d39c61dd3489e98d9393a2da9c062 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:01:29.712533  8746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 295d39c61dd3489e98d9393a2da9c062. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.713575  8744 tablet_service.cc:1511] Processing CreateTablet for tablet 9f81d7aa5ed04c68b603d272f24760e2 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:01:29.716058  8745 tablet_service.cc:1511] Processing CreateTablet for tablet 9ca09750f1b44e018d62c21aeb12a929 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:01:29.716312  8743 tablet_service.cc:1511] Processing CreateTablet for tablet 1148edfb8a5a41fdaced6489dfd22b63 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:01:29.717705  8745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ca09750f1b44e018d62c21aeb12a929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.719762  8742 tablet_service.cc:1511] Processing CreateTablet for tablet 05559e602d4b4ea3986701fa621706a0 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:01:29.721951  8742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05559e602d4b4ea3986701fa621706a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.724834  8741 tablet_service.cc:1511] Processing CreateTablet for tablet 5ee1e688ccd74392befec90535c1efa4 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:01:29.729398  8740 tablet_service.cc:1511] Processing CreateTablet for tablet 2217efcaa5fe4b729f1beef2f96e4189 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:01:29.733135  8739 tablet_service.cc:1511] Processing CreateTablet for tablet b50b38d6e5314ca190c9a14a5e0f655d (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:01:29.734926  8741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ee1e688ccd74392befec90535c1efa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.735939  8738 tablet_service.cc:1511] Processing CreateTablet for tablet 3524354c9db64c0c9b532bef2a0ceb6f (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:01:29.738456  8737 tablet_service.cc:1511] Processing CreateTablet for tablet 10f975b4911743768a7512f2ad3793c9 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:01:29.740219  8739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b50b38d6e5314ca190c9a14a5e0f655d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.745844  8811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e9173dea5bf4398a5e138cb273df07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.751587  8743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1148edfb8a5a41fdaced6489dfd22b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.752460  8737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10f975b4911743768a7512f2ad3793c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.753369  8740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2217efcaa5fe4b729f1beef2f96e4189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.758419  8744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f81d7aa5ed04c68b603d272f24760e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.774329  8738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3524354c9db64c0c9b532bef2a0ceb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.791934  8671 tablet_service.cc:1511] Processing CreateTablet for tablet 20fcd6d0c3304abda83f416721e807c5 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:01:29.793596  8671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20fcd6d0c3304abda83f416721e807c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.762483  8870 tablet_bootstrap.cc:492] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:29.763656  8871 tablet_bootstrap.cc:492] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:29.809814  8668 tablet_service.cc:1511] Processing CreateTablet for tablet 1f9ddea0a887446f910d38f8302e8172 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:01:29.810959  8870 ts_tablet_manager.cc:1403] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.263s	user 0.025s	sys 0.033s
I20260709 13:01:29.847635  8668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f9ddea0a887446f910d38f8302e8172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.813560  8871 tablet_bootstrap.cc:654] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:29.850158  8870 raft_consensus.cc:359] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:29.854095  8870 raft_consensus.cc:385] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:29.854470  8870 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:29.814886  8666 tablet_service.cc:1511] Processing CreateTablet for tablet 149c2129693643678d9da7765768609c (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:01:29.859236  8666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 149c2129693643678d9da7765768609c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.860131  8667 tablet_service.cc:1511] Processing CreateTablet for tablet 95fdf24d372f40b8877c2792102f34cf (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:01:29.861836  8667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95fdf24d372f40b8877c2792102f34cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.819832  8814 tablet_service.cc:1511] Processing CreateTablet for tablet 8611228be8dd4f08b2b609e202974a6a (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:01:29.864341  8814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8611228be8dd4f08b2b609e202974a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.864024  8813 tablet_service.cc:1511] Processing CreateTablet for tablet 67bfc59e963f4ad0ac128fd11eecac17 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:01:29.824916  8664 tablet_service.cc:1511] Processing CreateTablet for tablet 142891c077a34ba289adf90a99aee612 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:01:29.828660  8663 tablet_service.cc:1511] Processing CreateTablet for tablet 8acd9840694e4f62803bfe9810425202 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:01:29.867079  8663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8acd9840694e4f62803bfe9810425202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.867624  8670 tablet_service.cc:1511] Processing CreateTablet for tablet 55c31a55ecab4a17acb00373b55d6f1d (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:01:29.869832  8670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c31a55ecab4a17acb00373b55d6f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.889034  8664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142891c077a34ba289adf90a99aee612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.889741  8817 tablet_service.cc:1511] Processing CreateTablet for tablet ff71213103664c0bab63c42773d0880b (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:01:29.842882  8665 tablet_service.cc:1511] Processing CreateTablet for tablet c01533e2a5204b1ebccac1133e495542 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:01:29.891544  8739 tablet_service.cc:1511] Processing CreateTablet for tablet 09c390d8b5bb47c9a7d7a841a4fe135e (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:01:29.892377  8665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c01533e2a5204b1ebccac1133e495542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.892992  8739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09c390d8b5bb47c9a7d7a841a4fe135e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.845237  8669 tablet_service.cc:1511] Processing CreateTablet for tablet 8e9173dea5bf4398a5e138cb273df07c (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:01:29.895062  8669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e9173dea5bf4398a5e138cb273df07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.885154  8870 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:29.865684  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67bfc59e963f4ad0ac128fd11eecac17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.900853  8818 tablet_service.cc:1511] Processing CreateTablet for tablet c3796428796f4d6c8fd8e9ff2897d116 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:01:29.903517  8743 tablet_service.cc:1511] Processing CreateTablet for tablet 7fb0366ec71a44feb8c7b426e851dcc6 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:01:29.904212  8820 tablet_service.cc:1511] Processing CreateTablet for tablet 50ae323a197c4e5b994d66663473c7d2 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:01:29.917044  8740 tablet_service.cc:1511] Processing CreateTablet for tablet 20fcd6d0c3304abda83f416721e807c5 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:01:29.917510  8817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff71213103664c0bab63c42773d0880b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:29.930078  8566 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 65809.1us
I20260709 13:01:29.936885  8706 heartbeater.cc:499] Master 127.7.89.190:45099 was elected leader, sending a full tablet report...
I20260709 13:01:29.930425  8815 tablet_service.cc:1511] Processing CreateTablet for tablet 41d15a8ebfe8480d9861c1285faaf7b5 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:01:29.964339  8811 tablet_service.cc:1511] Processing CreateTablet for tablet 987153bedd0c4f86b726db1edfd1eec6 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:01:29.966766  8820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50ae323a197c4e5b994d66663473c7d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.966275  8741 tablet_service.cc:1511] Processing CreateTablet for tablet 1f9ddea0a887446f910d38f8302e8172 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:01:29.971007  8815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41d15a8ebfe8480d9861c1285faaf7b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.971657  8662 tablet_service.cc:1511] Processing CreateTablet for tablet 7d6f9069613f4a83a64061e85b6f97da (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:01:29.973063  8662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d6f9069613f4a83a64061e85b6f97da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.966346  8819 tablet_service.cc:1511] Processing CreateTablet for tablet 7fbae1ebcfaf4be58c5aecd1014cea01 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:01:29.983577  8743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fb0366ec71a44feb8c7b426e851dcc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.987358  8746 tablet_service.cc:1511] Processing CreateTablet for tablet 142891c077a34ba289adf90a99aee612 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:01:29.988138  8741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f9ddea0a887446f910d38f8302e8172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.986749  8811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 987153bedd0c4f86b726db1edfd1eec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.990975  8738 tablet_service.cc:1511] Processing CreateTablet for tablet c01533e2a5204b1ebccac1133e495542 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:01:29.992170  8818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3796428796f4d6c8fd8e9ff2897d116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.992535  8738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c01533e2a5204b1ebccac1133e495542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.995229  8740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20fcd6d0c3304abda83f416721e807c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.986027  8874 tablet_bootstrap.cc:492] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:30.004170  8746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142891c077a34ba289adf90a99aee612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.008064  8870 ts_tablet_manager.cc:1434] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.161s	user 0.020s	sys 0.005s
I20260709 13:01:30.008288  8819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fbae1ebcfaf4be58c5aecd1014cea01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.009085  8855 heartbeater.cc:499] Master 127.7.89.190:45099 was elected leader, sending a full tablet report...
I20260709 13:01:29.986990  8745 tablet_service.cc:1511] Processing CreateTablet for tablet 149c2129693643678d9da7765768609c (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:01:30.006755  8812 tablet_service.cc:1511] Processing CreateTablet for tablet 4f19e578ddc64100a13ab07a6b877a77 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:01:30.009465  8870 tablet_bootstrap.cc:492] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:30.010607  8812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f19e578ddc64100a13ab07a6b877a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.012360  8816 tablet_service.cc:1511] Processing CreateTablet for tablet 51388ea026cc4d098949a4a0327a35fc (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:01:30.014014  8816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51388ea026cc4d098949a4a0327a35fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.031184  8745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 149c2129693643678d9da7765768609c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:29.977919  8737 tablet_service.cc:1511] Processing CreateTablet for tablet 8acd9840694e4f62803bfe9810425202 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:01:30.035625  8737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8acd9840694e4f62803bfe9810425202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.035820  8876 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.036578  8876 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:30.038538  8781 heartbeater.cc:499] Master 127.7.89.190:45099 was elected leader, sending a full tablet report...
I20260709 13:01:30.039335  8871 tablet_bootstrap.cc:492] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:30.039948  8871 ts_tablet_manager.cc:1403] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.277s	user 0.005s	sys 0.024s
W20260709 13:01:30.051681  8566 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 66289.7us
W20260709 13:01:30.063356  8564 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 52470.2us
I20260709 13:01:30.070216  8744 tablet_service.cc:1511] Processing CreateTablet for tablet 8e9173dea5bf4398a5e138cb273df07c (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:01:30.086270  8871 raft_consensus.cc:359] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } }
I20260709 13:01:30.087189  8871 raft_consensus.cc:385] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.087502  8871 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.088335  8871 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } }
I20260709 13:01:30.099645  8744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e9173dea5bf4398a5e138cb273df07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.112932  8666 tablet_service.cc:1511] Processing CreateTablet for tablet 8611228be8dd4f08b2b609e202974a6a (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:01:30.132194  8742 tablet_service.cc:1511] Processing CreateTablet for tablet 95fdf24d372f40b8877c2792102f34cf (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:01:30.133922  8742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95fdf24d372f40b8877c2792102f34cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.138831  8814 tablet_service.cc:1511] Processing CreateTablet for tablet 1c963cd681ab4f45a5c339f9fd6e2687 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:01:30.149341  8666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8611228be8dd4f08b2b609e202974a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.144110  8743 tablet_service.cc:1511] Processing CreateTablet for tablet 55c31a55ecab4a17acb00373b55d6f1d (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:01:30.116214  8876 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:30.138871  8874 tablet_bootstrap.cc:654] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.163133  8743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c31a55ecab4a17acb00373b55d6f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.167876  8871 ts_tablet_manager.cc:1434] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.127s	user 0.010s	sys 0.003s
I20260709 13:01:30.169202  8871 tablet_bootstrap.cc:492] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:30.181026  8664 tablet_service.cc:1511] Processing CreateTablet for tablet c3796428796f4d6c8fd8e9ff2897d116 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:01:30.181604  8668 tablet_service.cc:1511] Processing CreateTablet for tablet 67bfc59e963f4ad0ac128fd11eecac17 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:01:30.182601  8664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3796428796f4d6c8fd8e9ff2897d116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.184849  8870 tablet_bootstrap.cc:654] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.184849  8670 tablet_service.cc:1511] Processing CreateTablet for tablet 50ae323a197c4e5b994d66663473c7d2 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:01:30.194597  8813 tablet_service.cc:1511] Processing CreateTablet for tablet 293a9dab5be3479a90d5803585682a00 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:01:30.197237  8668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67bfc59e963f4ad0ac128fd11eecac17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.200788  8819 tablet_service.cc:1511] Processing CreateTablet for tablet b88e9b3a77fe404e8e4caaefe74a04a2 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:01:30.212142  8814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c963cd681ab4f45a5c339f9fd6e2687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.221359  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293a9dab5be3479a90d5803585682a00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.228364  8818 tablet_service.cc:1511] Processing CreateTablet for tablet ac4bab9153b047238bffb6abbeb83601 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:01:30.230080  8818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac4bab9153b047238bffb6abbeb83601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.239058  8819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88e9b3a77fe404e8e4caaefe74a04a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.254516  8669 tablet_service.cc:1511] Processing CreateTablet for tablet ff71213103664c0bab63c42773d0880b (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:01:30.258870  8737 tablet_service.cc:1511] Processing CreateTablet for tablet 67bfc59e963f4ad0ac128fd11eecac17 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:01:30.260434  8670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50ae323a197c4e5b994d66663473c7d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.265437  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:30.266065  8871 tablet_bootstrap.cc:654] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.266499  8669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff71213103664c0bab63c42773d0880b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:30.267444  8794 leader_election.cc:343] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.269702  8738 tablet_service.cc:1511] Processing CreateTablet for tablet 50ae323a197c4e5b994d66663473c7d2 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:01:30.275846  8746 tablet_service.cc:1511] Processing CreateTablet for tablet ff71213103664c0bab63c42773d0880b (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:01:30.279942  8662 tablet_service.cc:1511] Processing CreateTablet for tablet 41d15a8ebfe8480d9861c1285faaf7b5 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:01:30.251643  8820 tablet_service.cc:1511] Processing CreateTablet for tablet bc61bbf8aae94862a95ef23cde2e06e8 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:01:30.280531  8671 tablet_service.cc:1511] Processing CreateTablet for tablet 987153bedd0c4f86b726db1edfd1eec6 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:01:30.211308  8739 tablet_service.cc:1511] Processing CreateTablet for tablet 7d6f9069613f4a83a64061e85b6f97da (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:01:30.289001  8665 tablet_service.cc:1511] Processing CreateTablet for tablet 7fbae1ebcfaf4be58c5aecd1014cea01 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:01:30.260434  8737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67bfc59e963f4ad0ac128fd11eecac17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.291599  8815 tablet_service.cc:1511] Processing CreateTablet for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:01:30.306871  8665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fbae1ebcfaf4be58c5aecd1014cea01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.308151  8671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 987153bedd0c4f86b726db1edfd1eec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.308518  8739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d6f9069613f4a83a64061e85b6f97da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.309938  8738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50ae323a197c4e5b994d66663473c7d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.314743  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:30.316442  8746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff71213103664c0bab63c42773d0880b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:30.316938  8793 leader_election.cc:343] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.317471  8793 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:30.318821  8876 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:30.318934  8870 tablet_bootstrap.cc:492] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:30.319864  8870 ts_tablet_manager.cc:1403] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.311s	user 0.021s	sys 0.011s
I20260709 13:01:30.319937  8815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.334263  8745 tablet_service.cc:1511] Processing CreateTablet for tablet c3796428796f4d6c8fd8e9ff2897d116 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:01:30.336058  8745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3796428796f4d6c8fd8e9ff2897d116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.328713  8870 raft_consensus.cc:359] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:30.340399  8870 raft_consensus.cc:385] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.340731  8870 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.341325  8871 tablet_bootstrap.cc:492] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:30.341871  8871 ts_tablet_manager.cc:1403] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.173s	user 0.017s	sys 0.006s
I20260709 13:01:30.341666  8870 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:30.344305  8870 ts_tablet_manager.cc:1434] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.003s
I20260709 13:01:30.345405  8870 tablet_bootstrap.cc:492] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:30.345124  8871 raft_consensus.cc:359] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:30.345932  8871 raft_consensus.cc:385] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.346258  8871 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.253474  8741 tablet_service.cc:1511] Processing CreateTablet for tablet 8611228be8dd4f08b2b609e202974a6a (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:01:30.347654  8741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8611228be8dd4f08b2b609e202974a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.347802  8876 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.348315  8876 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:30.352057  8876 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:30.351964  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:30.353785  8794 leader_election.cc:343] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.351966  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:30.354730  8812 tablet_service.cc:1511] Processing CreateTablet for tablet 364ad2097a2341cdb96752b9a370f9a9 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 13:01:30.356089  8793 leader_election.cc:343] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.356559  8793 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:30.357995  8876 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:30.360213  8811 tablet_service.cc:1511] Processing CreateTablet for tablet 389e9e63581e4d1b941ba995f8cdbfc4 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:01:30.360910  8662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41d15a8ebfe8480d9861c1285faaf7b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.350008  8740 tablet_service.cc:1511] Processing CreateTablet for tablet 41d15a8ebfe8480d9861c1285faaf7b5 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:01:30.375624  8740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41d15a8ebfe8480d9861c1285faaf7b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.367648  8813 tablet_service.cc:1511] Processing CreateTablet for tablet ae08d8f5271e475baa4344843405b254 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:01:30.368024  8820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc61bbf8aae94862a95ef23cde2e06e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.383049  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae08d8f5271e475baa4344843405b254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.360226  8744 tablet_service.cc:1511] Processing CreateTablet for tablet 51388ea026cc4d098949a4a0327a35fc (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:01:30.387982  8744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51388ea026cc4d098949a4a0327a35fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.369602  8814 tablet_service.cc:1511] Processing CreateTablet for tablet 300460316b0849bf8014930d5e9bfada (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:01:30.392074  8814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 300460316b0849bf8014930d5e9bfada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.391930  8666 tablet_service.cc:1511] Processing CreateTablet for tablet 51388ea026cc4d098949a4a0327a35fc (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:01:30.393049  8812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 364ad2097a2341cdb96752b9a370f9a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.399206  8817 tablet_service.cc:1511] Processing CreateTablet for tablet 614067c154284cfa9b7a01100f304cc9 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:01:30.400859  8817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 614067c154284cfa9b7a01100f304cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.393484  8666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51388ea026cc4d098949a4a0327a35fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.405299  8874 tablet_bootstrap.cc:492] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:30.405953  8874 ts_tablet_manager.cc:1403] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.421s	user 0.011s	sys 0.014s
I20260709 13:01:30.405838  8816 tablet_service.cc:1511] Processing CreateTablet for tablet 1e6257b62c5c415aad31f5fc646ce4d4 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:01:30.407430  8816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e6257b62c5c415aad31f5fc646ce4d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.409528  8874 raft_consensus.cc:359] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:30.418390  8811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389e9e63581e4d1b941ba995f8cdbfc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.419263  8871 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:30.421801  8871 ts_tablet_manager.cc:1434] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.080s	user 0.003s	sys 0.003s
I20260709 13:01:30.410434  8874 raft_consensus.cc:385] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.422318  8874 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.425529  8663 tablet_service.cc:1511] Processing CreateTablet for tablet 293a9dab5be3479a90d5803585682a00 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:01:30.429581  8818 tablet_service.cc:1511] Processing CreateTablet for tablet abd86f4e9b544531a7cbfa0042b4fa6c (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:01:30.362529  8667 tablet_service.cc:1511] Processing CreateTablet for tablet 4f19e578ddc64100a13ab07a6b877a77 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:01:30.431874  8743 tablet_service.cc:1511] Processing CreateTablet for tablet 987153bedd0c4f86b726db1edfd1eec6 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:01:30.433411  8743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 987153bedd0c4f86b726db1edfd1eec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.437935  8669 tablet_service.cc:1511] Processing CreateTablet for tablet 1c963cd681ab4f45a5c339f9fd6e2687 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:01:30.451464  8670 tablet_service.cc:1511] Processing CreateTablet for tablet b88e9b3a77fe404e8e4caaefe74a04a2 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:01:30.451578  8874 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:30.452158  8818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd86f4e9b544531a7cbfa0042b4fa6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.453130  8870 tablet_bootstrap.cc:654] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.454660  8663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293a9dab5be3479a90d5803585682a00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.457647  8664 tablet_service.cc:1511] Processing CreateTablet for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:01:30.463447  8879 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.451498  8871 tablet_bootstrap.cc:492] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:30.464094  8879 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } }
I20260709 13:01:30.474077  8742 tablet_service.cc:1511] Processing CreateTablet for tablet 1c963cd681ab4f45a5c339f9fd6e2687 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:01:30.496461  8664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.497814  8667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f19e578ddc64100a13ab07a6b877a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.498397  8668 tablet_service.cc:1511] Processing CreateTablet for tablet bc61bbf8aae94862a95ef23cde2e06e8 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:01:30.499863  8668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc61bbf8aae94862a95ef23cde2e06e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.499953  8737 tablet_service.cc:1511] Processing CreateTablet for tablet 4f19e578ddc64100a13ab07a6b877a77 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:01:30.501892  8669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c963cd681ab4f45a5c339f9fd6e2687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.503012  8662 tablet_service.cc:1511] Processing CreateTablet for tablet ac4bab9153b047238bffb6abbeb83601 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:01:30.479419  8879 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:30.514755  8670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88e9b3a77fe404e8e4caaefe74a04a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.532328  8871 tablet_bootstrap.cc:654] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.547356  8662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac4bab9153b047238bffb6abbeb83601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.526037  8665 tablet_service.cc:1511] Processing CreateTablet for tablet ae08d8f5271e475baa4344843405b254 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:01:30.551751  8665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae08d8f5271e475baa4344843405b254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.535810  8815 tablet_service.cc:1511] Processing CreateTablet for tablet 35e078746b204711a1e15f221c16e31a (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:01:30.564507  8739 tablet_service.cc:1511] Processing CreateTablet for tablet b88e9b3a77fe404e8e4caaefe74a04a2 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:01:30.579586  8746 tablet_service.cc:1511] Processing CreateTablet for tablet 7fbae1ebcfaf4be58c5aecd1014cea01 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:01:30.580415  8741 tablet_service.cc:1511] Processing CreateTablet for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:01:30.440418  8819 tablet_service.cc:1511] Processing CreateTablet for tablet 3a7eadf5c488447886a90612fbf1a2bb (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:01:30.582892  8671 tablet_service.cc:1511] Processing CreateTablet for tablet 389e9e63581e4d1b941ba995f8cdbfc4 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:01:30.529978  8742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c963cd681ab4f45a5c339f9fd6e2687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.605472  8737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f19e578ddc64100a13ab07a6b877a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.606917  8746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fbae1ebcfaf4be58c5aecd1014cea01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.613062  8738 tablet_service.cc:1511] Processing CreateTablet for tablet ac4bab9153b047238bffb6abbeb83601 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:01:30.614899  8738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac4bab9153b047238bffb6abbeb83601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.615154  8819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a7eadf5c488447886a90612fbf1a2bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.618275  8815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e078746b204711a1e15f221c16e31a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.619400  8739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88e9b3a77fe404e8e4caaefe74a04a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.620060  8812 tablet_service.cc:1511] Processing CreateTablet for tablet 3c997cc62c984d6e8962403382a451bb (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:01:30.621556  8812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c997cc62c984d6e8962403382a451bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.631883  8820 tablet_service.cc:1511] Processing CreateTablet for tablet 7e3bac87e89140ffa2dcde662c7fb621 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:01:30.633662  8820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e3bac87e89140ffa2dcde662c7fb621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.639652  8663 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 33) took 1006 ms. Trace:
I20260709 13:01:30.640280  8663 rpcz_store.cc:276] 0709 13:01:29.633769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.424417 (+790648us) service_pool.cc:224] Handling call
0709 13:01:30.454046 (+ 29629us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.454187 (+   141us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.547531 (+ 93344us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:30.638988 (+ 91457us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:30.639568 (+   580us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.657008  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:30.658155  8744 tablet_service.cc:1511] Processing CreateTablet for tablet 293a9dab5be3479a90d5803585682a00 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:01:30.667619  8744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293a9dab5be3479a90d5803585682a00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.675797  8663 tablet_service.cc:1511] Processing CreateTablet for tablet 614067c154284cfa9b7a01100f304cc9 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:01:30.677320  8663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 614067c154284cfa9b7a01100f304cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.677515  8871 tablet_bootstrap.cc:492] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:30.678155  8871 ts_tablet_manager.cc:1403] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.227s	user 0.003s	sys 0.013s
W20260709 13:01:30.681128  8627 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:30.670331  8745 tablet_service.cc:1511] Processing CreateTablet for tablet bc61bbf8aae94862a95ef23cde2e06e8 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:01:30.671489  8740 tablet_service.cc:1511] Processing CreateTablet for tablet ae08d8f5271e475baa4344843405b254 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:01:30.664620  8811 tablet_service.cc:1511] Processing CreateTablet for tablet 396dbfb60a3644b9b28c39c3dd36b322 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
W20260709 13:01:30.659018  8645 leader_election.cc:343] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.682485  8745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc61bbf8aae94862a95ef23cde2e06e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.687638  8740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae08d8f5271e475baa4344843405b254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.687744  8811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 396dbfb60a3644b9b28c39c3dd36b322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.689584  8666 tablet_service.cc:1511] Processing CreateTablet for tablet 364ad2097a2341cdb96752b9a370f9a9 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:01:30.691190  8666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 364ad2097a2341cdb96752b9a370f9a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.674353  8814 tablet_service.cc:1511] Processing CreateTablet for tablet 53cce7f107584cc28999319a785023ef (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:01:30.693511  8874 ts_tablet_manager.cc:1434] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.287s	user 0.050s	sys 0.062s
I20260709 13:01:30.693986  8814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53cce7f107584cc28999319a785023ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.694947  8874 tablet_bootstrap.cc:492] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:30.698184  8671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389e9e63581e4d1b941ba995f8cdbfc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.672576  8739 tablet_service.cc:1511] Processing CreateTablet for tablet 389e9e63581e4d1b941ba995f8cdbfc4 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:01:30.699882  8739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389e9e63581e4d1b941ba995f8cdbfc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.700997  8741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.697511  8743 tablet_service.cc:1511] Processing CreateTablet for tablet 364ad2097a2341cdb96752b9a370f9a9 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:01:30.705199  8743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 364ad2097a2341cdb96752b9a370f9a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.693801  8816 tablet_service.cc:1511] Processing CreateTablet for tablet 4de6a8f976f5458d807b777f3ac94a23 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:01:30.708714  8816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4de6a8f976f5458d807b777f3ac94a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.698078  8668 tablet_service.cc:1511] Processing CreateTablet for tablet 300460316b0849bf8014930d5e9bfada (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:01:30.714919  8871 raft_consensus.cc:359] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:30.734357  8818 tablet_service.cc:1511] Processing CreateTablet for tablet fea1a7951ca9443d8f4d05bd1519d442 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:01:30.739907  8884 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.726171  8871 raft_consensus.cc:385] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.741057  8813 tablet_service.cc:1511] Processing CreateTablet for tablet 9c86a8d1b46a4dad9149c50458be45d2 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:01:30.771164  8871 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.771994  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c86a8d1b46a4dad9149c50458be45d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.772114  8871 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:30.746999  8566 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 55177.1us
I20260709 13:01:30.747159  8817 tablet_service.cc:1511] Processing CreateTablet for tablet 3169f7f9f5a7440398ae5ebe01fc3c3f (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:01:30.775301  8884 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:30.757675  8815 tablet_service.cc:1511] Processing CreateTablet for tablet 304eb26fa48d4971bffbe6266846243b (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:01:30.767674  8668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 300460316b0849bf8014930d5e9bfada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.772150  8738 tablet_service.cc:1511] Processing CreateTablet for tablet 614067c154284cfa9b7a01100f304cc9 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:01:30.787292  8738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 614067c154284cfa9b7a01100f304cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.776988  8871 ts_tablet_manager.cc:1434] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.096s	user 0.007s	sys 0.003s
I20260709 13:01:30.784162  8884 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:30.806931  8664 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 35) took 1132 ms. Trace:
I20260709 13:01:30.811811  8879 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.809537  8815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304eb26fa48d4971bffbe6266846243b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.813583  8670 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 34) took 1139 ms. Trace:
I20260709 13:01:30.806595  8874 tablet_bootstrap.cc:654] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:30.815493  8667 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 30) took 1221 ms. Trace:
I20260709 13:01:30.812803  8664 rpcz_store.cc:276] 0709 13:01:29.674719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.456573 (+781854us) service_pool.cc:224] Handling call
0709 13:01:30.458597 (+  2024us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.480283 (+ 21686us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.654814 (+174531us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:30.750576 (+ 95762us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:30.806860 (+ 56284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":271}
I20260709 13:01:30.852870  8664 tablet_service.cc:1511] Processing CreateTablet for tablet 1e6257b62c5c415aad31f5fc646ce4d4 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:01:30.810601  8745 tablet_service.cc:1511] Processing CreateTablet for tablet 1e6257b62c5c415aad31f5fc646ce4d4 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:01:30.859351  8870 tablet_bootstrap.cc:492] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:30.860013  8870 ts_tablet_manager.cc:1403] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.515s	user 0.015s	sys 0.014s
I20260709 13:01:30.860127  8745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e6257b62c5c415aad31f5fc646ce4d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.840893  8871 tablet_bootstrap.cc:492] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:30.814225  8670 rpcz_store.cc:276] 0709 13:01:29.674274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.450222 (+775948us) service_pool.cc:224] Handling call
0709 13:01:30.514180 (+ 63958us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.514294 (+   114us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.608717 (+ 94423us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:30.752713 (+143996us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:30.800782 (+ 48069us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d51a2 00007f32e8316bb9 00007f32e7447d88 00007f32e750e973 00007f32e750a70f 00007f32e750aecd 00007f32e74b9400 00007f32e093eaf4 00007f32e093ea82
0709 13:01:30.813491 (+ 12709us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12267,"spinlock_wait_cycles":37126912}
I20260709 13:01:30.864759  8670 tablet_service.cc:1511] Processing CreateTablet for tablet abd86f4e9b544531a7cbfa0042b4fa6c (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:01:30.867363  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:30.816495  8667 rpcz_store.cc:276] 0709 13:01:29.594019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.361525 (+767506us) service_pool.cc:224] Handling call
0709 13:01:30.492271 (+130746us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.497201 (+  4930us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.685124 (+187923us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:30.793638 (+108514us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:30.815345 (+ 21707us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.873198  8665 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 38) took 1110 ms. Trace:
I20260709 13:01:30.873656  8665 rpcz_store.cc:276] 0709 13:01:29.762705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.524710 (+762005us) service_pool.cc:224] Handling call
0709 13:01:30.551066 (+ 26356us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.551193 (+   127us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.678800 (+127607us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:30.872610 (+193810us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:30.873130 (+   520us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.876937  8737 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39018 (request call id 30) took 1069 ms. Trace:
I20260709 13:01:30.877384  8737 rpcz_store.cc:276] 0709 13:01:29.807247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.498923 (+691676us) service_pool.cc:224] Handling call
0709 13:01:30.500798 (+  1875us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.500909 (+   111us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.645926 (+145017us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:30.876389 (+230463us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:30.876866 (+   477us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.872887  8667 tablet_service.cc:1511] Processing CreateTablet for tablet 3a7eadf5c488447886a90612fbf1a2bb (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:01:30.807770  8741 tablet_service.cc:1511] Processing CreateTablet for tablet 300460316b0849bf8014930d5e9bfada (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:01:30.879796  8741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 300460316b0849bf8014930d5e9bfada. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:30.879693  8647 leader_election.cc:343] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.865154  8818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fea1a7951ca9443d8f4d05bd1519d442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.854414  8664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e6257b62c5c415aad31f5fc646ce4d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.897270  8742 tablet_service.cc:1511] Processing CreateTablet for tablet 3a7eadf5c488447886a90612fbf1a2bb (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:01:30.898293  8667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a7eadf5c488447886a90612fbf1a2bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.813007  8879 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:30.880587  8647 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:30.895246  8817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3169f7f9f5a7440398ae5ebe01fc3c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.902364  8670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd86f4e9b544531a7cbfa0042b4fa6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.931752  8744 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39018 (request call id 33) took 1123 ms. Trace:
I20260709 13:01:30.932274  8744 rpcz_store.cc:276] 0709 13:01:29.808116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.635757 (+827641us) service_pool.cc:224] Handling call
0709 13:01:30.667025 (+ 31268us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.667150 (+   125us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.805617 (+138467us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:30.918655 (+113038us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:30.931670 (+ 13015us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.934513  8744 tablet_service.cc:1511] Processing CreateTablet for tablet 35e078746b204711a1e15f221c16e31a (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:01:30.935268  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:30.927227  8879 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:01:30.937099  8645 leader_election.cc:343] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:30.937389  8879 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:30.949640  8662 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 36) took 1254 ms. Trace:
I20260709 13:01:30.950738  8744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e078746b204711a1e15f221c16e31a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.952992  8669 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 32) took 1331 ms. Trace:
I20260709 13:01:30.953467  8669 rpcz_store.cc:276] 0709 13:01:29.622055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.436800 (+814745us) service_pool.cc:224] Handling call
0709 13:01:30.484276 (+ 47476us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.484402 (+   126us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.749236 (+264834us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:30.952373 (+203137us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:30.952918 (+   545us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4864}
I20260709 13:01:30.957396  8664 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 43) took 1117 ms. Trace:
I20260709 13:01:30.958861  8664 rpcz_store.cc:276] 0709 13:01:29.839708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.851707 (+1011999us) service_pool.cc:224] Handling call
0709 13:01:30.853872 (+  2165us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.853978 (+   106us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.941168 (+ 87190us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:30.955580 (+ 14412us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:30.957297 (+  1717us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.962024  8663 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 41) took 1176 ms. Trace:
I20260709 13:01:30.967218  8665 tablet_service.cc:1511] Processing CreateTablet for tablet 35e078746b204711a1e15f221c16e31a (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:01:30.968863  8665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e078746b204711a1e15f221c16e31a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.968719  8819 tablet_service.cc:1511] Processing CreateTablet for tablet 13d1c43d45594a8d9dc9c717fd6e4ca7 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:01:30.970237  8819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13d1c43d45594a8d9dc9c717fd6e4ca7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.918659  8870 raft_consensus.cc:359] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:30.918591  8737 tablet_service.cc:1511] Processing CreateTablet for tablet abd86f4e9b544531a7cbfa0042b4fa6c (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:01:30.975306  8870 raft_consensus.cc:385] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:30.975869  8870 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:30.962513  8663 rpcz_store.cc:276] 0709 13:01:29.785398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.674852 (+889454us) service_pool.cc:224] Handling call
0709 13:01:30.676748 (+  1896us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.676867 (+   119us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.797719 (+120852us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:30.961377 (+163658us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:30.961953 (+   576us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.976267  8737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd86f4e9b544531a7cbfa0042b4fa6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.979712  8739 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39018 (request call id 39) took 1169 ms. Trace:
I20260709 13:01:30.980187  8739 rpcz_store.cc:276] 0709 13:01:29.810067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.671619 (+861552us) service_pool.cc:224] Handling call
0709 13:01:30.699211 (+ 27592us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.699357 (+   146us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.851859 (+152502us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:30.979123 (+127264us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:30.979637 (+   514us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.983251  8669 tablet_service.cc:1511] Processing CreateTablet for tablet 9c86a8d1b46a4dad9149c50458be45d2 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:01:30.985106  8669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c86a8d1b46a4dad9149c50458be45d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.950237  8662 rpcz_store.cc:276] 0709 13:01:29.695577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.481604 (+786027us) service_pool.cc:224] Handling call
0709 13:01:30.546671 (+ 65067us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.546843 (+   172us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.725847 (+179004us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:30.949053 (+223206us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:30.949564 (+   511us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443648}
I20260709 13:01:30.987149  8743 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39018 (request call id 40) took 1176 ms. Trace:
I20260709 13:01:30.987814  8743 rpcz_store.cc:276] 0709 13:01:29.810404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.696327 (+885923us) service_pool.cc:224] Handling call
0709 13:01:30.704530 (+  8203us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.704680 (+   150us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.746308 (+ 41628us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:30.986454 (+240146us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:30.987071 (+   617us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.949966  8812 tablet_service.cc:1511] Processing CreateTablet for tablet 844a3710898640c49df7cfb8d1c568cf (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:01:30.993854  8742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a7eadf5c488447886a90612fbf1a2bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.994210  8812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 844a3710898640c49df7cfb8d1c568cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:30.977111  8870 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:30.983131  8663 tablet_service.cc:1511] Processing CreateTablet for tablet 7e3bac87e89140ffa2dcde662c7fb621 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:01:31.003602  8663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e3bac87e89140ffa2dcde662c7fb621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.004192  8871 tablet_bootstrap.cc:654] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.009663  8739 tablet_service.cc:1511] Processing CreateTablet for tablet 3c997cc62c984d6e8962403382a451bb (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
W20260709 13:01:31.012454  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 133011us
I20260709 13:01:31.023831  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:01:31.025975  8647 leader_election.cc:343] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.026547  8647 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:31.027848  8893 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.028483  8745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39018 (request call id 43) took 1167 ms. Trace:
I20260709 13:01:31.028913  8745 rpcz_store.cc:276] 0709 13:01:29.861521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.809575 (+948054us) service_pool.cc:224] Handling call
0709 13:01:30.859550 (+ 49975us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.859665 (+   115us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.963944 (+104279us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:31.027888 (+ 63944us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:31.028416 (+   528us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.031524  8874 tablet_bootstrap.cc:492] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:31.032152  8874 ts_tablet_manager.cc:1403] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.337s	user 0.015s	sys 0.026s
I20260709 13:01:31.035183  8746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39018 (request call id 29) took 1228 ms. Trace:
I20260709 13:01:31.037300  8814 tablet_service.cc:1511] Processing CreateTablet for tablet a4f0aa5ad2ef496d9a459b6742346f0a (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:01:30.992406  8662 tablet_service.cc:1511] Processing CreateTablet for tablet 396dbfb60a3644b9b28c39c3dd36b322 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:01:31.048020  8745 tablet_service.cc:1511] Processing CreateTablet for tablet 7e3bac87e89140ffa2dcde662c7fb621 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:01:31.043795  8739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c997cc62c984d6e8962403382a451bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.049393  8743 tablet_service.cc:1511] Processing CreateTablet for tablet 9c86a8d1b46a4dad9149c50458be45d2 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:01:31.050103  8745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e3bac87e89140ffa2dcde662c7fb621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.060534  8664 tablet_service.cc:1511] Processing CreateTablet for tablet 3c997cc62c984d6e8962403382a451bb (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:01:31.066672  8870 ts_tablet_manager.cc:1434] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.206s	user 0.008s	sys 0.004s
I20260709 13:01:31.068128  8870 tablet_bootstrap.cc:492] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:31.085112  8668 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 42) took 1292 ms. Trace:
I20260709 13:01:31.085646  8668 rpcz_store.cc:276] 0709 13:01:29.792546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.697109 (+904563us) service_pool.cc:224] Handling call
0709 13:01:30.767100 (+ 69991us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.767224 (+   124us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.972405 (+205181us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:31.084440 (+112035us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:31.085023 (+   583us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.087050  8743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c86a8d1b46a4dad9149c50458be45d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.041780  8746 rpcz_store.cc:276] 0709 13:01:29.806939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.493922 (+686983us) service_pool.cc:224] Handling call
0709 13:01:30.606326 (+112404us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.606446 (+   120us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.682587 (+ 76141us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:31.034451 (+351864us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:31.035103 (+   652us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1653632}
I20260709 13:01:31.092309  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:31.093281  8746 tablet_service.cc:1511] Processing CreateTablet for tablet 396dbfb60a3644b9b28c39c3dd36b322 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:01:31.094287  8671 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 39) took 1330 ms. Trace:
W20260709 13:01:31.094935  8719 leader_election.cc:343] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.096596  8669 tablet_service.cc:1511] Processing CreateTablet for tablet 53cce7f107584cc28999319a785023ef (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:01:31.099206  8671 rpcz_store.cc:276] 0709 13:01:29.764316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.581836 (+817520us) service_pool.cc:224] Handling call
0709 13:01:30.647843 (+ 66007us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.652800 (+  4957us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.967525 (+314725us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:31.093610 (+126085us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:31.094213 (+   603us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.103312  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:31.130110  8671 tablet_service.cc:1511] Processing CreateTablet for tablet fea1a7951ca9443d8f4d05bd1519d442 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:01:31.105484  8666 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 40) took 1337 ms. Trace:
I20260709 13:01:31.132476  8746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 396dbfb60a3644b9b28c39c3dd36b322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.106606  8814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4f0aa5ad2ef496d9a459b6742346f0a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:31.132865  8722 leader_election.cc:343] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.133430  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:31.134770  8884 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.105788  8745 tablet_service.cc:1511] Processing CreateTablet for tablet 53cce7f107584cc28999319a785023ef (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:01:31.107394  8668 tablet_service.cc:1511] Processing CreateTablet for tablet 4de6a8f976f5458d807b777f3ac94a23 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:01:31.136886  8745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53cce7f107584cc28999319a785023ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.110991  8874 raft_consensus.cc:359] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.117558  8664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c997cc62c984d6e8962403382a451bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.124123  8871 tablet_bootstrap.cc:492] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:31.138346  8871 ts_tablet_manager.cc:1403] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.325s	user 0.014s	sys 0.009s
I20260709 13:01:31.138674  8668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4de6a8f976f5458d807b777f3ac94a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.141888  8669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53cce7f107584cc28999319a785023ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.143078  8662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 396dbfb60a3644b9b28c39c3dd36b322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.146222  8893 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.146879  8893 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.146983  8744 tablet_service.cc:1511] Processing CreateTablet for tablet 4de6a8f976f5458d807b777f3ac94a23 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:01:31.148522  8744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4de6a8f976f5458d807b777f3ac94a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.151554  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:31.153358  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:31.151690  8893 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:31.132333  8666 rpcz_store.cc:276] 0709 13:01:29.768207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.595314 (+827107us) service_pool.cc:224] Handling call
0709 13:01:30.690587 (+ 95273us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.690734 (+   147us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:30.972267 (+281533us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:31.104950 (+132683us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:31.105412 (+   462us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:31.155452  8647 leader_election.cc:343] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:31.156028  8645 leader_election.cc:343] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.156491  8645 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:31.157977  8893 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.157740  8666 tablet_service.cc:1511] Processing CreateTablet for tablet 3169f7f9f5a7440398ae5ebe01fc3c3f (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:01:31.160053  8740 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39018 (request call id 38) took 1350 ms. Trace:
I20260709 13:01:31.160552  8740 rpcz_store.cc:276] 0709 13:01:29.809720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.670465 (+860745us) service_pool.cc:224] Handling call
0709 13:01:30.686800 (+ 16335us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.686921 (+   121us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:31.074273 (+387352us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:31.159429 (+ 85156us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:31.159976 (+   547us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.162578  8874 raft_consensus.cc:385] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.163036  8874 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.164938  8665 tablet_service.cc:1511] Processing CreateTablet for tablet 304eb26fa48d4971bffbe6266846243b (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:01:31.165760  8671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fea1a7951ca9443d8f4d05bd1519d442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.105945  8737 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39018 (request call id 44) took 1141 ms. Trace:
I20260709 13:01:31.142669  8871 raft_consensus.cc:359] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.167177  8871 raft_consensus.cc:385] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.174818  8871 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.170979  8874 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.179616  8665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304eb26fa48d4971bffbe6266846243b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.180207  8871 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.192582  8670 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 44) took 1244 ms. Trace:
I20260709 13:01:31.177289  8663 tablet_service.cc:1511] Processing CreateTablet for tablet 844a3710898640c49df7cfb8d1c568cf (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:01:31.166994  8737 rpcz_store.cc:276] 0709 13:01:29.964749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.878479 (+913730us) service_pool.cc:224] Handling call
0709 13:01:30.975567 (+ 97088us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.975721 (+   154us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:31.076080 (+100359us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:31.105391 (+ 29311us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:31.105876 (+   485us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.193166  8670 rpcz_store.cc:276] 0709 13:01:29.948651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.863651 (+915000us) service_pool.cc:224] Handling call
0709 13:01:30.865682 (+  2031us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.865784 (+   102us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:31.077358 (+211574us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:31.191891 (+114533us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:31.192493 (+   602us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.193344  8874 ts_tablet_manager.cc:1434] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.161s	user 0.006s	sys 0.004s
I20260709 13:01:31.195540  8737 tablet_service.cc:1511] Processing CreateTablet for tablet 304eb26fa48d4971bffbe6266846243b (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:01:31.197098  8737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304eb26fa48d4971bffbe6266846243b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.197984  8871 ts_tablet_manager.cc:1434] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.059s	user 0.003s	sys 0.005s
I20260709 13:01:31.211366  8874 tablet_bootstrap.cc:492] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:31.173496  8741 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39018 (request call id 42) took 1317 ms. Trace:
I20260709 13:01:31.183303  8666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3169f7f9f5a7440398ae5ebe01fc3c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.215539  8663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 844a3710898640c49df7cfb8d1c568cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.214989  8741 rpcz_store.cc:276] 0709 13:01:29.856592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.792292 (+935700us) service_pool.cc:224] Handling call
0709 13:01:30.879294 (+ 87002us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.879391 (+    97us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:31.035797 (+156406us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:31.172923 (+137126us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:31.173426 (+   503us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.183698  8667 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 45) took 1123 ms. Trace:
I20260709 13:01:31.216176  8667 rpcz_store.cc:276] 0709 13:01:30.059916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.871826 (+811910us) service_pool.cc:224] Handling call
0709 13:01:30.879011 (+  7185us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.879120 (+   109us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:31.001076 (+121956us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:31.168256 (+167180us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:31.183631 (+ 15375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.173152  8739 tablet_service.cc:1511] Processing CreateTablet for tablet 3169f7f9f5a7440398ae5ebe01fc3c3f (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:01:31.218356  8667 tablet_service.cc:1511] Processing CreateTablet for tablet a4f0aa5ad2ef496d9a459b6742346f0a (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:01:31.219528  8739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3169f7f9f5a7440398ae5ebe01fc3c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.219702  8667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4f0aa5ad2ef496d9a459b6742346f0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.183498  8870 tablet_bootstrap.cc:700] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9: Time spent opening tablet: real 0.109s	user 0.002s	sys 0.002s
I20260709 13:01:31.223193  8870 tablet_bootstrap.cc:654] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.224370  8742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39018 (request call id 45) took 1127 ms. Trace:
I20260709 13:01:31.225266  8871 tablet_bootstrap.cc:492] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:31.213456  8740 tablet_service.cc:1511] Processing CreateTablet for tablet fea1a7951ca9443d8f4d05bd1519d442 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:01:31.226743  8740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fea1a7951ca9443d8f4d05bd1519d442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.178177  8738 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39018 (request call id 41) took 1367 ms. Trace:
I20260709 13:01:31.218546  8741 tablet_service.cc:1511] Processing CreateTablet for tablet 844a3710898640c49df7cfb8d1c568cf (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:01:31.233742  8741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 844a3710898640c49df7cfb8d1c568cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.230733  8738 rpcz_store.cc:276] 0709 13:01:29.855062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.769829 (+914767us) service_pool.cc:224] Handling call
0709 13:01:30.786733 (+ 16904us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.786869 (+   136us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:31.067885 (+281016us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:31.177476 (+109591us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:31.178103 (+   627us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.240173  8745 tablet_service.cc:1511] Processing CreateTablet for tablet 13d1c43d45594a8d9dc9c717fd6e4ca7 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:01:31.224809  8742 rpcz_store.cc:276] 0709 13:01:30.096903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.895673 (+798770us) service_pool.cc:224] Handling call
0709 13:01:30.924056 (+ 28383us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:30.929535 (+  5479us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:31.162921 (+233386us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:31.223840 (+ 60919us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:31.224304 (+   464us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.241853  8745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13d1c43d45594a8d9dc9c717fd6e4ca7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.243103  8738 tablet_service.cc:1511] Processing CreateTablet for tablet a4f0aa5ad2ef496d9a459b6742346f0a (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:01:31.259490  8738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4f0aa5ad2ef496d9a459b6742346f0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.264130  8884 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.264771  8884 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.267201  8884 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:31.268572  8874 tablet_bootstrap.cc:654] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.268675  8670 tablet_service.cc:1511] Processing CreateTablet for tablet 13d1c43d45594a8d9dc9c717fd6e4ca7 (DEFAULT_TABLE table=test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:01:31.269235  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:31.271159  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:31.281152  8670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13d1c43d45594a8d9dc9c717fd6e4ca7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.281406  8830 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
W20260709 13:01:31.283529  8719 leader_election.cc:343] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.284065  8871 tablet_bootstrap.cc:654] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.284732  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:01:31.286306  8884 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:31.286748  8884 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.287112  8884 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.273720  8870 tablet_bootstrap.cc:492] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:31.275453  8895 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.291069  8895 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.293771  8895 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:31.295205  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:31.295854  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:31.296643  8681 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
W20260709 13:01:31.296981  8794 leader_election.cc:343] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.298034  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:31.298902  8664 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53334 (request call id 47) took 1099 ms. Trace:
I20260709 13:01:31.299650  8895 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:31.300083  8895 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.300448  8895 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.319228  8870 ts_tablet_manager.cc:1403] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.251s	user 0.011s	sys 0.007s
I20260709 13:01:31.329353  8895 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.299382  8664 rpcz_store.cc:276] 0709 13:01:30.199230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.040369 (+841139us) service_pool.cc:224] Handling call
0709 13:01:31.061898 (+ 21529us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:01:31.062043 (+   145us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:01:31.240103 (+178060us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:31.298242 (+ 58139us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:31.298831 (+   589us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.338035  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:31.339788  8871 tablet_bootstrap.cc:492] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
W20260709 13:01:31.340188  8794 leader_election.cc:343] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.340711  8871 ts_tablet_manager.cc:1403] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.116s	user 0.024s	sys 0.007s
I20260709 13:01:31.334003  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:31.341856  8681 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.344833  8871 raft_consensus.cc:359] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.345813  8871 raft_consensus.cc:385] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.346186  8871 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.347236  8871 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.336541  8884 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.333205  8870 raft_consensus.cc:359] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.351294  8870 raft_consensus.cc:385] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.351622  8870 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.352429  8870 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.350759  8895 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:31.355495  8871 ts_tablet_manager.cc:1434] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 13:01:31.357043  8871 tablet_bootstrap.cc:492] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:31.361968  8681 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:31.363663  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:31.365656  8895 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:31.366196  8895 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:31.367177  8871 tablet_bootstrap.cc:654] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.367136  8895 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.371079  8874 tablet_bootstrap.cc:492] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:31.371824  8874 ts_tablet_manager.cc:1403] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.161s	user 0.033s	sys 0.023s
I20260709 13:01:31.375917  8870 ts_tablet_manager.cc:1434] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.047s	user 0.008s	sys 0.007s
I20260709 13:01:31.376601  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:31.377156  8870 tablet_bootstrap.cc:492] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:31.377219  8874 raft_consensus.cc:359] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.378089  8874 raft_consensus.cc:385] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.378408  8874 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
W20260709 13:01:31.378983  8719 leader_election.cc:343] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.380887  8874 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.386174  8871 tablet_bootstrap.cc:492] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:31.386886  8871 ts_tablet_manager.cc:1403] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.030s	user 0.023s	sys 0.004s
I20260709 13:01:31.390272  8871 raft_consensus.cc:359] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.391144  8871 raft_consensus.cc:385] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.391235  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:31.391537  8871 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.391906  8830 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.392423  8871 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.401377  8870 tablet_bootstrap.cc:654] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.404748  8871 ts_tablet_manager.cc:1434] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.017s	user 0.002s	sys 0.005s
I20260709 13:01:31.406109  8871 tablet_bootstrap.cc:492] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:31.399494  8884 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:31.411943  8830 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:31.413381  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:01:31.414763  8884 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:31.415319  8884 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:31.416260  8884 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.420099  8579 catalog_manager.cc:5697] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:31.422878  8871 tablet_bootstrap.cc:654] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.425005  8874 ts_tablet_manager.cc:1434] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.053s	user 0.022s	sys 0.017s
I20260709 13:01:31.426295  8874 tablet_bootstrap.cc:492] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:31.434373  8874 tablet_bootstrap.cc:654] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.446449  8870 tablet_bootstrap.cc:492] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:31.447240  8870 ts_tablet_manager.cc:1403] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.070s	user 0.015s	sys 0.007s
I20260709 13:01:31.447768  8871 tablet_bootstrap.cc:492] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:31.448318  8871 ts_tablet_manager.cc:1403] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.043s	user 0.008s	sys 0.008s
I20260709 13:01:31.447595  8578 catalog_manager.cc:5697] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:31.451965  8871 raft_consensus.cc:359] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.452703  8871 raft_consensus.cc:385] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.453014  8871 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.452834  8870 raft_consensus.cc:359] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.454066  8870 raft_consensus.cc:385] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.453933  8871 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.454547  8870 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.459743  8870 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.461365  8874 tablet_bootstrap.cc:492] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:31.462110  8874 ts_tablet_manager.cc:1403] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.012s
I20260709 13:01:31.462889  8870 ts_tablet_manager.cc:1434] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:01:31.464124  8870 tablet_bootstrap.cc:492] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:31.460631  8871 ts_tablet_manager.cc:1434] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.004s
I20260709 13:01:31.465541  8871 tablet_bootstrap.cc:492] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:31.468052  8874 raft_consensus.cc:359] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.469041  8874 raft_consensus.cc:385] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.469465  8874 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.470485  8874 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.471139  8893 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.471984  8870 tablet_bootstrap.cc:654] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.471693  8893 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.473764  8871 tablet_bootstrap.cc:654] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.473764  8874 ts_tablet_manager.cc:1434] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:01:31.475389  8874 tablet_bootstrap.cc:492] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:31.477257  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:31.479329  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:31.481928  8645 leader_election.cc:343] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:31.484393  8647 leader_election.cc:343] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.485168  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:31.486083  8874 tablet_bootstrap.cc:654] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.489307  8893 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:31.490321  8893 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.504281  8884 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.505261  8884 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.509871  8884 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:31.512440  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:31.513578  8681 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:31.515215  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:31.516281  8830 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:31.518117  8722 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:31.519320  8884 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:31.519876  8884 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.520323  8884 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.524873  8902 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.525857  8902 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.532750  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:31.534135  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:01:31.537474  8722 leader_election.cc:343] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:31.539004  8719 leader_election.cc:343] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.538571  8884 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.539557  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:31.540752  8902 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:31.541324  8902 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.541706  8871 tablet_bootstrap.cc:492] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:31.541890  8884 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:31.542594  8871 ts_tablet_manager.cc:1403] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.077s	user 0.033s	sys 0.041s
I20260709 13:01:31.543944  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:31.544974  8830 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.547295  8871 raft_consensus.cc:359] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.548532  8871 raft_consensus.cc:385] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.546416  8874 tablet_bootstrap.cc:492] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:31.549053  8871 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.549371  8874 ts_tablet_manager.cc:1403] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.074s	user 0.022s	sys 0.015s
I20260709 13:01:31.550143  8871 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.552348  8870 tablet_bootstrap.cc:492] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:31.552958  8870 ts_tablet_manager.cc:1403] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.089s	user 0.039s	sys 0.033s
I20260709 13:01:31.553256  8871 ts_tablet_manager.cc:1434] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:01:31.558135  8871 tablet_bootstrap.cc:492] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:31.554754  8874 raft_consensus.cc:359] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.559587  8874 raft_consensus.cc:385] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.560026  8874 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.560916  8874 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.562497  8870 raft_consensus.cc:359] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } }
I20260709 13:01:31.563521  8870 raft_consensus.cc:385] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.563675  8874 ts_tablet_manager.cc:1434] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:01:31.564025  8870 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.558878  8830 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:31.565011  8870 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } }
I20260709 13:01:31.566143  8722 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:31.567152  8884 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:31.567663  8884 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:31.567855  8870 ts_tablet_manager.cc:1434] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 13:01:31.565011  8874 tablet_bootstrap.cc:492] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:31.568589  8884 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.569530  8870 tablet_bootstrap.cc:492] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:31.574052  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:31.574918  8681 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.576741  8874 tablet_bootstrap.cc:654] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.577958  8870 tablet_bootstrap.cc:654] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.582712  8871 tablet_bootstrap.cc:654] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.589408  8913 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.590106  8913 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.594213  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:31.594688  8681 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:31.595125  8756 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:31.596627  8870 tablet_bootstrap.cc:492] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:31.597314  8870 ts_tablet_manager.cc:1403] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.000s
I20260709 13:01:31.597124  8645 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:31.597692  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:31.598244  8893 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:31.598393  8830 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:31.598706  8893 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.599104  8893 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.600240  8913 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:31.601336  8870 raft_consensus.cc:359] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.602187  8870 raft_consensus.cc:385] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.602556  8870 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.594321  8874 tablet_bootstrap.cc:492] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:31.603989  8874 ts_tablet_manager.cc:1403] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.011s
I20260709 13:01:31.603439  8870 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.606315  8870 ts_tablet_manager.cc:1434] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:01:31.607538  8870 tablet_bootstrap.cc:492] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:31.607779  8874 raft_consensus.cc:359] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.608731  8874 raft_consensus.cc:385] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.609076  8874 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.610090  8874 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.612864  8874 ts_tablet_manager.cc:1434] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:01:31.613448  8871 tablet_bootstrap.cc:492] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:31.614041  8871 ts_tablet_manager.cc:1403] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.056s	user 0.013s	sys 0.011s
I20260709 13:01:31.614046  8874 tablet_bootstrap.cc:492] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:31.615885  8870 tablet_bootstrap.cc:654] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.617110  8871 raft_consensus.cc:359] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.618041  8871 raft_consensus.cc:385] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.618369  8871 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.619228  8871 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.622430  8871 ts_tablet_manager.cc:1434] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:01:31.623551  8871 tablet_bootstrap.cc:492] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:31.625725  8874 tablet_bootstrap.cc:654] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.628159  8870 tablet_bootstrap.cc:492] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:31.628825  8870 ts_tablet_manager.cc:1403] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.007s
I20260709 13:01:31.628536  8578 catalog_manager.cc:5697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:31.630565  8871 tablet_bootstrap.cc:654] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.632223  8870 raft_consensus.cc:359] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.633090  8870 raft_consensus.cc:385] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.633471  8870 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.634388  8870 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.637162  8870 ts_tablet_manager.cc:1434] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:01:31.638451  8870 tablet_bootstrap.cc:492] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:31.647928  8870 tablet_bootstrap.cc:654] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.648141  8893 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.650734  8874 tablet_bootstrap.cc:492] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:31.651450  8874 ts_tablet_manager.cc:1403] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.005s
I20260709 13:01:31.652047  8871 tablet_bootstrap.cc:492] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:31.652585  8871 ts_tablet_manager.cc:1403] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.011s
I20260709 13:01:31.653854  8893 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:31.655463  8871 raft_consensus.cc:359] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.655170  8874 raft_consensus.cc:359] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } }
I20260709 13:01:31.655371  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:31.657759  8871 raft_consensus.cc:385] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.658159  8874 raft_consensus.cc:385] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.658700  8756 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.658903  8874 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.659926  8874 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } }
I20260709 13:01:31.660876  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:31.661506  8830 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.662422  8874 ts_tablet_manager.cc:1434] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:01:31.663614  8874 tablet_bootstrap.cc:492] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:31.658718  8871 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.667804  8871 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.668563  8870 tablet_bootstrap.cc:492] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:31.669202  8870 ts_tablet_manager.cc:1403] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.000s
I20260709 13:01:31.670410  8830 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:31.671968  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:31.673017  8893 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:31.672780  8870 raft_consensus.cc:359] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.673569  8893 raft_consensus.cc:697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:31.673688  8870 raft_consensus.cc:385] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.674077  8870 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.674508  8893 consensus_queue.cc:237] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.675266  8874 tablet_bootstrap.cc:654] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.674854  8870 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.670586  8871 ts_tablet_manager.cc:1434] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:01:31.679771  8871 tablet_bootstrap.cc:492] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:31.685959  8870 ts_tablet_manager.cc:1434] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
W20260709 13:01:31.686585  8627 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:31.687781  8870 tablet_bootstrap.cc:492] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:31.688417  8913 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.688935  8913 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.691226  8913 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:31.693948  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:31.694113  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:31.694870  8830 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 1: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:01:31.696750  8647 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:31.697332  8870 tablet_bootstrap.cc:654] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.697978  8913 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.707160  8913 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.711643  8871 tablet_bootstrap.cc:654] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.712152  8870 tablet_bootstrap.cc:492] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:31.712960  8870 ts_tablet_manager.cc:1403] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.008s
I20260709 13:01:31.715775  8895 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.716634  8895 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.719156  8884 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.719991  8884 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } }
I20260709 13:01:31.722127  8756 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:31.724725  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:31.728300  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:31.732815  8870 raft_consensus.cc:359] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.733716  8870 raft_consensus.cc:385] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.734057  8870 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.734735  8874 tablet_bootstrap.cc:492] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:31.735312  8874 ts_tablet_manager.cc:1403] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.072s	user 0.028s	sys 0.037s
I20260709 13:01:31.735030  8870 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.737272  8870 ts_tablet_manager.cc:1434] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.001s
I20260709 13:01:31.738368  8874 raft_consensus.cc:359] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.739053  8874 raft_consensus.cc:385] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.739357  8874 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.730283  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:31.741001  8870 tablet_bootstrap.cc:492] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:31.740749  8874 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.741520  8680 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:31.743664  8719 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:31.746644  8916 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:31.747081  8916 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.747414  8916 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.739457  8830 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:31.746407  8900 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.731513  8884 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221)
I20260709 13:01:31.737592  8578 catalog_manager.cc:5697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:31.740007  8793 leader_election.cc:343] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.771088  8916 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } }
I20260709 13:01:31.761027  8874 ts_tablet_manager.cc:1434] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:01:31.774214  8874 tablet_bootstrap.cc:492] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:31.740181  8895 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:31.768090  8900 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.775018  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:31.776999  8794 leader_election.cc:343] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.777313  8900 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:31.777576  8794 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:31.778558  8870 tablet_bootstrap.cc:654] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.780036  8900 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.780130  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:31.782022  8916 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221)
W20260709 13:01:31.782055  8794 leader_election.cc:343] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.783499  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:31.784006  8871 tablet_bootstrap.cc:492] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:31.784184  8680 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.784519  8871 ts_tablet_manager.cc:1403] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.105s	user 0.024s	sys 0.024s
I20260709 13:01:31.785034  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:31.787573  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:31.787616  8871 raft_consensus.cc:359] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.788494  8830 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:31.787545  8793 leader_election.cc:343] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.790433  8793 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:31.791774  8870 tablet_bootstrap.cc:492] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:31.792273  8870 ts_tablet_manager.cc:1403] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.002s
I20260709 13:01:31.793166  8900 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.793887  8874 tablet_bootstrap.cc:654] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.795279  8870 raft_consensus.cc:359] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.796077  8870 raft_consensus.cc:385] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.796370  8870 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.797048  8870 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.799000  8870 ts_tablet_manager.cc:1434] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.002s
I20260709 13:01:31.799974  8870 tablet_bootstrap.cc:492] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:31.801537  8874 tablet_bootstrap.cc:492] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:31.801645  8680 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:31.802088  8874 ts_tablet_manager.cc:1403] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.002s
I20260709 13:01:31.803663  8719 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:31.804747  8916 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:31.805719  8830 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:31.806095  8916 raft_consensus.cc:697] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:31.806486  8870 tablet_bootstrap.cc:654] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.788519  8871 raft_consensus.cc:385] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.807005  8871 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.807727  8871 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.807554  8916 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } }
I20260709 13:01:31.810230  8871 ts_tablet_manager.cc:1434] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:01:31.811734  8874 raft_consensus.cc:359] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.812500  8874 raft_consensus.cc:385] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.812777  8874 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.817618  8871 tablet_bootstrap.cc:492] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:31.818150  8874 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.821691  8874 ts_tablet_manager.cc:1434] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.005s
I20260709 13:01:31.822739  8874 tablet_bootstrap.cc:492] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:31.832469  8900 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.833390  8900 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.837706  8756 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.841457  8893 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.842377  8893 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.845549  8893 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:31.847240  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:31.847750  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:31.848093  8755 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:31.848425  8830 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:31.853217  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:31.854516  8893 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:31.855002  8893 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.855401  8893 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.857369  8913 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.858166  8913 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.861238  8913 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:31.861586  8578 catalog_manager.cc:5697] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:31.864043  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:31.864435  8871 tablet_bootstrap.cc:654] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.864686  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:01:31.868000  8645 leader_election.cc:343] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:31.870980  8647 leader_election.cc:343] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:31.871415  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:31.872192  8913 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:31.874409  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:31.879091  8895 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:31.877674  8884 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.876279  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:31.879436  8680 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:31.880115  8755 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:31.879962  8884 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.879266  8874 tablet_bootstrap.cc:654] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:31.881539  8794 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:31.883826  8870 tablet_bootstrap.cc:492] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:31.883773  8895 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:31.874698  8900 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:31.884397  8895 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.884826  8870 ts_tablet_manager.cc:1403] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.085s	user 0.037s	sys 0.031s
I20260709 13:01:31.885448  8895 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.878544  8893 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.887660  8893 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:31.891666  8884 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:31.899129  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:31.899833  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:31.900556  8829 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.898382  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:31.902316  8680 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:31.902786  8900 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.903316  8900 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.903968  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:31.906157  8900 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:31.898459  8870 raft_consensus.cc:359] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.907220  8870 raft_consensus.cc:385] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:31.907639  8870 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:31.908987  8870 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.912911  8870 ts_tablet_manager.cc:1434] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.001s
I20260709 13:01:31.918917  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:31.919368  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:31.919818  8680 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:31.920321  8754 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.894251  8895 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:31.921247  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:31.922200  8900 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:31.922592  8900 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.924052  8900 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.923858  8895 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:31.925197  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:31.927099  8753 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.928644  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:31.929821  8870 tablet_bootstrap.cc:492] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:31.929888  8680 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:31.930773  8884 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:31.932674  8884 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:01:31.933158  8884 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:31.934111  8895 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:31.945199  8916 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:31.939064  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:31.947209  8756 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:31.955228  8829 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:31.970427  8647 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:31.957247  8754 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:31.991567  8893 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:31.996807  8893 raft_consensus.cc:697] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:31.996807  8902 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:31.996167  8874 tablet_bootstrap.cc:492] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:31.998551  8874 ts_tablet_manager.cc:1403] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.176s	user 0.016s	sys 0.000s
I20260709 13:01:31.972151  8895 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.997855  8893 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.006785  8902 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:32.001760  8895 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.014014  8895 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:32.021060  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:32.021653  8874 raft_consensus.cc:359] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.022719  8874 raft_consensus.cc:385] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.023108  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:32.023139  8874 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.022317  8677 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:32.026154  8874 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.029750  8754 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:32.031143  8753 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:32.032788  8794 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:32.033620  8871 tablet_bootstrap.cc:492] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:32.033982  8794 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:32.034340  8871 ts_tablet_manager.cc:1403] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.217s	user 0.014s	sys 0.012s
I20260709 13:01:32.037390  8895 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:32.039197  8895 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.023903  8680 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:32.041353  8874 ts_tablet_manager.cc:1434] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 13:01:32.043134  8874 tablet_bootstrap.cc:492] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:32.051002  8870 tablet_bootstrap.cc:654] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.055184  8930 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:32.055830  8930 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:32.057029  8930 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.039662  8895 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.052776  8871 raft_consensus.cc:359] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.069363  8871 raft_consensus.cc:385] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.075408  8871 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.078438  8871 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.085376  8871 ts_tablet_manager.cc:1434] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.003s
I20260709 13:01:32.086438  8871 tablet_bootstrap.cc:492] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:32.099869  8944 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.100953  8944 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.112892  8902 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.119817  8871 tablet_bootstrap.cc:654] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.125409  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:32.119414  8944 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:01:32.127148  8794 leader_election.cc:343] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.125684  8884 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.122282  8902 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.141463  8756 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:32.143682  8884 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:32.144759  8913 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.145367  8913 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.156926  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:32.157799  8678 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:32.165632  8895 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.161283  8913 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:32.168713  8895 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:32.172345  8902 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:32.172706  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:32.173364  8756 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.159636  8944 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:01:32.175710  8944 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:32.179526  8944 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:32.178485  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:32.165341  8900 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.180487  8944 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.182163  8944 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:32.185075  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:32.185811  8681 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:32.188810  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:32.188894  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:32.189592  8753 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.189814  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:32.189916  8828 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:32.190521  8754 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:32.183311  8900 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:32.193197  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:32.194916  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:32.198021  8647 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:32.202035  8893 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:32.202467  8893 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.202831  8893 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.205308  8874 tablet_bootstrap.cc:654] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.207193  8871 tablet_bootstrap.cc:492] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:32.207929  8871 ts_tablet_manager.cc:1403] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.122s	user 0.041s	sys 0.014s
I20260709 13:01:32.198022  8900 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:32.211246  8900 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.211661  8900 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.216672  8913 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:32.217082  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:32.217909  8679 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.218873  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:32.220781  8952 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:32.221249  8952 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.221625  8952 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.214741  8681 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.216154  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:32.266960  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:32.268278  8829 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:32.273305  8871 raft_consensus.cc:359] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.274133  8871 raft_consensus.cc:385] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.274447  8871 raft_consensus.cc:740] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.275905  8871 consensus_queue.cc:260] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.278013  8871 ts_tablet_manager.cc:1434] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.001s
I20260709 13:01:32.279304  8900 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.281914  8753 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:32.282997  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:32.283751  8678 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.285588  8871 tablet_bootstrap.cc:492] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:32.292681  8794 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:32.295244  8756 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:32.297385  8870 tablet_bootstrap.cc:492] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:32.297195  8794 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:32.297989  8870 ts_tablet_manager.cc:1403] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.368s	user 0.037s	sys 0.030s
I20260709 13:01:32.298151  8902 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:32.298378  8913 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:01:32.302302  8944 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:32.298857  8893 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.302980  8944 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:32.303999  8944 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.305001  8893 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:32.310298  8578 catalog_manager.cc:5697] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:32.343063  8950 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 13:01:32.299954  8930 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:32.311050  8679 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:32.378554  8870 raft_consensus.cc:359] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.366773  8900 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:32.368175  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:32.402287  8870 raft_consensus.cc:385] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.402820  8870 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.403723  8870 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.381325  8930 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:32.418718  8870 ts_tablet_manager.cc:1434] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.120s	user 0.016s	sys 0.003s
I20260709 13:01:32.419090  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:32.423089  8827 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.430667  8870 tablet_bootstrap.cc:492] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:32.401934  8952 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.440889  8952 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:32.442210  8930 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.458973  8874 tablet_bootstrap.cc:492] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:32.500181  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:32.504519  8681 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:32.511907  8756 raft_consensus.cc:2393] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 1: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:01:32.511641  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:32.513137  8680 raft_consensus.cc:2393] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 1: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:01:32.520164  8874 ts_tablet_manager.cc:1403] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.473s	user 0.020s	sys 0.017s
I20260709 13:01:32.527221  8900 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.527139  8913 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.528460  8900 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.528791  8913 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.527040  8893 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:32.532863  8794 leader_election.cc:343] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.536564  8952 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:32.540958  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:32.542878  8893 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.547327  8722 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:32.548481  8900 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:32.548619  8874 raft_consensus.cc:359] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.558733  8874 raft_consensus.cc:385] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.559823  8874 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.549567  8893 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:32.550701  8913 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:32.550173  8952 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:32.552317  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:32.553630  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:32.555395  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:32.564011  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:32.564742  8830 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:01:32.574023  8871 tablet_bootstrap.cc:654] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.572851  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:32.596501  8824 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
W20260709 13:01:32.601002  8647 leader_election.cc:343] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:32.603811  8794 leader_election.cc:343] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.604789  8647 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:32.565521  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:32.610881  8893 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:32.610572  8722 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:01:32.611385  8893 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.612367  8893 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.612181  8895 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:32.607595  8874 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.620191  8964 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 13:01:32.622584  8678 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:32.620736  8956 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:32.623958  8895 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:32.630425  8956 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:32.624765  8827 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
W20260709 13:01:32.628218  8793 leader_election.cc:343] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.631552  8793 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:32.624686  8752 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:32.632087  8956 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.633086  8895 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:32.633949  8962 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:01:32.635646  8647 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:32.636757  8913 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:01:32.640767  8645 leader_election.cc:343] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:32.641340  8645 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:32.654311  8870 tablet_bootstrap.cc:654] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.647425  8965 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:01:32.644232  8913 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:32.658015  8913 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.662556  8893 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.665970  8893 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:32.666749  8874 ts_tablet_manager.cc:1434] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.146s	user 0.005s	sys 0.007s
I20260709 13:01:32.692073  8965 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:32.700266  8960 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.107s	user 0.006s	sys 0.000s
I20260709 13:01:32.703529  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:32.705783  8930 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:32.706386  8930 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:32.707520  8930 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.719725  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:32.720494  8752 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.723690  8895 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:01:32.734480  8627 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:01:32.762082  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 120269us
I20260709 13:01:32.764760  8874 tablet_bootstrap.cc:492] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:32.773084  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:32.774659  8826 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.787386  8575 catalog_manager.cc:5697] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:32.808279  8968 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:01:32.812731  8752 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:32.819026  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:32.825354  8972 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:32.826102  8972 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:32.827627  8972 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.836974  8874 tablet_bootstrap.cc:654] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.837661  8870 tablet_bootstrap.cc:492] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:32.841308  8870 ts_tablet_manager.cc:1403] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.412s	user 0.018s	sys 0.011s
I20260709 13:01:32.843107  8575 catalog_manager.cc:5697] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:32.845461  8895 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:32.845338  8575 catalog_manager.cc:5697] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:32.848969  8870 raft_consensus.cc:359] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.849934  8870 raft_consensus.cc:385] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.850281  8870 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.868268  8870 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.871764  8870 ts_tablet_manager.cc:1434] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:01:32.851248  8826 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:32.874990  8870 tablet_bootstrap.cc:492] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:32.881669  8930 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:32.886026  8578 catalog_manager.cc:5697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:32.891294  8874 tablet_bootstrap.cc:492] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:32.894454  8874 ts_tablet_manager.cc:1403] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.163s	user 0.006s	sys 0.011s
I20260709 13:01:32.896584  8870 tablet_bootstrap.cc:654] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.893817  8871 tablet_bootstrap.cc:492] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:32.897363  8871 ts_tablet_manager.cc:1403] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.612s	user 0.040s	sys 0.025s
I20260709 13:01:32.901338  8576 catalog_manager.cc:5697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:32.902863  8871 raft_consensus.cc:359] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.905467  8871 raft_consensus.cc:385] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.906505  8871 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.907418  8871 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.909938  8871 ts_tablet_manager.cc:1434] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:01:32.901365  8874 raft_consensus.cc:359] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.910480  8874 raft_consensus.cc:385] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.910863  8874 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.911868  8874 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.914390  8874 ts_tablet_manager.cc:1434] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:01:32.917692  8874 tablet_bootstrap.cc:492] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:32.919168  8870 tablet_bootstrap.cc:492] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:32.916132  8871 tablet_bootstrap.cc:492] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:32.919752  8870 ts_tablet_manager.cc:1403] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.045s	user 0.014s	sys 0.006s
I20260709 13:01:32.920692  8944 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:32.922901  8870 raft_consensus.cc:359] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.923800  8870 raft_consensus.cc:385] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.924098  8870 raft_consensus.cc:740] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.924870  8870 consensus_queue.cc:260] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.927534  8870 ts_tablet_manager.cc:1434] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:01:32.928900  8870 tablet_bootstrap.cc:492] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:32.933079  8874 tablet_bootstrap.cc:654] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.936205  8870 tablet_bootstrap.cc:654] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.933856  8871 tablet_bootstrap.cc:654] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:32.940056  8952 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.940757  8952 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.944939  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:32.945240  8952 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:32.945890  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:32.946909  8681 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 1: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:01:32.952039  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:32.953693  8952 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:32.986004  8874 tablet_bootstrap.cc:492] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:32.986709  8874 ts_tablet_manager.cc:1403] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.069s	user 0.012s	sys 0.021s
I20260709 13:01:32.986824  8870 tablet_bootstrap.cc:492] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:32.987488  8870 ts_tablet_manager.cc:1403] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.059s	user 0.019s	sys 0.008s
I20260709 13:01:32.989840  8952 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:32.986791  8871 tablet_bootstrap.cc:492] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:32.992156  8871 ts_tablet_manager.cc:1403] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.076s	user 0.021s	sys 0.001s
I20260709 13:01:32.990036  8874 raft_consensus.cc:359] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.995057  8874 raft_consensus.cc:385] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.995016  8871 raft_consensus.cc:359] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.995527  8874 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.995887  8871 raft_consensus.cc:385] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:32.996186  8871 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:32.996490  8874 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.996886  8871 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:32.998809  8874 ts_tablet_manager.cc:1434] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:01:32.998940  8871 ts_tablet_manager.cc:1434] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:01:32.997937  8576 catalog_manager.cc:5697] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:33.000159  8874 tablet_bootstrap.cc:492] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:33.000394  8871 tablet_bootstrap.cc:492] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:32.990304  8870 raft_consensus.cc:359] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.003538  8870 raft_consensus.cc:385] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.003863  8870 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.007020  8870 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.007982  8874 tablet_bootstrap.cc:654] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.008234  8871 tablet_bootstrap.cc:654] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.009331  8870 ts_tablet_manager.cc:1434] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:01:33.010460  8870 tablet_bootstrap.cc:492] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:33.017331  8870 tablet_bootstrap.cc:654] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.023020  8871 tablet_bootstrap.cc:492] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:33.023228  8874 tablet_bootstrap.cc:492] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:33.023558  8871 ts_tablet_manager.cc:1403] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.011s
I20260709 13:01:33.023869  8874 ts_tablet_manager.cc:1403] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.008s
I20260709 13:01:33.026724  8871 raft_consensus.cc:359] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.027429  8871 raft_consensus.cc:385] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.027725  8871 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.028288  8874 raft_consensus.cc:359] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.029057  8874 raft_consensus.cc:385] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.028585  8871 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.029311  8874 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.028728  8578 catalog_manager.cc:5697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:33.030099  8874 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.031209  8871 ts_tablet_manager.cc:1434] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.005s
I20260709 13:01:33.032531  8871 tablet_bootstrap.cc:492] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:33.033305  8870 tablet_bootstrap.cc:492] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:33.033883  8870 ts_tablet_manager.cc:1403] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.000s
I20260709 13:01:33.037599  8870 raft_consensus.cc:359] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.038347  8870 raft_consensus.cc:385] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.038662  8870 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.039392  8870 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.043437  8874 ts_tablet_manager.cc:1434] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 13:01:33.044929  8874 tablet_bootstrap.cc:492] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:33.047937  8870 ts_tablet_manager.cc:1434] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.004s
I20260709 13:01:33.050567  8871 tablet_bootstrap.cc:654] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.055769  8870 tablet_bootstrap.cc:492] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:33.061930  8951 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.062565  8951 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.066757  8944 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.067581  8944 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.069571  8874 tablet_bootstrap.cc:654] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.069806  8951 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:33.073958  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:33.075311  8679 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:33.078881  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:33.079731  8826 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:33.079646  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:33.081382  8719 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:01:33.081521  8794 leader_election.cc:343] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.082027  8944 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:33.083214  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:33.083942  8679 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:33.085000  8951 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:33.085510  8951 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.085403  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:33.086100  8951 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.087934  8944 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:33.088485  8944 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.088971  8944 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.089710  8870 tablet_bootstrap.cc:654] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.098294  8871 tablet_bootstrap.cc:492] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:33.099354  8871 ts_tablet_manager.cc:1403] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.067s	user 0.019s	sys 0.018s
I20260709 13:01:33.098435  8951 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.102886  8871 raft_consensus.cc:359] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.103631  8871 raft_consensus.cc:385] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.103910  8871 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.104880  8956 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.104669  8871 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.105401  8956 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.106436  8871 ts_tablet_manager.cc:1434] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:01:33.108500  8956 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:33.109555  8871 tablet_bootstrap.cc:492] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:33.110832  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:33.111532  8681 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:33.111840  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:33.112534  8913 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.113204  8913 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:33.118005  8722 leader_election.cc:343] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.117991  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:33.119024  8913 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:33.119815  8871 tablet_bootstrap.cc:654] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:33.120335  8647 leader_election.cc:343] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.127955  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:33.132068  8871 tablet_bootstrap.cc:492] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:33.132635  8952 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:33.132696  8871 ts_tablet_manager.cc:1403] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.007s
I20260709 13:01:33.132720  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:33.133284  8952 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.133692  8952 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.135785  8874 tablet_bootstrap.cc:492] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:33.136371  8874 ts_tablet_manager.cc:1403] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.092s	user 0.028s	sys 0.038s
I20260709 13:01:33.136336  8871 raft_consensus.cc:359] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.137243  8871 raft_consensus.cc:385] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.137553  8871 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.139191  8826 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:33.138967  8871 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.139886  8874 raft_consensus.cc:359] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.140717  8874 raft_consensus.cc:385] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.141000  8874 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.141577  8871 ts_tablet_manager.cc:1434] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:01:33.141465  8956 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:33.142813  8871 tablet_bootstrap.cc:492] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:33.142654  8874 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.144127  8972 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:33.144600  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:33.145363  8679 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.154924  8874 ts_tablet_manager.cc:1434] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:01:33.153555  8913 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.161898  8913 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.164597  8930 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:33.172657  8913 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:33.152490  8944 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.174685  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:33.176118  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:33.176831  8681 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:33.177393  8647 leader_election.cc:343] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.178825  8874 tablet_bootstrap.cc:492] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:33.184441  8944 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:33.186205  8951 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:33.187628  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:33.188107  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:33.187644  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:33.190286  8826 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:33.190665  8645 leader_election.cc:343] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.191355  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:33.192082  8871 tablet_bootstrap.cc:654] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.194346  8913 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:33.195472  8794 leader_election.cc:343] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.213439  8870 tablet_bootstrap.cc:492] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:33.213979  8870 ts_tablet_manager.cc:1403] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.159s	user 0.021s	sys 0.013s
I20260709 13:01:33.217800  8944 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.218339  8944 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.222189  8944 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:33.222251  8826 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:33.223826  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:33.226747  8951 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:33.227290  8951 raft_consensus.cc:697] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:33.228191  8951 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.231674  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:33.249311  8681 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:33.248075  8679 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
W20260709 13:01:33.251881  8794 leader_election.cc:343] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.232080  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:33.253350  8677 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:33.253626  8678 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:33.257381  8874 tablet_bootstrap.cc:654] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.257189  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:33.258661  8930 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:33.258971  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:33.259174  8930 raft_consensus.cc:697] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:33.259930  8944 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:33.260360  8944 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.260083  8930 consensus_queue.cc:237] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.260807  8944 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.211521  8952 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.267482  8753 raft_consensus.cc:2393] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 1: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:01:33.269002  8913 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:33.230634  8870 raft_consensus.cc:359] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.274027  8870 raft_consensus.cc:385] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.274431  8870 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.275259  8870 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.277613  8870 ts_tablet_manager.cc:1434] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.000s
I20260709 13:01:33.284641  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:33.286324  8972 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.299109  8913 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:33.306010  8952 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:33.307086  8870 tablet_bootstrap.cc:492] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:33.289922  8944 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.308393  8972 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:33.309949  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:33.316421  8930 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:01:33.328146  8677 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:33.334954  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:01:33.343812  8722 leader_election.cc:343] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.344595  8871 tablet_bootstrap.cc:492] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:33.345431  8871 ts_tablet_manager.cc:1403] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.203s	user 0.017s	sys 0.013s
I20260709 13:01:33.354736  8679 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:33.339087  8951 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:01:33.357990  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:33.356995  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:33.361644  8677 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.360402  8944 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:33.366413  8871 raft_consensus.cc:359] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.369222  8871 raft_consensus.cc:385] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.368530  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:01:33.373641  8794 leader_election.cc:343] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.374826  8951 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:33.376008  8951 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:33.377414  8951 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.341435  8575 catalog_manager.cc:5697] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:33.369671  8871 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.388535  8870 tablet_bootstrap.cc:654] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.399216  8871 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.402549  8964 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.403231  8964 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:33.404167  9005 log.cc:927] Time spent T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:01:33.406526  8964 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:33.410066  8871 ts_tablet_manager.cc:1434] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.064s	user 0.012s	sys 0.000s
I20260709 13:01:33.408491  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:33.411263  8871 tablet_bootstrap.cc:492] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
W20260709 13:01:33.411991  8647 leader_election.cc:343] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.414608  8874 tablet_bootstrap.cc:492] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:33.415297  8874 ts_tablet_manager.cc:1403] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.237s	user 0.055s	sys 0.064s
I20260709 13:01:33.417891  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:33.422212  8645 leader_election.cc:343] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.422945  8645 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:33.424393  8913 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:33.436178  8964 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:33.436719  8944 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.437563  8944 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.441346  8874 raft_consensus.cc:359] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.442302  8874 raft_consensus.cc:385] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.442772  8874 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.444445  8874 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.445892  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:33.429024  8677 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:33.454226  8952 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.454960  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:33.454901  8952 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.456060  8874 ts_tablet_manager.cc:1434] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.001s
W20260709 13:01:33.456538  8793 leader_election.cc:343] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.458067  8874 tablet_bootstrap.cc:492] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:33.458155  8930 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:33.460382  8930 raft_consensus.cc:697] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:33.461381  8930 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.462157  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:33.459314  8952 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:33.459787  8956 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.467317  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:33.467276  8956 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.464457  8944 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:01:33.464648  8719 leader_election.cc:343] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.468091  8826 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:33.444007  8576 catalog_manager.cc:5697] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:33.466481  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:33.470156  8753 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:33.472015  8794 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:01:33.472373  8722 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:01:33.473722  8895 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:33.474290  8895 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.474792  8895 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.479089  8952 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:33.479823  8952 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.480321  8952 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.510281  8956 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:33.549691  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:33.550597  8829 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:33.541501  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:33.554225  8722 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:33.555322  8956 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:33.555801  8956 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.556146  8956 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.566042  8677 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:33.569303  8575 catalog_manager.cc:5697] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:33.583146  8874 tablet_bootstrap.cc:654] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.584280  8871 tablet_bootstrap.cc:654] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.585536  8870 tablet_bootstrap.cc:492] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:33.586057  8870 ts_tablet_manager.cc:1403] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.308s	user 0.019s	sys 0.015s
I20260709 13:01:33.588419  8895 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.590003  8870 raft_consensus.cc:359] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.591138  8870 raft_consensus.cc:385] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.591554  8870 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.593017  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:33.592821  8870 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.595570  8956 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.596045  8871 tablet_bootstrap.cc:492] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:33.596625  8871 ts_tablet_manager.cc:1403] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.186s	user 0.009s	sys 0.005s
I20260709 13:01:33.596401  8870 ts_tablet_manager.cc:1434] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 13:01:33.599215  8895 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:33.599566  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:33.600220  8680 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.600445  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:01:33.599233  8793 leader_election.cc:343] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.609278  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:33.610261  8829 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.611672  8680 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:33.607627  8952 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.612875  8756 raft_consensus.cc:2393] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 1: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:01:33.607638  8956 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:33.615422  8719 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:33.615492  8794 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:33.616802  8952 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:33.616834  8930 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:33.617370  8956 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:33.617856  8870 tablet_bootstrap.cc:492] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:33.618124  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:33.618757  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:01:33.620069  8719 leader_election.cc:343] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.620354  8826 raft_consensus.cc:2393] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 1: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:01:33.617947  8956 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:33.621654  8722 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:33.621762  8956 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.623051  8952 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:33.606971  8576 catalog_manager.cc:5697] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:33.624397  8930 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.625694  8874 tablet_bootstrap.cc:492] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:33.626119  8874 ts_tablet_manager.cc:1403] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.168s	user 0.015s	sys 0.017s
I20260709 13:01:33.624886  8930 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.629771  8829 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:33.630402  8874 raft_consensus.cc:359] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.631156  8874 raft_consensus.cc:385] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.631433  8874 raft_consensus.cc:740] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.634138  8871 raft_consensus.cc:359] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.635169  8871 raft_consensus.cc:385] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.635370  8871 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.634573  8874 consensus_queue.cc:260] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.636404  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:33.637087  8680 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:33.636436  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:33.638409  8874 ts_tablet_manager.cc:1434] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:01:33.639617  8793 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:01:33.640667  8794 leader_election.cc:343] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.641099  8895 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:33.641592  8895 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.632108  8930 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:33.642043  8895 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.647784  8871 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.648888  8874 tablet_bootstrap.cc:492] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:33.658367  8871 ts_tablet_manager.cc:1434] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.061s	user 0.002s	sys 0.005s
I20260709 13:01:33.661899  8871 tablet_bootstrap.cc:492] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:33.662077  8895 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.663125  8870 tablet_bootstrap.cc:654] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.665611  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:33.666443  8680 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.668695  8874 tablet_bootstrap.cc:654] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.671684  8871 tablet_bootstrap.cc:654] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.681828  8956 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:33.683676  8870 tablet_bootstrap.cc:492] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:33.684314  8870 ts_tablet_manager.cc:1403] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.076s	user 0.008s	sys 0.011s
I20260709 13:01:33.687602  8870 raft_consensus.cc:359] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.688424  8870 raft_consensus.cc:385] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.688570  8871 tablet_bootstrap.cc:492] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:33.688772  8870 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.689133  8871 ts_tablet_manager.cc:1403] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.003s
I20260709 13:01:33.690184  8870 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.692631  8680 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:33.692823  8871 raft_consensus.cc:359] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.695138  8870 ts_tablet_manager.cc:1434] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:01:33.694267  8793 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:33.694455  8578 catalog_manager.cc:5697] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:33.699294  8930 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:33.699863  8870 tablet_bootstrap.cc:492] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:33.700273  8930 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:33.701316  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:33.701190  8930 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.698786  8871 raft_consensus.cc:385] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.705432  8871 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.706350  8871 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.708701  8871 ts_tablet_manager.cc:1434] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:01:33.709792  8871 tablet_bootstrap.cc:492] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
W20260709 13:01:33.710639  8794 leader_election.cc:343] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.714068  8930 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:33.715478  8895 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:33.718354  8956 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:33.730957  8871 tablet_bootstrap.cc:654] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.732000  8874 tablet_bootstrap.cc:492] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:33.732574  8874 ts_tablet_manager.cc:1403] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.090s	user 0.010s	sys 0.012s
I20260709 13:01:33.739156  8874 raft_consensus.cc:359] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.740006  8874 raft_consensus.cc:385] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.740345  8874 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.741745  8874 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.742861  8870 tablet_bootstrap.cc:654] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.748880  8871 tablet_bootstrap.cc:492] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:33.749444  8871 ts_tablet_manager.cc:1403] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.006s
I20260709 13:01:33.751178  8874 ts_tablet_manager.cc:1434] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:01:33.752349  8874 tablet_bootstrap.cc:492] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:33.752579  8871 raft_consensus.cc:359] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.753391  8871 raft_consensus.cc:385] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.753674  8871 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.754407  8871 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.757020  8871 ts_tablet_manager.cc:1434] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:01:33.758220  8871 tablet_bootstrap.cc:492] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:33.759830  8756 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.771009  8577 catalog_manager.cc:5697] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:33.783006  8870 tablet_bootstrap.cc:492] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:33.783625  8870 ts_tablet_manager.cc:1403] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.084s	user 0.012s	sys 0.001s
I20260709 13:01:33.788008  8870 raft_consensus.cc:359] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.788770  8870 raft_consensus.cc:385] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.789058  8870 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.789789  8870 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.792394  8874 tablet_bootstrap.cc:654] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.792169  8930 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:33.793116  8870 ts_tablet_manager.cc:1434] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:01:33.796554  8871 tablet_bootstrap.cc:654] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:33.801534  8627 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:33.807547  8870 tablet_bootstrap.cc:492] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:33.814833  8913 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.815500  8913 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.838600  8913 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:33.841009  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:33.842777  8645 leader_election.cc:343] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.843237  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:01:33.845849  8647 leader_election.cc:343] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.846450  8647 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:33.848487  8893 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:33.849186  8870 tablet_bootstrap.cc:654] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.854061  8871 tablet_bootstrap.cc:492] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:33.854578  8871 ts_tablet_manager.cc:1403] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.097s	user 0.022s	sys 0.027s
I20260709 13:01:33.858507  8874 tablet_bootstrap.cc:492] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:33.859081  8874 ts_tablet_manager.cc:1403] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.107s	user 0.007s	sys 0.006s
I20260709 13:01:33.859150  8871 raft_consensus.cc:359] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.859997  8871 raft_consensus.cc:385] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.860262  8871 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.861001  8871 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.862123  8874 raft_consensus.cc:359] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.862864  8874 raft_consensus.cc:385] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.863121  8874 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.864923  8874 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.866773  8870 tablet_bootstrap.cc:492] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:33.867314  8871 ts_tablet_manager.cc:1434] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:01:33.868496  8871 tablet_bootstrap.cc:492] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:33.875025  8871 tablet_bootstrap.cc:654] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.876901  8930 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.877602  8870 ts_tablet_manager.cc:1403] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.070s	user 0.014s	sys 0.000s
I20260709 13:01:33.877521  8930 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:33.877980  8722 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Couldn't send request to peer e814ed2013244fa6be42623ea5c0c7b9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:33.879815  8930 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:33.880959  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:33.881273  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:33.881687  8680 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:33.881590  8870 raft_consensus.cc:359] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.882682  8870 raft_consensus.cc:385] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.882953  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:01:33.883668  8794 leader_election.cc:343] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.884142  8874 ts_tablet_manager.cc:1434] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.002s
I20260709 13:01:33.884594  8930 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:33.885048  8930 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.885288  8874 tablet_bootstrap.cc:492] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:33.885368  8930 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.886793  8871 tablet_bootstrap.cc:492] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:33.887306  8871 ts_tablet_manager.cc:1403] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.007s
I20260709 13:01:33.893486  8930 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.883069  8870 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.894999  8870 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.899642  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:33.900324  8680 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.901495  8870 ts_tablet_manager.cc:1434] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:01:33.902734  8870 tablet_bootstrap.cc:492] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:33.895279  8871 raft_consensus.cc:359] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.899286  8874 tablet_bootstrap.cc:654] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.906889  8871 raft_consensus.cc:385] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.907280  8871 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.907969  8871 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.909749  8871 ts_tablet_manager.cc:1434] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:01:33.912072  8871 tablet_bootstrap.cc:492] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:33.914022  8680 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:33.915180  8956 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.915977  8956 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.917063  8874 tablet_bootstrap.cc:492] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:33.917094  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:33.917881  8874 ts_tablet_manager.cc:1403] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.001s
I20260709 13:01:33.918450  8956 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:01:33.918996  8794 leader_election.cc:343] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:33.919735  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:33.920001  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:33.920460  8930 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:33.920785  8677 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:33.920492  8826 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:33.922314  8719 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:33.923144  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:33.923285  8956 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:33.923642  8874 raft_consensus.cc:359] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.923796  8956 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.926121  8956 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.927968  8871 tablet_bootstrap.cc:654] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.931419  8900 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:33.932080  8900 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:33.932754  8870 tablet_bootstrap.cc:654] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.933050  8900 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.935279  8956 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.935844  8874 raft_consensus.cc:385] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.936194  8874 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.936932  8874 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.939105  8874 ts_tablet_manager.cc:1434] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:01:33.940750  8874 tablet_bootstrap.cc:492] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:33.953702  8930 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:01:33.969364  8874 tablet_bootstrap.cc:654] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:33.971443  8956 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:33.972455  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:33.975057  8678 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.976919  8870 tablet_bootstrap.cc:492] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:33.977461  8870 ts_tablet_manager.cc:1403] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.075s	user 0.012s	sys 0.004s
I20260709 13:01:33.983295  8756 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:33.988510  8678 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:33.988224  8870 raft_consensus.cc:359] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.989620  8870 raft_consensus.cc:385] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.990016  8870 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.990808  8870 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.991235  8871 tablet_bootstrap.cc:492] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:33.991797  8871 ts_tablet_manager.cc:1403] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.080s	user 0.014s	sys 0.020s
I20260709 13:01:33.990032  8719 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:33.993474  8870 ts_tablet_manager.cc:1434] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.002s
I20260709 13:01:33.993611  8956 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:33.995316  8956 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:33.996220  8956 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.996935  8871 raft_consensus.cc:359] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:33.997663  8871 raft_consensus.cc:385] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:33.997943  8871 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:33.998497  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:33.999210  8826 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.004112  8870 tablet_bootstrap.cc:492] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:34.007683  8871 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.010372  8871 ts_tablet_manager.cc:1434] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:01:34.011694  8871 tablet_bootstrap.cc:492] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:34.015594  8576 catalog_manager.cc:5697] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:34.018018  8930 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:34.018091  8900 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.020563  8900 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.023016  8900 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:34.024609  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:34.025370  8678 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:34.026975  8793 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:34.027871  8900 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:34.028297  8900 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.028690  8900 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.029534  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:34.035154  8794 leader_election.cc:343] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.040536  9000 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:34.045531  8900 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.015657  8826 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:34.063686  8900 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:34.072010  8871 tablet_bootstrap.cc:654] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.085358  8951 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:34.091037  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:01:34.099467  8794 leader_election.cc:343] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.102473  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:34.103276  8677 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.104802  8870 tablet_bootstrap.cc:654] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.112006  8677 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:34.115156  8951 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:34.115852  8793 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:34.118566  8900 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:34.119132  8900 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:34.120038  8874 tablet_bootstrap.cc:492] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:34.120013  8900 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.120652  8874 ts_tablet_manager.cc:1403] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.180s	user 0.020s	sys 0.028s
I20260709 13:01:34.124049  8874 raft_consensus.cc:359] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.124881  8874 raft_consensus.cc:385] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.125175  8874 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.125808  8874 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.130153  8874 ts_tablet_manager.cc:1434] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:01:34.131765  8874 tablet_bootstrap.cc:492] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:34.136992  8871 tablet_bootstrap.cc:492] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:34.137601  8871 ts_tablet_manager.cc:1403] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.126s	user 0.011s	sys 0.007s
I20260709 13:01:34.142549  8871 raft_consensus.cc:359] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.147578  8874 tablet_bootstrap.cc:654] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.147650  8871 raft_consensus.cc:385] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.148156  8871 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.149060  8871 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.151212  8871 ts_tablet_manager.cc:1434] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.005s
I20260709 13:01:34.146795  8576 catalog_manager.cc:5697] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:34.152341  8871 tablet_bootstrap.cc:492] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:34.156028  8870 tablet_bootstrap.cc:492] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:34.156661  8895 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.158978  8895 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.161374  8871 tablet_bootstrap.cc:654] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.161505  8895 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:34.162478  8870 ts_tablet_manager.cc:1403] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.155s	user 0.014s	sys 0.012s
I20260709 13:01:34.163354  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:34.165292  8794 leader_election.cc:343] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.168999  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:34.169853  8678 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:34.173764  8874 tablet_bootstrap.cc:492] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:34.174371  8874 ts_tablet_manager.cc:1403] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.043s	user 0.020s	sys 0.000s
I20260709 13:01:34.175256  8793 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:34.177944  8874 raft_consensus.cc:359] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.179148  8874 raft_consensus.cc:385] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.179469  8874 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.180287  8900 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:34.180909  8900 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.180308  8874 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.181362  8900 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.183208  8874 ts_tablet_manager.cc:1434] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:01:34.185165  8874 tablet_bootstrap.cc:492] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:34.186771  8893 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.187335  8893 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.189625  8893 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:34.192399  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:01:34.194213  8647 leader_election.cc:343] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.195601  8893 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.196975  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:34.197378  8893 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.196379  8895 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.199095  8900 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.199654  8895 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.201797  8895 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:34.202114  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:34.202805  8680 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.203770  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:34.172683  8578 catalog_manager.cc:5697] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:34.197863  8753 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:34.194854  8870 raft_consensus.cc:359] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.205940  8870 raft_consensus.cc:385] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.206339  8870 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.206535  8893 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:34.207136  8870 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.207406  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:34.207860  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:34.209348  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:34.209677  8647 leader_election.cc:343] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.211639  8874 tablet_bootstrap.cc:654] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:34.212641  8645 leader_election.cc:343] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.213179  8645 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:34.213996  8964 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:34.214164  8870 ts_tablet_manager.cc:1434] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 13:01:34.214425  8964 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.214903  8964 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.216423  8870 tablet_bootstrap.cc:492] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:34.219437  8895 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.220005  8895 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.222308  8895 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:34.223021  8956 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.224687  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:34.224845  8956 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.227377  8956 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:01:34.228514  8794 leader_election.cc:343] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.229238  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:34.230122  8678 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 1: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:01:34.233992  8913 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:34.235481  8870 tablet_bootstrap.cc:654] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.236002  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:34.240540  8753 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.243873  8870 tablet_bootstrap.cc:492] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:34.244165  8893 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.244582  8870 ts_tablet_manager.cc:1403] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.000s
I20260709 13:01:34.248740  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:34.248088  8893 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.250407  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:34.247849  8870 raft_consensus.cc:359] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.251466  8870 raft_consensus.cc:385] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.251765  8870 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.252594  8870 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.255119  8870 ts_tablet_manager.cc:1434] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:01:34.256259  8870 tablet_bootstrap.cc:492] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:34.255908  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:34.257036  8826 raft_consensus.cc:2393] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 1: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:01:34.261545  8871 tablet_bootstrap.cc:492] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:34.261797  8964 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.262857  8871 ts_tablet_manager.cc:1403] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.111s	user 0.025s	sys 0.026s
I20260709 13:01:34.262838  8893 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:34.263736  8722 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:34.265236  8793 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:34.265863  8680 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:34.267704  8895 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:34.269909  8964 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:34.270053  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:34.270105  8871 raft_consensus.cc:359] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.270934  8871 raft_consensus.cc:385] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.271035  8756 raft_consensus.cc:2393] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 1: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:01:34.271131  8870 tablet_bootstrap.cc:654] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:34.271086  8793 leader_election.cc:343] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.271227  8871 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.271818  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:34.272177  8871 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.272722  8895 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:34.273116  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:34.274504  8871 ts_tablet_manager.cc:1434] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
W20260709 13:01:34.274741  8647 leader_election.cc:343] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.275645  8871 tablet_bootstrap.cc:492] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:34.279358  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:01:34.281329  8794 leader_election.cc:343] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.282075  8900 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:34.284904  8956 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:34.285374  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:34.286051  8756 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.286144  8645 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:34.287205  8893 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.283546  8871 tablet_bootstrap.cc:654] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.290740  8900 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:34.284394  8793 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:34.295425  8893 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:34.295849  8895 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:34.296418  8895 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:34.297349  8895 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.300402  8944 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:34.303960  8756 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:34.306789  8964 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.307420  8964 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.308305  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:34.310778  8913 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:34.311411  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:34.312250  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:34.313480  8647 leader_election.cc:343] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.317472  8874 tablet_bootstrap.cc:492] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:34.318187  8874 ts_tablet_manager.cc:1403] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.133s	user 0.017s	sys 0.003s
I20260709 13:01:34.319115  8964 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:01:34.319195  8645 leader_election.cc:343] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.320575  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:34.323088  8893 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:34.323784  8870 tablet_bootstrap.cc:492] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:34.324417  8870 ts_tablet_manager.cc:1403] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.068s	user 0.013s	sys 0.000s
I20260709 13:01:34.326018  8871 tablet_bootstrap.cc:492] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:34.326839  8871 ts_tablet_manager.cc:1403] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.004s
I20260709 13:01:34.329164  8913 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:34.330469  8913 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.331622  8871 raft_consensus.cc:359] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.332605  8871 raft_consensus.cc:385] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.332975  8871 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.333981  8871 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.336570  8870 raft_consensus.cc:359] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.337044  8871 ts_tablet_manager.cc:1434] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 13:01:34.338361  8871 tablet_bootstrap.cc:492] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:34.338209  8874 raft_consensus.cc:359] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.338963  8874 raft_consensus.cc:385] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.339233  8874 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.340021  8874 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.342682  8874 ts_tablet_manager.cc:1434] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:01:34.343736  8874 tablet_bootstrap.cc:492] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:34.344609  9053 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.337338  8870 raft_consensus.cc:385] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.345443  8870 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.346597  9053 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.350041  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:34.350836  8754 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:34.352881  8645 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:34.354226  8964 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:34.354696  8964 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.355067  8964 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.355706  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:34.356572  8871 tablet_bootstrap.cc:654] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.354219  9053 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:34.356511  8826 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:34.357969  8874 tablet_bootstrap.cc:654] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.346352  8870 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.367426  8829 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.374069  8944 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.374887  8754 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.374979  8944 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.380964  8944 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:34.383126  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:34.383958  8680 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:34.387475  8793 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:34.389294  8944 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:34.387496  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:34.390595  8964 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.395337  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:01:34.395866  8794 leader_election.cc:343] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.396178  8756 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.403872  8956 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:34.412186  8930 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:34.416411  8870 ts_tablet_manager.cc:1434] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.092s	user 0.008s	sys 0.000s
I20260709 13:01:34.418308  8870 tablet_bootstrap.cc:492] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:34.402136  8576 catalog_manager.cc:5697] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:34.389775  8944 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.428802  8944 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.431867  8577 catalog_manager.cc:5697] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:34.438572  8870 tablet_bootstrap.cc:654] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.443008  8964 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:34.443071  8756 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:34.445941  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:34.446744  8826 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.447103  8944 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.449759  8944 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:34.454540  8645 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:34.455673  8964 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:34.456415  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:01:34.458527  8794 leader_election.cc:343] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.457177  8964 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:34.459528  8956 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:34.459949  8964 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.479128  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:34.480870  8680 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.485306  8578 catalog_manager.cc:5697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:34.499070  8871 tablet_bootstrap.cc:492] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:34.499711  8871 ts_tablet_manager.cc:1403] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.162s	user 0.025s	sys 0.040s
I20260709 13:01:34.501355  8874 tablet_bootstrap.cc:492] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:34.501925  8874 ts_tablet_manager.cc:1403] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.158s	user 0.012s	sys 0.006s
I20260709 13:01:34.504317  8871 raft_consensus.cc:359] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.505131  8871 raft_consensus.cc:385] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.505457  8871 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.506321  8871 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.510102  8871 ts_tablet_manager.cc:1434] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:01:34.514302  8871 tablet_bootstrap.cc:492] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:34.519145  8870 tablet_bootstrap.cc:492] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:34.519757  8870 ts_tablet_manager.cc:1403] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.102s	user 0.019s	sys 0.001s
I20260709 13:01:34.524214  8870 raft_consensus.cc:359] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.524991  8870 raft_consensus.cc:385] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.525372  8870 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.518160  8874 raft_consensus.cc:359] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.526382  8874 raft_consensus.cc:385] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.526722  8874 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.527813  8870 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.530292  8870 ts_tablet_manager.cc:1434] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:01:34.531085  8944 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:34.532150  8871 tablet_bootstrap.cc:654] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.533375  8874 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.536398  8874 ts_tablet_manager.cc:1434] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 13:01:34.537480  8874 tablet_bootstrap.cc:492] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:34.540766  8871 tablet_bootstrap.cc:492] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:34.541618  8870 tablet_bootstrap.cc:492] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:34.543717  8871 ts_tablet_manager.cc:1403] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.001s
I20260709 13:01:34.546535  8826 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:34.547852  8871 raft_consensus.cc:359] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.554996  8871 raft_consensus.cc:385] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.555348  8871 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.556581  8871 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.556859  8680 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:34.563445  8874 tablet_bootstrap.cc:654] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.559650  8871 ts_tablet_manager.cc:1434] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.002s
I20260709 13:01:34.565352  8871 tablet_bootstrap.cc:492] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:34.569123  8793 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:34.570123  8900 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:34.570683  8900 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:34.571703  8900 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.572628  8871 tablet_bootstrap.cc:654] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.562712  8870 tablet_bootstrap.cc:654] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.590319  8874 tablet_bootstrap.cc:492] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:34.590914  8874 ts_tablet_manager.cc:1403] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.054s	user 0.016s	sys 0.008s
I20260709 13:01:34.593843  8870 tablet_bootstrap.cc:492] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:34.594404  8871 tablet_bootstrap.cc:492] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:34.594504  8870 ts_tablet_manager.cc:1403] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.008s
I20260709 13:01:34.595072  8871 ts_tablet_manager.cc:1403] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.002s
I20260709 13:01:34.594553  8874 raft_consensus.cc:359] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.597368  8874 raft_consensus.cc:385] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.597666  8874 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.597004  8578 catalog_manager.cc:5697] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:34.598260  8871 raft_consensus.cc:359] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.598409  8874 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.598078  8870 raft_consensus.cc:359] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.599218  8871 raft_consensus.cc:385] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.599670  8870 raft_consensus.cc:385] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.602720  8870 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.603618  8870 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.605582  8870 ts_tablet_manager.cc:1434] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:01:34.606716  8870 tablet_bootstrap.cc:492] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:34.599741  8871 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.601055  8874 ts_tablet_manager.cc:1434] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:01:34.609982  8871 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.616487  8871 ts_tablet_manager.cc:1434] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:01:34.617965  8871 tablet_bootstrap.cc:492] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:34.619346  8900 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.619902  8900 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.622032  8900 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:34.624085  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:34.624485  8871 tablet_bootstrap.cc:654] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:34.626040  8794 leader_election.cc:343] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.629568  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:34.630410  8677 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:34.632709  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:34.634687  9059 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:34.635267  9059 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.635748  9059 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.638080  9000 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.638689  9000 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.642791  8752 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.644277  8874 tablet_bootstrap.cc:492] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:34.643996  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:34.648588  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:34.651242  8677 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 1: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:01:34.649523  9000 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:34.656273  8871 tablet_bootstrap.cc:492] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:34.656277  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:34.657212  8871 ts_tablet_manager.cc:1403] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.040s	user 0.018s	sys 0.020s
I20260709 13:01:34.656273  9059 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.659545  8900 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:34.660266  9059 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:34.660439  8871 raft_consensus.cc:359] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.661141  8871 raft_consensus.cc:385] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.661439  8871 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.652657  8874 tablet_bootstrap.cc:654] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.661626  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:34.662194  8871 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.653718  8870 tablet_bootstrap.cc:654] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:34.663225  8794 leader_election.cc:343] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.666575  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:34.664573  8871 ts_tablet_manager.cc:1434] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:01:34.667354  8677 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.668135  8871 tablet_bootstrap.cc:492] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:34.675880  8871 tablet_bootstrap.cc:654] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.676614  8900 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:34.675940  9000 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:34.692366  8677 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:34.693951  8874 tablet_bootstrap.cc:492] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:34.694528  8874 ts_tablet_manager.cc:1403] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.001s
I20260709 13:01:34.699033  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:34.699994  9059 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:34.700565  9059 raft_consensus.cc:697] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:34.701984  8874 raft_consensus.cc:359] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.708745  8874 raft_consensus.cc:385] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.709240  8874 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.710253  8874 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.713661  8874 ts_tablet_manager.cc:1434] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:01:34.715263  8874 tablet_bootstrap.cc:492] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:34.720703  8964 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.721324  8964 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.723881  8874 tablet_bootstrap.cc:654] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.725484  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:34.726083  8964 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:34.721217  9059 consensus_queue.cc:237] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:34.727690  8645 leader_election.cc:343] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.727856  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:34.730342  8870 tablet_bootstrap.cc:492] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
W20260709 13:01:34.730376  8647 leader_election.cc:343] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.731101  8870 ts_tablet_manager.cc:1403] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.125s	user 0.010s	sys 0.010s
I20260709 13:01:34.731043  8647 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:34.732389  8964 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:34.734179  8900 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.737535  8964 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.737414  8900 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.738047  8964 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.738067  8870 raft_consensus.cc:359] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.739209  8870 raft_consensus.cc:385] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.740823  8900 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:34.741241  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:34.739614  8870 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.741900  8677 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:34.743211  8913 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.743522  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:34.743898  8913 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.746400  8900 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:34.746840  8900 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.747206  8900 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.747010  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:34.749200  8794 leader_election.cc:343] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.749714  8913 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:34.742722  8870 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.752400  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:34.753181  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:34.753859  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:34.754977  8647 leader_election.cc:343] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:34.756137  8645 leader_election.cc:343] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:34.757021  8645 leader_election.cc:343] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.757465  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:34.758301  8913 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:34.767462  8964 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:34.770747  8870 ts_tablet_manager.cc:1434] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.004s
I20260709 13:01:34.771996  9053 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.774487  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:34.774667  8870 tablet_bootstrap.cc:492] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:34.774879  9053 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.778091  8874 tablet_bootstrap.cc:492] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:34.778648  8874 ts_tablet_manager.cc:1403] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.064s	user 0.014s	sys 0.004s
I20260709 13:01:34.781105  8871 tablet_bootstrap.cc:492] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:34.781539  8871 ts_tablet_manager.cc:1403] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.114s	user 0.011s	sys 0.017s
I20260709 13:01:34.782982  9053 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:34.780663  8944 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.783607  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:34.784914  8944 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.787691  8754 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.788713  8871 raft_consensus.cc:359] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.789469  8871 raft_consensus.cc:385] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.789721  8871 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
W20260709 13:01:34.793385  8645 leader_election.cc:343] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.794477  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:34.795171  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:34.795979  8677 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:34.795290  8756 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:34.796892  8871 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.790594  8944 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:34.808882  8793 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:34.813328  9053 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:34.813231  8944 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:34.816566  8900 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.817126  8944 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.819962  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:34.821530  8944 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.820927  8871 ts_tablet_manager.cc:1434] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.000s
I20260709 13:01:34.826316  8826 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 1: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:01:34.829607  8900 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:34.831606  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:34.844033  8677 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.840098  8930 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:34.841881  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:01:34.838320  8647 leader_election.cc:343] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.848071  8647 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:34.849658  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:34.839869  8874 raft_consensus.cc:359] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.850785  9053 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:34.850871  8874 raft_consensus.cc:385] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:34.854190  8874 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:34.855067  8874 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.855628  8930 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:34.857504  8874 ts_tablet_manager.cc:1434] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.079s	user 0.005s	sys 0.000s
I20260709 13:01:34.863904  8871 tablet_bootstrap.cc:492] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:34.862316  8677 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:34.875327  9053 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:34.879155  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:34.889027  8874 tablet_bootstrap.cc:492] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
W20260709 13:01:34.900784  8627 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:34.888450  9000 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.904918  9000 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.908263  9000 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:34.909343  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:34.910897  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:34.912627  8719 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:34.902642  8577 catalog_manager.cc:5697] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:34.893527  8900 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:34.917583  8900 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:34.918561  8900 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.923439  9000 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:34.928049  8944 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:34.936838  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:34.937193  8944 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:34.939213  8870 tablet_bootstrap.cc:654] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:34.943006  8677 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.945633  8913 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:01:34.926590  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 82344.4us
W20260709 13:01:34.968075  8794 leader_election.cc:343] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:34.970280  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:34.974673  8751 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:34.978971  8578 catalog_manager.cc:5697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:34.985693  8647 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Couldn't send request to peer e814ed2013244fa6be42623ea5c0c7b9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:34.986835  8913 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:35.006302  8874 tablet_bootstrap.cc:654] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.019253  8871 tablet_bootstrap.cc:654] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.010324  8677 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:35.032109  8793 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:35.034664  8944 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:35.035914  8944 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:35.036835  8944 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.040627  8751 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:35.046339  8870 tablet_bootstrap.cc:492] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:35.046211  9081 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.046999  8870 ts_tablet_manager.cc:1403] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.273s	user 0.023s	sys 0.015s
I20260709 13:01:35.046937  9081 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.049479  9081 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:35.050097  8870 raft_consensus.cc:359] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.050941  8870 raft_consensus.cc:385] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.051218  8870 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.058686  8870 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.070482  8870 ts_tablet_manager.cc:1434] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:01:35.059767  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:35.071640  8678 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:35.081418  8871 tablet_bootstrap.cc:492] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:35.083859  8793 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:35.083968  8871 ts_tablet_manager.cc:1403] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.220s	user 0.019s	sys 0.009s
I20260709 13:01:35.082703  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:35.086751  8870 tablet_bootstrap.cc:492] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:35.086972  8900 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:35.088555  8900 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.090144  8900 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.090984  9000 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.091504  9000 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:35.098023  8794 leader_election.cc:343] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.099036  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:35.099772  8678 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:35.101265  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:35.102550  8952 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:35.103003  8952 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.103367  8952 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.103449  9000 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:35.105293  9081 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:35.109552  8577 catalog_manager.cc:5697] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:35.115456  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:01:35.121410  8722 leader_election.cc:343] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.127146  8874 tablet_bootstrap.cc:492] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:35.127735  8874 ts_tablet_manager.cc:1403] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.269s	user 0.022s	sys 0.034s
I20260709 13:01:35.135336  8870 tablet_bootstrap.cc:654] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.122334  8900 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:35.147696  8794 consensus_peers.cc:597] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:35.150264  8874 raft_consensus.cc:359] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.156903  8874 raft_consensus.cc:385] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.157363  8874 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.155766  8871 raft_consensus.cc:359] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.158412  8874 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.162737  8900 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:35.164083  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:35.166602  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:35.169322  8677 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.154050  8952 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.159507  8871 raft_consensus.cc:385] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.170230  8871 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.171273  8871 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.171757  8952 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:01:35.173147  8794 leader_election.cc:343] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.176404  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:35.177626  8871 ts_tablet_manager.cc:1434] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.093s	user 0.009s	sys 0.000s
I20260709 13:01:35.177868  8874 ts_tablet_manager.cc:1434] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.050s	user 0.004s	sys 0.004s
I20260709 13:01:35.178174  8677 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:35.179148  8871 tablet_bootstrap.cc:492] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:35.179165  8874 tablet_bootstrap.cc:492] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
W20260709 13:01:35.179550  8722 leader_election.cc:343] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.182108  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:35.182934  8677 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.188737  8870 tablet_bootstrap.cc:492] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:35.194972  8870 ts_tablet_manager.cc:1403] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.110s	user 0.012s	sys 0.013s
I20260709 13:01:35.196563  8871 tablet_bootstrap.cc:654] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.198652  8870 raft_consensus.cc:359] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.193499  8677 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:35.199585  8870 raft_consensus.cc:385] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.200881  8870 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.201090  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:35.201361  8874 tablet_bootstrap.cc:654] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.201856  8870 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.202406  8952 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:35.203091  8952 raft_consensus.cc:697] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:35.204315  8870 ts_tablet_manager.cc:1434] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:01:35.204042  8952 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.205572  8870 tablet_bootstrap.cc:492] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:35.211932  8871 tablet_bootstrap.cc:492] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:35.212595  8871 ts_tablet_manager.cc:1403] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.004s
I20260709 13:01:35.179628  8793 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:35.215297  8870 tablet_bootstrap.cc:654] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.215755  8871 raft_consensus.cc:359] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.216527  8871 raft_consensus.cc:385] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.219785  8895 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:35.216805  8871 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.220587  8895 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:35.221086  8871 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.221482  8895 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.225929  8871 ts_tablet_manager.cc:1434] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.001s
I20260709 13:01:35.229882  8871 tablet_bootstrap.cc:492] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
W20260709 13:01:35.247108  8794 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:35.247915  9081 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:35.250385  8874 tablet_bootstrap.cc:492] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:35.251104  8874 ts_tablet_manager.cc:1403] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.072s	user 0.004s	sys 0.019s
I20260709 13:01:35.255323  8874 raft_consensus.cc:359] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.258683  8874 raft_consensus.cc:385] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.259135  8874 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.259510  8870 tablet_bootstrap.cc:492] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:35.260107  8870 ts_tablet_manager.cc:1403] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.055s	user 0.012s	sys 0.033s
I20260709 13:01:35.262374  8871 tablet_bootstrap.cc:654] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.260608  8874 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.267820  8874 ts_tablet_manager.cc:1434] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.002s
I20260709 13:01:35.268158  8870 raft_consensus.cc:359] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.269076  8870 raft_consensus.cc:385] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.269433  8870 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.270161  8870 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.268921  9053 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.269814  8874 tablet_bootstrap.cc:492] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:35.274214  8870 ts_tablet_manager.cc:1434] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:01:35.271222  9053 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.276549  8870 tablet_bootstrap.cc:492] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:35.283425  9053 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:35.282191  8895 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.284376  8895 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.289762  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:35.290782  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:35.297086  8874 tablet_bootstrap.cc:654] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.298926  8895 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:35.300072  8576 catalog_manager.cc:5697] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:35.303543  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:35.304929  8752 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
W20260709 13:01:35.303946  8645 leader_election.cc:343] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.305276  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:35.306897  8677 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
W20260709 13:01:35.307446  8647 leader_election.cc:343] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.308041  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:35.308269  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:35.309124  9081 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:35.309502  9081 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.309851  9081 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:35.310161  8870 tablet_bootstrap.cc:654] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.315064  9053 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:35.330160  8577 catalog_manager.cc:5697] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:35.344113  9081 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.349675  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:35.351037  8680 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:35.361510  9081 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:35.371749  8874 tablet_bootstrap.cc:492] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:35.372165  8871 tablet_bootstrap.cc:492] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:35.372300  8874 ts_tablet_manager.cc:1403] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.103s	user 0.014s	sys 0.008s
I20260709 13:01:35.372706  8871 ts_tablet_manager.cc:1403] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.143s	user 0.011s	sys 0.009s
I20260709 13:01:35.375243  8874 raft_consensus.cc:359] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.376060  8874 raft_consensus.cc:385] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.375787  8871 raft_consensus.cc:359] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.376389  8874 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.377117  8870 tablet_bootstrap.cc:492] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:35.376648  8871 raft_consensus.cc:385] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.377664  8871 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.377395  8874 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.377908  8870 ts_tablet_manager.cc:1403] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.102s	user 0.024s	sys 0.045s
I20260709 13:01:35.378403  8871 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.380112  8874 ts_tablet_manager.cc:1434] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:01:35.380060  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:35.380843  8871 ts_tablet_manager.cc:1434] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:01:35.380824  8752 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:35.381351  8874 tablet_bootstrap.cc:492] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:35.381515  8870 raft_consensus.cc:359] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.382313  8870 raft_consensus.cc:385] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.382680  8870 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.383764  8870 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.401727  8680 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:01:35.403416  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:35.409436  8752 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:01:35.414948  9081 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:35.415904  9081 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:35.417109  9081 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:35.421536  8794 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:35.405057  8871 tablet_bootstrap.cc:492] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:35.406689  8870 ts_tablet_manager.cc:1434] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:01:35.407583  8874 tablet_bootstrap.cc:654] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.424032  8870 tablet_bootstrap.cc:492] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:35.430790  8874 tablet_bootstrap.cc:492] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:35.431335  8870 tablet_bootstrap.cc:654] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.431985  8874 ts_tablet_manager.cc:1403] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.001s
I20260709 13:01:35.439474  8874 raft_consensus.cc:359] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.440340  8874 raft_consensus.cc:385] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.440573  8827 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.440635  8874 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.442605  8874 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.444341  8870 tablet_bootstrap.cc:492] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:35.444943  8870 ts_tablet_manager.cc:1403] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.004s
I20260709 13:01:35.444628  8576 catalog_manager.cc:5697] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 2, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:35.448089  8870 raft_consensus.cc:359] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.448813  8870 raft_consensus.cc:385] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.449088  8870 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.450133  8952 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.450027  8870 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.450680  8952 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.452875  8952 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:35.452880  8870 ts_tablet_manager.cc:1434] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:01:35.430281  8871 tablet_bootstrap.cc:654] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.454157  8870 tablet_bootstrap.cc:492] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:35.455967  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:35.456657  8830 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:35.459583  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:35.460040  8722 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:35.460357  8677 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:35.463762  8870 tablet_bootstrap.cc:654] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.466737  8951 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:35.467168  8951 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.467548  8951 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.468708  8870 tablet_bootstrap.cc:492] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:35.469205  8870 ts_tablet_manager.cc:1403] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.015s	user 0.007s	sys 0.005s
I20260709 13:01:35.470827  8952 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.471630  8870 raft_consensus.cc:359] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.472337  8870 raft_consensus.cc:385] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.471603  8952 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.472539  8870 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.473160  8870 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.475301  8870 ts_tablet_manager.cc:1434] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:01:35.476472  8870 tablet_bootstrap.cc:492] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:35.477152  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:35.478085  8895 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:35.482394  8952 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:35.483479  8870 tablet_bootstrap.cc:654] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.488283  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:01:35.490064  8856 tablet.cc:2406] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:01:35.492070  8874 ts_tablet_manager.cc:1434] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.000s
I20260709 13:01:35.495862  8900 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.496140  8951 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.496520  8900 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.499079  8900 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:35.499524  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:35.500013  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:35.500240  8678 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.501137  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:35.501314  8952 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:35.502810  8794 leader_election.cc:343] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.504359  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:35.505077  8677 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:35.509446  8870 tablet_bootstrap.cc:492] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:35.510039  8870 ts_tablet_manager.cc:1403] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.005s
I20260709 13:01:35.511049  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:35.513077  8870 raft_consensus.cc:359] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.513827  8870 raft_consensus.cc:385] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.514104  8870 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.514902  8870 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.516088  9081 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:35.516484  9081 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.516826  9081 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.516852  8870 ts_tablet_manager.cc:1434] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:01:35.522451  8870 tablet_bootstrap.cc:492] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:35.526968  9081 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.529214  8678 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:35.530772  8870 tablet_bootstrap.cc:654] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.530578  8719 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:35.527711  8874 tablet_bootstrap.cc:492] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:35.531095  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:35.532070  8952 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:01:35.533002  8794 leader_election.cc:343] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.533617  8952 raft_consensus.cc:697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:35.534798  8952 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.535511  9081 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:35.545831  8871 tablet_bootstrap.cc:492] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:35.546713  8871 ts_tablet_manager.cc:1403] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.142s	user 0.006s	sys 0.031s
I20260709 13:01:35.550328  9053 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.550994  9053 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.549458  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:35.551985  8829 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.553597  9053 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:35.556404  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:01:35.558936  8647 leader_election.cc:343] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.538648  8951 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:35.561151  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:35.561895  8678 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.566255  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:35.572391  8874 tablet_bootstrap.cc:654] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.580369  8829 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:35.584234  9081 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:01:35.593003  8645 leader_election.cc:343] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.593660  8645 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:35.595448  9054 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:35.603739  9081 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.604413  9081 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.608336  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:35.609148  8752 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:35.610950  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:35.611976  8900 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:35.612401  8900 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.612892  8900 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.618320  8870 tablet_bootstrap.cc:492] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:35.619131  8870 ts_tablet_manager.cc:1403] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.101s	user 0.038s	sys 0.041s
I20260709 13:01:35.622110  8870 raft_consensus.cc:359] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.622939  8870 raft_consensus.cc:385] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.623202  8870 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.623865  8870 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.625998  8870 ts_tablet_manager.cc:1434] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:01:35.620299  8871 raft_consensus.cc:359] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.630371  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:35.631197  8679 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:35.630661  8576 catalog_manager.cc:5697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:35.631975  9081 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:35.633236  8871 raft_consensus.cc:385] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.633569  8871 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
W20260709 13:01:35.635210  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 89522.6us
I20260709 13:01:35.634492  8871 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.636700  9059 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:35.638268  8870 tablet_bootstrap.cc:492] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:35.640199  9000 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.640765  9000 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.643621  8829 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.645366  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:35.646154  8870 tablet_bootstrap.cc:654] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.651736  8871 ts_tablet_manager.cc:1434] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.105s	user 0.007s	sys 0.000s
I20260709 13:01:35.657131  9000 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:35.658329  8870 tablet_bootstrap.cc:492] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:35.658913  8870 ts_tablet_manager.cc:1403] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.006s
I20260709 13:01:35.661175  8871 tablet_bootstrap.cc:492] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:35.662153  8870 raft_consensus.cc:359] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.663079  8870 raft_consensus.cc:385] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.663442  8870 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.666265  8874 tablet_bootstrap.cc:492] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:35.667017  8874 ts_tablet_manager.cc:1403] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.140s	user 0.012s	sys 0.005s
I20260709 13:01:35.664453  8870 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.670329  8874 raft_consensus.cc:359] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.672278  8874 raft_consensus.cc:385] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.672636  8874 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:35.673028  8871 tablet_bootstrap.cc:654] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:35.673310  8794 consensus_peers.cc:597] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:35.673486  8874 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.677345  8874 ts_tablet_manager.cc:1434] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.005s
I20260709 13:01:35.680133  8870 ts_tablet_manager.cc:1434] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:01:35.681378  8870 tablet_bootstrap.cc:492] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:35.681142  9000 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:35.683914  8874 tablet_bootstrap.cc:492] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:35.684461  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:35.685132  8753 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.693250  8751 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.694952  8722 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:35.602252  8678 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:35.703245  9081 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.703898  9081 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.706459  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:35.708038  8895 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:35.721465  8895 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:35.728615  9053 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.729557  9053 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:35.741878  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47049.1us
I20260709 13:01:35.743628  8900 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.760814  8900 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:35.768668  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:35.770694  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:35.765275  9059 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.771531  8678 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.771809  9059 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.772324  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:35.769502  8756 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:35.775415  8645 leader_election.cc:343] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.775491  9059 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:35.784472  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:35.786305  8794 leader_election.cc:343] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.752543  8895 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.803469  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:35.805223  8756 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
W20260709 13:01:35.807183  8793 leader_election.cc:343] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.811029  8874 tablet_bootstrap.cc:654] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.812849  8793 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:35.815193  9059 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:35.826116  9000 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:35.827294  8870 tablet_bootstrap.cc:654] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.841585  8678 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:35.842199  8900 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.843147  8900 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.831014  9053 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:35.850822  8900 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:35.860468  8952 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.867735  8952 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.879915  8871 tablet_bootstrap.cc:492] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:35.865860  9081 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:35.890184  8871 ts_tablet_manager.cc:1403] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.229s	user 0.042s	sys 0.044s
I20260709 13:01:35.887285  8952 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:35.892799  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:35.909101  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:35.909969  8679 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:35.903193  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:35.890072  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:35.922848  8719 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:35.924361  9000 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:35.924935  9000 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.925406  9000 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:35.927842  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:35.928646  8679 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
W20260709 13:01:35.929574  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 50027.1us
I20260709 13:01:35.920631  8871 raft_consensus.cc:359] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.932958  8871 raft_consensus.cc:385] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:35.933337  8871 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
W20260709 13:01:35.933931  8647 leader_election.cc:343] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.934191  9081 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:35.934684  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:35.934818  9081 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:01:35.937060  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 56961us
I20260709 13:01:35.936585  9081 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:35.939615  8913 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:35.934415  8871 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:35.948220  8793 leader_election.cc:343] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:35.951536  8793 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:35.955672  8871 ts_tablet_manager.cc:1434] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.000s
I20260709 13:01:35.965978  8900 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:35.956923  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:35.971457  8677 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:01:35.973966  9114 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:35.960733  9125 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:35.989526  8871 tablet_bootstrap.cc:492] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:36.005851  8871 tablet_bootstrap.cc:654] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:35.986370  9125 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.028874  8870 tablet_bootstrap.cc:492] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:36.017366  9125 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.029657  8870 ts_tablet_manager.cc:1403] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.349s	user 0.029s	sys 0.024s
I20260709 13:01:36.041484  8576 catalog_manager.cc:5697] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:36.053296  8870 raft_consensus.cc:359] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.054181  8870 raft_consensus.cc:385] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.054503  8870 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.054723  9000 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.060595  8870 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.073581  8874 tablet_bootstrap.cc:492] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:36.057443  9124 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:01:36.079205  8870 ts_tablet_manager.cc:1434] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.001s
I20260709 13:01:36.079337  8874 ts_tablet_manager.cc:1403] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.396s	user 0.047s	sys 0.047s
W20260709 13:01:36.079654  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 60844.3us
I20260709 13:01:36.087766  8870 tablet_bootstrap.cc:492] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
W20260709 13:01:36.112707  9127 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:01:36.110761  9133 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.052s
I20260709 13:01:36.122028  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:36.136861  8751 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 1: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:01:36.152426  8874 raft_consensus.cc:359] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.153337  8874 raft_consensus.cc:385] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.153625  8874 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.156571  8874 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.160192  8874 ts_tablet_manager.cc:1434] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.000s
I20260709 13:01:36.154270  9123 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.160836  9123 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:36.155915  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 43551.2us
I20260709 13:01:36.162545  9125 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.164139  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:36.176957  8829 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 1: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:01:36.178238  9125 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:36.180506  8874 tablet_bootstrap.cc:492] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:36.190219  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:36.194012  9054 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.194713  9054 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.200131  8871 tablet_bootstrap.cc:492] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:36.200719  8871 ts_tablet_manager.cc:1403] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.231s	user 0.023s	sys 0.019s
I20260709 13:01:36.202735  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:36.203502  8680 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.204800  9000 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:36.209532  9054 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:36.211608  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:36.212415  8826 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:36.213766  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:36.214004  8647 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:01:36.215423  8793 leader_election.cc:343] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.219856  9054 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:36.219730  8680 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:36.222409  8719 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:36.223405  9114 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:36.224308  9114 raft_consensus.cc:697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:36.225075  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:36.220626  9054 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.237466  9054 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.237113  8952 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:36.225800  8756 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:36.245301  8870 tablet_bootstrap.cc:654] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.225523  9114 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.248039  9124 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:36.263149  8577 catalog_manager.cc:5697] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:36.266461  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:36.267751  8826 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 1: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:01:36.274845  9123 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:36.282444  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:36.284866  8871 raft_consensus.cc:359] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.285825  8871 raft_consensus.cc:385] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.286167  8871 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.286389  8756 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:01:36.290462  9123 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.295217  9123 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:36.297151  8794 leader_election.cc:343] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.297556  9123 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:36.292342  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:36.297706  8794 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:36.300026  9124 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:36.300199  8874 tablet_bootstrap.cc:654] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.300941  8871 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.306430  8871 ts_tablet_manager.cc:1434] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.105s	user 0.008s	sys 0.001s
I20260709 13:01:36.307753  9140 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.308408  9140 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.277972  9054 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.330906  8871 tablet_bootstrap.cc:492] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
W20260709 13:01:36.331604  8627 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:36.315158  9140 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:36.337746  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:36.342041  9125 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:01:36.368315  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:36.368214  9054 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:36.374150  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:36.370437  8870 tablet_bootstrap.cc:492] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:36.374936  8756 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.375224  8870 ts_tablet_manager.cc:1403] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.293s	user 0.015s	sys 0.011s
I20260709 13:01:36.349740  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:01:36.377944  8645 leader_election.cc:343] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.378988  8679 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:36.388504  8793 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:36.393376  8756 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:36.395776  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:36.389657  8895 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:36.399259  8895 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.399698  8895 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:36.401609  8794 leader_election.cc:343] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.402091  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:36.404875  8913 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:36.405462  8577 catalog_manager.cc:5697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.411080  9137 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:01:36.415536  8870 raft_consensus.cc:359] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.421429  8870 raft_consensus.cc:385] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.421795  8870 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.422423  8870 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.421257  9081 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:36.424332  8913 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:36.425513  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:36.425457  8913 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.433126  8870 ts_tablet_manager.cc:1434] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.001s
I20260709 13:01:36.439095  8895 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.442798  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:36.443622  8677 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.445811  8895 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:36.446087  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:36.449924  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:36.450763  8827 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.447075  8829 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:36.466696  8870 tablet_bootstrap.cc:492] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
W20260709 13:01:36.469935  8794 leader_election.cc:343] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.472265  8871 tablet_bootstrap.cc:654] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:36.474603  8794 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:36.475492  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:36.492991  9054 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:36.493460  9054 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.493815  9054 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.475299  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:36.499439  8827 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:36.511469  8677 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
W20260709 13:01:36.507143  8794 leader_election.cc:343] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.522315  8870 tablet_bootstrap.cc:654] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.525296  8874 tablet_bootstrap.cc:492] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:36.526065  8874 ts_tablet_manager.cc:1403] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.353s	user 0.033s	sys 0.038s
I20260709 13:01:36.528313  9054 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.530138  8793 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:36.531075  8895 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:36.532210  8895 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:36.533241  8895 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.543720  9054 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:36.544278  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:36.544497  8895 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:01:36.545745  8645 leader_election.cc:343] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.551050  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:36.551841  8827 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.556519  8870 tablet_bootstrap.cc:492] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:36.558347  8870 ts_tablet_manager.cc:1403] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.099s	user 0.013s	sys 0.003s
I20260709 13:01:36.562180  9151 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:01:36.559511  8577 catalog_manager.cc:5697] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:36.573725  8871 tablet_bootstrap.cc:492] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:36.574348  8871 ts_tablet_manager.cc:1403] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.244s	user 0.024s	sys 0.006s
I20260709 13:01:36.582687  8827 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:36.582830  8871 raft_consensus.cc:359] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.583698  8871 raft_consensus.cc:385] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.584041  8871 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.583561  8576 catalog_manager.cc:5697] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.584825  8871 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.589721  8874 raft_consensus.cc:359] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.591095  8874 raft_consensus.cc:385] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.593230  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:36.591574  8874 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.598230  8913 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:36.598553  8870 raft_consensus.cc:359] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.599471  8913 raft_consensus.cc:697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:36.599748  8870 raft_consensus.cc:385] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.600198  8870 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.600510  8913 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.602591  8871 ts_tablet_manager.cc:1434] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.028s	user 0.002s	sys 0.005s
I20260709 13:01:36.603927  8871 tablet_bootstrap.cc:492] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:36.617835  8874 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.608929  8870 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.620498  9137 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.621057  9137 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.628296  8870 ts_tablet_manager.cc:1434] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.069s	user 0.010s	sys 0.000s
I20260709 13:01:36.634814  8874 ts_tablet_manager.cc:1434] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.108s	user 0.013s	sys 0.000s
I20260709 13:01:36.638841  8874 tablet_bootstrap.cc:492] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:36.646303  8870 tablet_bootstrap.cc:492] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:36.653328  9053 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.657671  9053 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.665578  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:36.668393  8645 leader_election.cc:343] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.673861  9137 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:36.675648  9053 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:36.676638  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:36.678848  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:01:36.681520  8794 leader_election.cc:343] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.684082  8577 catalog_manager.cc:5697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:36.683001  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:01:36.688022  8647 leader_election.cc:343] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.691030  8647 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:01:36.695597  8793 leader_election.cc:343] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.696161  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:36.694512  9054 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:36.708547  8871 tablet_bootstrap.cc:654] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.701269  9124 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:36.728271  8952 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.728902  8952 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.731362  8952 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:36.737205  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:36.738240  8678 raft_consensus.cc:2393] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 1: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:01:36.741916  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:36.744663  8870 tablet_bootstrap.cc:654] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.745644  8722 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:36.746663  9114 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.746905  8871 tablet_bootstrap.cc:492] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:36.744663  8874 tablet_bootstrap.cc:654] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.747721  8871 ts_tablet_manager.cc:1403] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.144s	user 0.017s	sys 0.010s
I20260709 13:01:36.752204  8871 raft_consensus.cc:359] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.753552  8871 raft_consensus.cc:385] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.753993  8871 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.755582  8871 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.756858  8870 tablet_bootstrap.cc:492] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:36.770596  8870 ts_tablet_manager.cc:1403] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.135s	user 0.021s	sys 0.000s
I20260709 13:01:36.768631  9129 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:01:36.773696  8870 raft_consensus.cc:359] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.775400  8870 raft_consensus.cc:385] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.775768  8870 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.776525  8870 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.777904  8874 tablet_bootstrap.cc:492] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:36.778483  8874 ts_tablet_manager.cc:1403] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.141s	user 0.018s	sys 0.013s
I20260709 13:01:36.779064  8870 ts_tablet_manager.cc:1434] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:01:36.780246  8870 tablet_bootstrap.cc:492] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:36.759123  8871 ts_tablet_manager.cc:1434] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.001s
I20260709 13:01:36.782078  8874 raft_consensus.cc:359] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.782866  8874 raft_consensus.cc:385] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.783139  8874 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.783584  8871 tablet_bootstrap.cc:492] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:36.783903  8874 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.785881  8874 ts_tablet_manager.cc:1434] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:01:36.757671  9114 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:36.787626  8874 tablet_bootstrap.cc:492] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:36.790792  8871 tablet_bootstrap.cc:654] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.796626  8870 tablet_bootstrap.cc:654] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.796626  8874 tablet_bootstrap.cc:654] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.803970  9114 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:36.809816  9114 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.810592  9114 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.813949  9114 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:36.821035  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:36.839910  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:36.840941  8829 raft_consensus.cc:2393] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 1: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:01:36.847754  8722 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:36.849519  9114 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.868629  9114 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:36.879948  9053 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:36.902913  9053 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:36.910502  9054 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.911562  9054 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.916677  9054 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:36.922752  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:01:36.925086  8647 leader_election.cc:343] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.937196  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:36.940585  8645 leader_election.cc:343] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.941116  8645 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:36.945341  8913 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:36.945488  8874 tablet_bootstrap.cc:492] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:36.946256  8874 ts_tablet_manager.cc:1403] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.159s	user 0.031s	sys 0.035s
I20260709 13:01:36.950217  8874 raft_consensus.cc:359] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.951164  8874 raft_consensus.cc:385] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.951488  8874 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.952302  8874 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.954178  8874 ts_tablet_manager.cc:1434] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:01:36.957831  8874 tablet_bootstrap.cc:492] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:36.961624  8871 tablet_bootstrap.cc:492] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:36.962219  8871 ts_tablet_manager.cc:1403] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.179s	user 0.061s	sys 0.052s
I20260709 13:01:36.962371  9059 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.963321  8870 tablet_bootstrap.cc:492] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:36.963832  8870 ts_tablet_manager.cc:1403] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.184s	user 0.061s	sys 0.043s
I20260709 13:01:36.965411  8871 raft_consensus.cc:359] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.966210  8871 raft_consensus.cc:385] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.966531  8871 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.966591  8870 raft_consensus.cc:359] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.962925  9059 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.967391  8870 raft_consensus.cc:385] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:36.967694  8870 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:36.968364  8870 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.970454  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:36.971472  8678 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:36.973484  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:36.974437  9137 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:36.974917  9137 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.975281  9137 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:36.975873  8874 tablet_bootstrap.cc:654] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:36.977432  8871 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:36.978855  8870 ts_tablet_manager.cc:1434] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:01:36.981070  8871 ts_tablet_manager.cc:1434] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:01:36.982372  8871 tablet_bootstrap.cc:492] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:36.984141  8870 tablet_bootstrap.cc:492] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:36.987897  9059 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:36.990414  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:36.967222  9151 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
W20260709 13:01:36.994234  8794 leader_election.cc:343] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:36.991729  8871 tablet_bootstrap.cc:654] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.004832  8870 tablet_bootstrap.cc:654] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.019037  9059 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:37.028913  9137 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.033300  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:37.034771  9137 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:01:37.041872  8794 leader_election.cc:343] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:37.044148  8874 tablet_bootstrap.cc:492] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:37.047365  8874 ts_tablet_manager.cc:1403] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.087s	user 0.016s	sys 0.010s
I20260709 13:01:37.050004  8870 tablet_bootstrap.cc:492] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:37.050601  8870 ts_tablet_manager.cc:1403] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.067s	user 0.015s	sys 0.005s
I20260709 13:01:37.050702  8874 raft_consensus.cc:359] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.052170  8874 raft_consensus.cc:385] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.052490  8874 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.054275  8870 raft_consensus.cc:359] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.055279  8870 raft_consensus.cc:385] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.055564  8870 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.056593  8870 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.056962  8913 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.057106  8874 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.057559  8913 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.057003  8871 tablet_bootstrap.cc:492] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:37.058604  8871 ts_tablet_manager.cc:1403] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.077s	user 0.017s	sys 0.018s
I20260709 13:01:37.059062  8870 ts_tablet_manager.cc:1434] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:01:37.060230  8870 tablet_bootstrap.cc:492] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:37.061699  8871 raft_consensus.cc:359] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.062506  8871 raft_consensus.cc:385] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.062839  8871 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.063611  8871 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.066222  8871 ts_tablet_manager.cc:1434] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:01:37.068502  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:37.069684  8874 ts_tablet_manager.cc:1434] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:01:37.069278  8826 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:37.071273  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:37.071676  8870 tablet_bootstrap.cc:654] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.072221  9053 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:37.072636  9053 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.072811  8874 tablet_bootstrap.cc:492] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:37.073024  9053 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.080560  8871 tablet_bootstrap.cc:492] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:37.082157  8874 tablet_bootstrap.cc:654] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.085657  9140 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:37.087405  9059 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.088006  9059 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.088728  8871 tablet_bootstrap.cc:654] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.091614  8913 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:37.092828  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:37.093657  8678 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.126686  9137 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.127504  9137 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.096776  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:37.188771  9114 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.189833  9114 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.194828  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:37.200372  9114 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:37.208021  8874 tablet_bootstrap.cc:492] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:37.208810  8874 ts_tablet_manager.cc:1403] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.136s	user 0.021s	sys 0.006s
I20260709 13:01:37.208657  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:37.212011  8753 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.227677  8870 tablet_bootstrap.cc:492] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:37.225009  9124 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.228266  8870 ts_tablet_manager.cc:1403] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.168s	user 0.038s	sys 0.045s
I20260709 13:01:37.228559  9137 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:37.228442  9124 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.229676  9053 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:37.234244  8645 leader_election.cc:343] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:37.234244  8793 leader_election.cc:343] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:37.231542  8870 raft_consensus.cc:359] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.241457  8870 raft_consensus.cc:385] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.241936  8870 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.242924  8870 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.246148  8870 ts_tablet_manager.cc:1434] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:01:37.235965  9124 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:37.236364  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:37.236960  9053 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:37.237619  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:37.239166  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:37.255581  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:37.256438  8679 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:37.225826  9059 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:37.253096  8870 tablet_bootstrap.cc:492] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:37.258785  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:37.261312  8794 leader_election.cc:343] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:37.262643  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:01:37.263557  8794 leader_election.cc:343] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:37.286240  8793 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:37.285828  8719 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:37.287945  9129 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:37.290134  9151 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:37.290683  9151 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.291046  9151 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.291522  8678 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
W20260709 13:01:37.284192  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 197611us
I20260709 13:01:37.296533  8871 tablet_bootstrap.cc:492] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:37.297154  8871 ts_tablet_manager.cc:1403] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.217s	user 0.037s	sys 0.060s
I20260709 13:01:37.302048  9151 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.304561  9151 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:37.308213  8870 tablet_bootstrap.cc:654] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:37.309057  8793 leader_election.cc:343] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:37.309578  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:37.310807  9059 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:37.310909  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:37.311664  9059 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:37.312116  9059 raft_consensus.cc:697] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:37.313045  9059 consensus_queue.cc:237] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.317231  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:37.318022  8678 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.321265  8871 raft_consensus.cc:359] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.321923  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:37.378923  8871 raft_consensus.cc:385] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.326685  8874 raft_consensus.cc:359] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.328495  8678 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:37.379428  8871 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.330292  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:37.380524  8824 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.380460  8871 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.391117  8874 raft_consensus.cc:385] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.392279  8874 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.394129  8874 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.408684  8874 ts_tablet_manager.cc:1434] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.178s	user 0.009s	sys 0.002s
W20260709 13:01:37.409670  8645 leader_election.cc:343] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:37.410683  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 76536.6us
I20260709 13:01:37.412434  8874 tablet_bootstrap.cc:492] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:37.413484  8793 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:37.416860  8871 ts_tablet_manager.cc:1434] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.119s	user 0.010s	sys 0.000s
I20260709 13:01:37.420432  9124 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:37.427489  8871 tablet_bootstrap.cc:492] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:37.427793  9124 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:37.428836  9124 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:37.463912  8794 leader_election.cc:343] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:37.464993  8794 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:01:37.472777  9188 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:01:37.476040  8576 catalog_manager.cc:5697] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.477595  9059 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:37.484707  8824 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:37.519343  8870 tablet_bootstrap.cc:492] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:37.519904  8870 ts_tablet_manager.cc:1403] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.267s	user 0.028s	sys 0.053s
W20260709 13:01:37.524668  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 60833.7us
W20260709 13:01:37.526230  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 63063.3us
I20260709 13:01:37.520978  9140 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.530691  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:37.531273  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:37.532694  9182 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:37.534152  9182 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:37.535749  9140 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.538606  9140 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:37.540601  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:37.541870  8870 raft_consensus.cc:359] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:37.542330  8645 leader_election.cc:343] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:37.543331  9182 raft_consensus.cc:697] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:37.544335  9182 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.542917  8870 raft_consensus.cc:385] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.548384  8870 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.549203  8870 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.556782  8871 tablet_bootstrap.cc:654] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.558537  8753 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.559595  8870 ts_tablet_manager.cc:1434] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.039s	user 0.005s	sys 0.003s
I20260709 13:01:37.560845  8870 tablet_bootstrap.cc:492] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
W20260709 13:01:37.562217  8627 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:37.563303  8874 tablet_bootstrap.cc:654] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.602691  8578 catalog_manager.cc:5697] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.614923  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:37.635390  8870 tablet_bootstrap.cc:654] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:37.636921  8794 leader_election.cc:343] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:37.643110  8874 tablet_bootstrap.cc:492] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:37.643697  8874 ts_tablet_manager.cc:1403] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.232s	user 0.012s	sys 0.003s
I20260709 13:01:37.654572  8871 tablet_bootstrap.cc:492] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:37.655314  8871 ts_tablet_manager.cc:1403] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.228s	user 0.031s	sys 0.038s
I20260709 13:01:37.654738  8874 raft_consensus.cc:359] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.655759  8874 raft_consensus.cc:385] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.656105  8874 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.657049  8874 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:37.657609  8647 leader_election.cc:343] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:37.658164  8647 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:37.658918  8871 raft_consensus.cc:359] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.659420  9182 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:37.659643  8871 raft_consensus.cc:385] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.659770  8874 ts_tablet_manager.cc:1434] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.001s
I20260709 13:01:37.659988  8871 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.663736  8871 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.665748  9114 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.666385  8871 ts_tablet_manager.cc:1434] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 13:01:37.666344  9114 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.667650  8871 tablet_bootstrap.cc:492] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:37.670183  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:37.671036  8678 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:37.671156  9114 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:37.672520  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:37.664975  8870 tablet_bootstrap.cc:492] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:37.673454  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:37.674572  8824 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:37.674584  8870 ts_tablet_manager.cc:1403] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.114s	user 0.014s	sys 0.003s
I20260709 13:01:37.673592  9114 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:37.679016  9114 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.679528  9114 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.680840  8871 tablet_bootstrap.cc:654] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.687616  8870 raft_consensus.cc:359] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.688977  8870 raft_consensus.cc:385] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.693749  8870 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.688628  8874 tablet_bootstrap.cc:492] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:37.701773  8870 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.717062  8870 ts_tablet_manager.cc:1434] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 13:01:37.750165  8870 tablet_bootstrap.cc:492] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
W20260709 13:01:37.726189  8794 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:37.753376  9059 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.754048  9059 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.757715  9059 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:37.763159  8578 catalog_manager.cc:5697] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.773815  8874 tablet_bootstrap.cc:654] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.776171  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:37.776968  8678 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:37.775753  9114 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.780156  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:37.780941  8675 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.786351  9129 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.787289  9129 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.791096  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:37.791774  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:37.791851  8677 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:37.793242  9059 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:37.793664  9059 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.794023  9059 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.797310  9114 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:37.799865  8675 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:37.800348  8719 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:37.801040  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:37.804838  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:37.805341  8870 tablet_bootstrap.cc:654] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.806030  8952 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:37.806519  8952 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.807014  8952 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.812490  9129 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:37.814563  8871 tablet_bootstrap.cc:492] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:37.815163  8871 ts_tablet_manager.cc:1403] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.148s	user 0.028s	sys 0.055s
I20260709 13:01:37.814961  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:37.815708  8827 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.817378  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:37.818198  8824 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:37.811647  9186 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:01:37.816543  8794 leader_election.cc:343] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:37.820428  9186 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:37.821770  9186 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.824553  8871 raft_consensus.cc:359] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.825279  8952 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.826480  8871 raft_consensus.cc:385] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.826857  8871 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.827873  8871 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.831072  9059 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.835381  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:37.836167  8675 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.840085  8827 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:37.843128  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:37.849725  8871 ts_tablet_manager.cc:1434] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.001s
I20260709 13:01:37.850111  8952 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:37.850169  8677 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:37.852996  8871 tablet_bootstrap.cc:492] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:37.843472  9059 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:37.844777  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:37.848284  8874 tablet_bootstrap.cc:492] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:37.858335  8874 ts_tablet_manager.cc:1403] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.195s	user 0.012s	sys 0.012s
W20260709 13:01:37.864600  8794 leader_election.cc:343] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:37.867281  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:37.868660  8824 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:37.873657  8794 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:01:37.876307  8794 consensus_peers.cc:597] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:37.879024  8677 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:37.881052  8719 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:01:37.882727  9129 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:37.885610  8870 tablet_bootstrap.cc:492] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:37.886195  8870 ts_tablet_manager.cc:1403] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.136s	user 0.018s	sys 0.009s
I20260709 13:01:37.849655  8675 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:37.887236  9129 raft_consensus.cc:697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:37.888631  9129 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.892813  8871 tablet_bootstrap.cc:654] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:37.893466  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:37.897697  8874 raft_consensus.cc:359] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.934991  8578 catalog_manager.cc:5697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.943116  8874 raft_consensus.cc:385] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.943526  8874 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.944234  8874 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:37.941167  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 47812.9us
I20260709 13:01:37.947575  8870 raft_consensus.cc:359] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.948387  8870 raft_consensus.cc:385] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:37.948742  8870 raft_consensus.cc:740] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:37.950169  8874 ts_tablet_manager.cc:1434] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.091s	user 0.009s	sys 0.018s
I20260709 13:01:37.949478  8870 consensus_queue.cc:260] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.958364  8874 tablet_bootstrap.cc:492] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:37.962503  8870 ts_tablet_manager.cc:1434] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.076s	user 0.005s	sys 0.022s
I20260709 13:01:37.967036  8870 tablet_bootstrap.cc:492] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:37.895373  9059 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:37.970232  9124 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:37.971517  9059 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:37.972570  9059 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.973461  8824 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:37.977487  8871 tablet_bootstrap.cc:492] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:37.978209  8871 ts_tablet_manager.cc:1403] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.125s	user 0.009s	sys 0.011s
I20260709 13:01:37.977299  9182 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.982576  9182 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:37.986291  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:37.987608  9182 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:01:37.990232  8645 leader_election.cc:343] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:37.996521  9191 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.997378  9191 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.004176  9186 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.006038  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:38.006909  9186 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.011847  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:38.013125  8677 raft_consensus.cc:2393] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 1: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:01:38.024365  9186 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:38.028390  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:38.028457  8824 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:38.040254  8578 catalog_manager.cc:5697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:38.042605  8870 tablet_bootstrap.cc:654] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:38.052410  8645 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:38.054214  8871 raft_consensus.cc:359] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.055279  8871 raft_consensus.cc:385] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:38.055529  8871 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:38.054795  9191 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:38.060168  9140 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:38.054432  9190 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:01:38.065299  8874 tablet_bootstrap.cc:654] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:38.066845  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:38.068872  8871 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.069525  9114 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:38.074944  8871 ts_tablet_manager.cc:1434] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.095s	user 0.006s	sys 0.002s
I20260709 13:01:38.080103  8647 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:38.081637  9054 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:38.082049  9054 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.081419  8576 catalog_manager.cc:5697] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:38.082423  9054 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:38.096760  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:01:38.094136  9211 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:01:38.105227  9054 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.112066  8678 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 1: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:01:38.119995  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:38.123659  8871 tablet_bootstrap.cc:492] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:38.132884  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:38.138377  9124 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.139429  9124 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.144585  9114 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:38.149760  8645 leader_election.cc:343] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:38.129604  8794 leader_election.cc:343] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:38.153108  9124 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:38.163863  9189 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.162992  9054 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:38.182273  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:38.192058  8874 tablet_bootstrap.cc:492] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:38.192772  8874 ts_tablet_manager.cc:1403] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.235s	user 0.021s	sys 0.010s
I20260709 13:01:38.195896  9216 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:38.205412  8871 tablet_bootstrap.cc:654] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:38.199596  8874 raft_consensus.cc:359] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.181970  9189 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.206102  8874 raft_consensus.cc:385] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:38.206538  8874 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:38.207859  9189 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:38.207582  8874 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.210392  8870 tablet_bootstrap.cc:492] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:38.212915  8870 ts_tablet_manager.cc:1403] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.246s	user 0.023s	sys 0.019s
I20260709 13:01:38.217034  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:38.234000  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 67668.9us
I20260709 13:01:38.238390  8952 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:01:38.257059  8952 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.257735  8952 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.262228  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:38.265055  9216 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:38.265622  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:38.268416  8874 ts_tablet_manager.cc:1434] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.075s	user 0.009s	sys 0.000s
I20260709 13:01:38.271842  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:38.285403  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:38.289296  8870 raft_consensus.cc:359] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.290169  8870 raft_consensus.cc:385] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:38.293212  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:38.290469  8870 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:38.272996  8952 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:38.298169  8870 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.300345  9129 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:38.303459  8874 tablet_bootstrap.cc:492] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:38.332033  8870 ts_tablet_manager.cc:1434] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.094s	user 0.008s	sys 0.000s
I20260709 13:01:38.338860  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:38.368423  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 73462us
I20260709 13:01:38.371500  8870 tablet_bootstrap.cc:492] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:38.381299  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:01:38.396425  8794 leader_election.cc:343] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:38.403656  8793 leader_election.cc:343] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:38.404687  8793 leader_election.cc:343] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:38.407383  8793 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:01:38.425773  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 128016us
I20260709 13:01:38.429365  9217 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:38.455030  8794 leader_election.cc:343] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:38.455650  8794 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:38.462221  8826 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:38.482143  8870 tablet_bootstrap.cc:654] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:38.477453  9059 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:01:38.492936  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 68518.4us
I20260709 13:01:38.503649  8874 tablet_bootstrap.cc:654] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:38.475167  9229 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.506929  9229 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.509675  9229 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:38.521119  8871 tablet_bootstrap.cc:492] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:38.521988  8871 ts_tablet_manager.cc:1403] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.399s	user 0.024s	sys 0.013s
I20260709 13:01:38.527127  8752 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:38.556413  8647 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:38.558336  9230 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:38.559100  9230 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:38.560140  9230 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.590590  8952 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:38.566176  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:38.664062  8645 leader_election.cc:343] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:38.679908  8871 raft_consensus.cc:359] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.680873  8871 raft_consensus.cc:385] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:38.681170  8871 raft_consensus.cc:740] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:38.686185  9216 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:38.686494  8871 consensus_queue.cc:260] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.703095  9114 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:01:38.706331  8870 tablet_bootstrap.cc:492] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:38.706948  8870 ts_tablet_manager.cc:1403] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.356s	user 0.052s	sys 0.054s
I20260709 13:01:38.710244  8874 tablet_bootstrap.cc:492] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:38.710371  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:38.716814  8874 ts_tablet_manager.cc:1403] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.414s	user 0.019s	sys 0.016s
I20260709 13:01:38.721515  9237 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.722095  9237 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:38.724375  8794 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:38.757503  8870 raft_consensus.cc:359] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.758370  8870 raft_consensus.cc:385] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:38.758002  8874 raft_consensus.cc:359] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.758718  8870 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:38.758827  8871 ts_tablet_manager.cc:1434] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.235s	user 0.012s	sys 0.021s
I20260709 13:01:38.758920  8874 raft_consensus.cc:385] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:38.759373  8874 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:38.759627  8870 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.761039  9237 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:38.762523  8870 ts_tablet_manager.cc:1434] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.055s	user 0.016s	sys 0.014s
I20260709 13:01:38.763134  8871 tablet_bootstrap.cc:492] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:38.764847  8870 tablet_bootstrap.cc:492] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
W20260709 13:01:38.768533  8647 leader_election.cc:343] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:38.769151  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:38.769826  9114 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:38.770396  9237 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:38.772251  8870 tablet_bootstrap.cc:654] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:38.773310  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:38.775090  8645 leader_election.cc:343] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:38.779219  8627 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:38.760128  8874 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.782749  8874 ts_tablet_manager.cc:1434] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.003s
I20260709 13:01:38.783882  8874 tablet_bootstrap.cc:492] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:38.784622  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:38.785377  8827 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:38.794346  8871 tablet_bootstrap.cc:654] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:38.811225  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:38.812788  9182 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:38.813237  9182 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.814981  9182 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:38.832041  9223 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.835862  9223 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.847764  8870 tablet_bootstrap.cc:492] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:38.848404  8870 ts_tablet_manager.cc:1403] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.085s	user 0.018s	sys 0.002s
I20260709 13:01:38.851624  9223 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:38.859865  8575 catalog_manager.cc:5697] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:38.891696  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:38.904628  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:38.909114  8874 tablet_bootstrap.cc:654] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:38.909968  8794 leader_election.cc:343] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:38.916251  8793 leader_election.cc:343] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:38.916720  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:38.918182  8871 tablet_bootstrap.cc:492] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:38.918747  8871 ts_tablet_manager.cc:1403] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.159s	user 0.017s	sys 0.011s
I20260709 13:01:38.920074  9234 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:38.930783  8871 raft_consensus.cc:359] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.935552  8870 raft_consensus.cc:359] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.945179  8870 raft_consensus.cc:385] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:38.945986  8870 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:38.933760  8871 raft_consensus.cc:385] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:38.948637  8871 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:38.949756  8870 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.957696  8870 ts_tablet_manager.cc:1434] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.109s	user 0.010s	sys 0.000s
I20260709 13:01:38.940513  9182 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.959203  8870 tablet_bootstrap.cc:492] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:38.957965  8871 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:38.964854  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:01:38.966447  8645 leader_election.cc:343] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:38.969377  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:38.970131  8827 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:38.991262  8871 ts_tablet_manager.cc:1434] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.001s
I20260709 13:01:38.992769  8871 tablet_bootstrap.cc:492] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:38.964854  9182 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:39.000134  8827 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:39.002703  8874 tablet_bootstrap.cc:492] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:39.003305  8874 ts_tablet_manager.cc:1403] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.220s	user 0.011s	sys 0.007s
I20260709 13:01:39.012171  8874 raft_consensus.cc:359] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.014716  8874 raft_consensus.cc:385] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.012570  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:39.015133  8874 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.016386  8874 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.017253  9230 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:39.017777  9230 raft_consensus.cc:697] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:39.018723  9230 consensus_queue.cc:237] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.019707  8874 ts_tablet_manager.cc:1434] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.004s
I20260709 13:01:39.020831  8874 tablet_bootstrap.cc:492] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:39.014360  8870 tablet_bootstrap.cc:654] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.036196  8871 tablet_bootstrap.cc:654] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.047592  8874 tablet_bootstrap.cc:654] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.055984  8870 tablet_bootstrap.cc:492] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:39.056785  8870 ts_tablet_manager.cc:1403] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.098s	user 0.018s	sys 0.004s
I20260709 13:01:39.065888  8870 raft_consensus.cc:359] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.066823  8870 raft_consensus.cc:385] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.067137  8870 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.067920  8870 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.071372  9186 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.076584  8870 ts_tablet_manager.cc:1434] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.002s
I20260709 13:01:39.076596  9186 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.078172  8870 tablet_bootstrap.cc:492] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:39.082280  9186 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:39.083580  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:39.084611  8830 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:39.086509  8870 tablet_bootstrap.cc:654] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.086531  8722 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:39.088675  9186 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:39.089579  9186 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.090121  9186 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.099751  8870 tablet_bootstrap.cc:492] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:39.100432  8870 ts_tablet_manager.cc:1403] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.007s
I20260709 13:01:39.103839  8870 raft_consensus.cc:359] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.104689  8870 raft_consensus.cc:385] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.105021  8870 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.106022  8870 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.109117  8870 ts_tablet_manager.cc:1434] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:01:39.110388  8870 tablet_bootstrap.cc:492] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:39.113049  9217 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.113852  9217 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.118337  9217 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:39.119844  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:39.121093  8870 tablet_bootstrap.cc:654] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.121312  8679 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:39.123358  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:39.126331  9217 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:39.127089  9217 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.127581  9217 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.136600  8870 tablet_bootstrap.cc:492] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:39.137275  8870 ts_tablet_manager.cc:1403] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.008s
I20260709 13:01:39.140848  8870 raft_consensus.cc:359] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.141654  8870 raft_consensus.cc:385] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.141930  8870 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.142640  8870 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.144932  8870 ts_tablet_manager.cc:1434] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:01:39.146088  8870 tablet_bootstrap.cc:492] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:39.153522  8870 tablet_bootstrap.cc:654] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.155778  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:39.156636  8748 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
W20260709 13:01:39.158605  8645 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:39.162273  8870 tablet_bootstrap.cc:492] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:39.164347  8870 ts_tablet_manager.cc:1403] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:01:39.166939  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:39.167882  9221 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.168419  9221 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.175223  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:01:39.182238  8793 leader_election.cc:343] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.167909  8679 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:39.187435  9186 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.231632  8870 raft_consensus.cc:359] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.232606  8870 raft_consensus.cc:385] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.232982  8870 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.235024  8874 tablet_bootstrap.cc:492] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:39.234938  8870 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.235703  8874 ts_tablet_manager.cc:1403] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.215s	user 0.031s	sys 0.031s
I20260709 13:01:39.244705  9222 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.249173  8870 ts_tablet_manager.cc:1434] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.084s	user 0.007s	sys 0.001s
I20260709 13:01:39.248983  9222 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.251456  9222 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:39.251961  9186 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:39.253633  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:39.254073  8870 tablet_bootstrap.cc:492] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:39.253918  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:39.254567  8827 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.237528  9217 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.255656  8576 catalog_manager.cc:5697] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:39.256878  8871 tablet_bootstrap.cc:492] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:39.257403  8871 ts_tablet_manager.cc:1403] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.265s	user 0.022s	sys 0.039s
W20260709 13:01:39.258975  8793 leader_election.cc:343] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.258240  8874 raft_consensus.cc:359] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.262456  8871 raft_consensus.cc:359] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.268118  8874 raft_consensus.cc:385] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.268152  8871 raft_consensus.cc:385] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.268574  8874 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.268617  8871 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.272470  8874 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.274852  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:39.259817  9221 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:39.282043  8874 ts_tablet_manager.cc:1434] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.001s
I20260709 13:01:39.285151  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:39.291453  8871 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.282243  9230 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:01:39.282457  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:01:39.287736  8794 leader_election.cc:343] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.294139  8794 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:39.297673  8681 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.305176  8874 tablet_bootstrap.cc:492] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:39.308698  8871 ts_tablet_manager.cc:1434] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.001s
I20260709 13:01:39.308954  9217 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:39.310043  8871 tablet_bootstrap.cc:492] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:39.311089  8870 tablet_bootstrap.cc:654] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.313144  8827 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:39.313941  8874 tablet_bootstrap.cc:654] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.328728  9234 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:39.331300  8722 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:01:39.331802  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 44111.5us
I20260709 13:01:39.332307  9186 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:39.332948  9186 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:39.333981  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:39.335104  8676 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.333947  9186 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:39.346863  8794 leader_election.cc:343] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.347414  8794 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:39.349447  8681 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:39.350982  9234 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:39.360008  8870 tablet_bootstrap.cc:492] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:39.360555  8870 ts_tablet_manager.cc:1403] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.107s	user 0.026s	sys 0.025s
I20260709 13:01:39.362112  8676 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:39.364771  8870 raft_consensus.cc:359] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.365602  8870 raft_consensus.cc:385] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.365904  8870 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.365705  8578 catalog_manager.cc:5697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:39.366655  8870 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.369061  8870 ts_tablet_manager.cc:1434] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:01:39.370224  8870 tablet_bootstrap.cc:492] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:39.377566  8870 tablet_bootstrap.cc:654] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.380115  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:39.380831  8748 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.385769  9234 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.387187  9234 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.389822  9234 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:39.390565  8870 tablet_bootstrap.cc:492] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:39.391214  8870 ts_tablet_manager.cc:1403] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260709 13:01:39.392172  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:39.393569  9217 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:39.394459  8870 raft_consensus.cc:359] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.395282  8870 raft_consensus.cc:385] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.395290  9217 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:39.395608  8870 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.396427  8748 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:39.396534  8870 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.398432  8870 ts_tablet_manager.cc:1434] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:01:39.399610  8870 tablet_bootstrap.cc:492] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
I20260709 13:01:39.401816  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:39.402004  9217 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.406867  8870 tablet_bootstrap.cc:654] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.408474  8871 tablet_bootstrap.cc:654] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.409827  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:39.410681  8681 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:39.413568  8793 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:39.415176  9234 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:39.415872  9234 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.416782  9234 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.425647  8874 tablet_bootstrap.cc:492] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:39.426203  8874 ts_tablet_manager.cc:1403] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.121s	user 0.016s	sys 0.006s
I20260709 13:01:39.436157  8870 tablet_bootstrap.cc:492] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
W20260709 13:01:39.430524  8794 leader_election.cc:343] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.434573  9234 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.446000  8870 ts_tablet_manager.cc:1403] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.047s	user 0.018s	sys 0.018s
I20260709 13:01:39.447784  9234 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:39.448529  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:39.449210  8676 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.429239  8874 raft_consensus.cc:359] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.449690  8874 raft_consensus.cc:385] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.449983  8874 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.429283  8871 tablet_bootstrap.cc:492] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:39.451066  8871 ts_tablet_manager.cc:1403] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.141s	user 0.005s	sys 0.013s
I20260709 13:01:39.450595  8870 raft_consensus.cc:359] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.451534  8870 raft_consensus.cc:385] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.451844  8870 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.452553  8870 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.453969  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:39.454133  9234 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.453514  8874 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.455868  9191 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.456411  9191 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.456063  9234 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.459785  9191 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:39.460611  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:01:39.466427  8793 leader_election.cc:343] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.468262  8676 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:39.471306  8793 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:39.471441  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:39.472119  9191 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.473021  9191 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.472122  8870 ts_tablet_manager.cc:1434] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:01:39.475162  9191 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:39.475713  9222 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:39.476260  9222 raft_consensus.cc:697] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:39.476819  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:39.477494  8676 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 0.
I20260709 13:01:39.477438  9222 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.479075  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:39.479959  9124 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:39.480346  9124 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.480472  9234 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:39.480746  9124 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.481148  8871 raft_consensus.cc:359] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.481889  8871 raft_consensus.cc:385] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.482161  8871 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.482489  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:39.483058  8874 ts_tablet_manager.cc:1434] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.007s
I20260709 13:01:39.484215  8874 tablet_bootstrap.cc:492] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:39.490903  8871 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:39.491561  8793 leader_election.cc:343] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.476194  8870 tablet_bootstrap.cc:492] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Bootstrap starting.
W20260709 13:01:39.493410  8794 leader_election.cc:343] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:39.496028  8794 leader_election.cc:343] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.496512  8794 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:39.498374  9190 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:39.492437  9124 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.509421  9124 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:39.515892  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:39.516701  8870 tablet_bootstrap.cc:654] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.519503  8871 ts_tablet_manager.cc:1434] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.068s	user 0.004s	sys 0.004s
I20260709 13:01:39.520839  8871 tablet_bootstrap.cc:492] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
W20260709 13:01:39.521821  8645 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:39.522276  9190 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.522818  9190 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.519618  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:39.525126  9190 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:39.526970  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:39.527588  8676 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.528940  8870 tablet_bootstrap.cc:492] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: No bootstrap required, opened a new log
I20260709 13:01:39.529506  8870 ts_tablet_manager.cc:1403] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.006s
I20260709 13:01:39.530977  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:39.532420  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:39.533263  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:39.534725  8794 leader_election.cc:343] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:39.536026  8793 leader_election.cc:343] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.536042  8870 raft_consensus.cc:359] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.537262  8870 raft_consensus.cc:385] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.537529  8870 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.538239  8870 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:39.539927  8794 leader_election.cc:343] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.540544  8794 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:39.541082  8870 ts_tablet_manager.cc:1434] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:01:39.544773  9190 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:39.546172  8874 tablet_bootstrap.cc:654] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:39.547318  8794 leader_election.cc:343] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:01:39.548156  8794 leader_election.cc:343] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.548590  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:39.549338  9190 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:39.559760  8871 tablet_bootstrap.cc:654] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.553253  8676 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:01:39.561789  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:39.566953  8871 tablet_bootstrap.cc:492] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:39.567560  8871 ts_tablet_manager.cc:1403] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.000s
I20260709 13:01:39.569572  8874 tablet_bootstrap.cc:492] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:39.570129  8874 ts_tablet_manager.cc:1403] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.086s	user 0.010s	sys 0.000s
I20260709 13:01:39.570596  8871 raft_consensus.cc:359] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.571528  8871 raft_consensus.cc:385] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.571803  8871 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.572649  8871 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.573220  8874 raft_consensus.cc:359] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.566915  9124 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:39.574865  8871 ts_tablet_manager.cc:1434] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:01:39.574847  9124 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:01:39.578599  9236 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.579239  9236 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.581838  9236 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:39.584408  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:39.576082  9124 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:39.586030  8645 leader_election.cc:343] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.592003  8871 tablet_bootstrap.cc:492] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:39.599375  8871 tablet_bootstrap.cc:654] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.599664  9182 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:39.611214  8753 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.614027  8871 tablet_bootstrap.cc:492] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:39.615254  8871 ts_tablet_manager.cc:1403] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.009s
I20260709 13:01:39.617326  8952 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.617871  8952 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.618575  8871 raft_consensus.cc:359] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.619343  8871 raft_consensus.cc:385] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.619624  8871 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.619889  8952 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:39.620306  8871 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.621526  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:39.622218  8827 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 0.
I20260709 13:01:39.623525  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:39.623838  8871 ts_tablet_manager.cc:1434] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:01:39.624895  8871 tablet_bootstrap.cc:492] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:39.625963  8952 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:39.626456  8952 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.627409  8952 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.632421  8871 tablet_bootstrap.cc:654] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.626483  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:39.632102  8576 catalog_manager.cc:5697] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:39.634720  8719 leader_election.cc:343] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.573889  8874 raft_consensus.cc:385] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.635529  8874 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.636667  8874 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.639007  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:39.641824  8827 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:39.647156  8871 tablet_bootstrap.cc:492] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:39.647810  8871 ts_tablet_manager.cc:1403] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.003s
I20260709 13:01:39.651324  8871 raft_consensus.cc:359] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.652102  8871 raft_consensus.cc:385] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.652403  8871 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.653120  8871 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.653573  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:39.654847  9228 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:39.655248  9228 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.655577  9228 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.655876  8871 ts_tablet_manager.cc:1434] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:01:39.661715  8874 ts_tablet_manager.cc:1434] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.091s	user 0.003s	sys 0.005s
I20260709 13:01:39.663045  8874 tablet_bootstrap.cc:492] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:39.666740  9228 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.668416  8576 catalog_manager.cc:5697] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:39.670419  8576 catalog_manager.cc:5697] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:39.674311  9228 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:39.676601  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:39.677393  8827 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.679194  8874 tablet_bootstrap.cc:654] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.689229  8827 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:39.690945  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:39.691921  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:39.692142  9228 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:39.692895  9228 raft_consensus.cc:697] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:39.694056  9228 consensus_queue.cc:237] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:39.697039  8645 leader_election.cc:343] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.709635  8871 tablet_bootstrap.cc:492] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:39.719030  8871 tablet_bootstrap.cc:654] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.711599  8952 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.736229  8952 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:39.733363  8871 tablet_bootstrap.cc:492] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:39.736686  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:01:39.737694  8677 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.737183  8871 ts_tablet_manager.cc:1403] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.009s
I20260709 13:01:39.744637  8871 raft_consensus.cc:359] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.745682  8871 raft_consensus.cc:385] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.746069  8871 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.749213  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:39.749941  8830 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.758651  8871 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.759776  9186 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:39.761103  8871 ts_tablet_manager.cc:1434] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:01:39.762339  8871 tablet_bootstrap.cc:492] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:39.761730  8830 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:39.769327  8874 tablet_bootstrap.cc:492] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:39.769943  8874 ts_tablet_manager.cc:1403] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.107s	user 0.020s	sys 0.035s
I20260709 13:01:39.774080  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:39.775264  9186 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:39.772964  8874 raft_consensus.cc:359] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.775949  8874 raft_consensus.cc:385] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.776259  8874 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.778338  9186 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:01:39.779497  9186 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.781466  8677 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:01:39.788332  9228 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.789727  8871 tablet_bootstrap.cc:654] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.789216  9228 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.807416  9228 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:39.808755  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:39.809049  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:39.809518  8827 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:39.810662  9236 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.811203  9236 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.813400  9236 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:39.814951  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:39.815193  8874 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.817049  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:39.817485  8874 ts_tablet_manager.cc:1434] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.047s	user 0.002s	sys 0.005s
I20260709 13:01:39.817720  8827 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:39.818912  8874 tablet_bootstrap.cc:492] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:39.819099  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:39.819999  9228 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:39.820384  9228 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.820734  9228 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:39.821112  8645 leader_election.cc:343] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.823828  9236 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:39.825217  9236 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.825693  9236 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.824600  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:39.828009  8645 leader_election.cc:343] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.852671  8874 tablet_bootstrap.cc:654] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.851046  8578 catalog_manager.cc:5697] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:39.852859  9186 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:39.859213  9228 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.859987  8753 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.864959  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:39.865814  8827 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.880981  8575 catalog_manager.cc:5697] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:39.892108  9218 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.892756  9218 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.895762  9218 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:39.896097  9237 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.896672  9237 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.897307  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:39.899153  9237 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:39.901216  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:39.902096  8829 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:39.903079  8874 tablet_bootstrap.cc:492] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:39.903694  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:39.903834  8874 ts_tablet_manager.cc:1403] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.085s	user 0.013s	sys 0.009s
I20260709 13:01:39.908188  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:39.907806  8874 raft_consensus.cc:359] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:39.908876  8627 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:01:39.908934  8874 raft_consensus.cc:385] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.909446  8874 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.908573  8871 tablet_bootstrap.cc:492] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:39.910907  9230 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:39.911381  9230 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.911830  9230 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.910944  8874 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:01:39.925990  8645 leader_election.cc:343] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.927503  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:01:39.935245  8794 leader_election.cc:343] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.937340  9218 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:39.911356  8871 ts_tablet_manager.cc:1403] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.149s	user 0.011s	sys 0.004s
I20260709 13:01:39.939370  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:39.941824  8871 raft_consensus.cc:359] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.942673  8871 raft_consensus.cc:385] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:39.943019  8871 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:39.874462  9236 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.944479  8874 ts_tablet_manager.cc:1434] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.000s
I20260709 13:01:39.866146  9228 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:01:39.950206  8794 consensus_peers.cc:597] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:39.950341  9114 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.950727  8874 tablet_bootstrap.cc:492] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:39.950973  9114 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.954130  9114 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:39.930408  9230 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.955631  8677 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 1: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:01:39.956835  8827 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:39.958137  9236 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:39.958094  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:39.960884  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:01:39.961364  8645 leader_election.cc:343] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:39.961535  8827 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:39.963064  8871 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.969807  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:39.959733  9228 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:39.971230  9228 raft_consensus.cc:697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:39.972221  9228 consensus_queue.cc:237] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:39.975667  9230 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:39.980770  8871 ts_tablet_manager.cc:1434] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 13:01:39.980123  8827 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:39.982199  8871 tablet_bootstrap.cc:492] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:39.982072  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:39.982893  8830 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:01:39.985159  8794 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:39.985597  8874 tablet_bootstrap.cc:654] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:39.985970  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:39.988292  9230 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:39.988839  9230 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:39.992302  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:39.993542  8793 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:39.993655  9230 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.001230  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:40.001650  9182 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.002209  9182 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.004885  9182 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.005019  9217 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:40.008054  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:40.008940  8827 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:40.010131  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:40.011093  9182 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:40.011538  9182 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.012529  9182 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.015425  8830 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:40.017297  9190 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:01:40.018088  8645 leader_election.cc:343] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:40.018860  9218 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:40.019457  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:40.022734  8753 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.022923  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:40.026463  8645 leader_election.cc:343] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:40.027143  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:01:40.031018  8645 leader_election.cc:343] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:40.034257  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:40.036194  8952 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:40.033066  8578 catalog_manager.cc:5697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:40.042282  9230 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:40.058828  9230 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:40.067737  9230 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.078191  9182 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.087500  9182 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.094741  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:40.095264  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:40.095768  8827 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.103039  8874 tablet_bootstrap.cc:492] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:40.103685  8874 ts_tablet_manager.cc:1403] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.153s	user 0.028s	sys 0.008s
W20260709 13:01:40.105538  8645 leader_election.cc:343] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:40.115428  8871 tablet_bootstrap.cc:654] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:40.108816  8874 raft_consensus.cc:359] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.117270  8874 raft_consensus.cc:385] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:40.126775  9237 consensus_queue.cc:1048] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:40.117746  8874 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
W20260709 13:01:40.133081  8645 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:40.128844  9292 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:01:40.140591  8874 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.155694  8874 ts_tablet_manager.cc:1434] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.002s
I20260709 13:01:40.139470  8827 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:40.172049  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:40.191692  9186 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:40.193394  8748 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.195080  9236 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:40.225204  8874 tablet_bootstrap.cc:492] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:40.226969  9236 raft_consensus.cc:697] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:40.263990  9236 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.273784  8871 tablet_bootstrap.cc:492] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:40.274501  8871 ts_tablet_manager.cc:1403] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.293s	user 0.027s	sys 0.014s
I20260709 13:01:40.271577  8578 catalog_manager.cc:5697] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:40.276446  8578 catalog_manager.cc:5697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:40.307538  8871 raft_consensus.cc:359] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.308454  8871 raft_consensus.cc:385] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:40.308815  8871 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:40.307971  8874 tablet_bootstrap.cc:654] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:40.313059  8871 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.315834  8871 ts_tablet_manager.cc:1434] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 13:01:40.317158  8871 tablet_bootstrap.cc:492] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:40.324885  8871 tablet_bootstrap.cc:654] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
W20260709 13:01:40.326761  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 92525.6us
I20260709 13:01:40.328050  8952 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:40.328028  8575 catalog_manager.cc:5697] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:40.337873  9302 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:40.360319  8952 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.360862  8952 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.362954  8952 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.364362  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:40.366905  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:40.367842  8829 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 1: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:01:40.379248  8722 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:40.380010  8952 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.387341  8874 tablet_bootstrap.cc:492] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:40.387943  8874 ts_tablet_manager.cc:1403] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.172s	user 0.009s	sys 0.006s
I20260709 13:01:40.388984  8952 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:40.404157  8871 tablet_bootstrap.cc:492] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:40.404786  8871 ts_tablet_manager.cc:1403] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.088s	user 0.015s	sys 0.014s
I20260709 13:01:40.408124  8871 raft_consensus.cc:359] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.408926  8871 raft_consensus.cc:385] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:40.409328  8871 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:40.410115  8871 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.411687  8874 raft_consensus.cc:359] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.412505  8874 raft_consensus.cc:385] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:40.412789  8871 ts_tablet_manager.cc:1434] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:01:40.412799  8874 raft_consensus.cc:740] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:40.414187  8871 tablet_bootstrap.cc:492] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Bootstrap starting.
I20260709 13:01:40.413784  8874 consensus_queue.cc:260] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.417829  8874 ts_tablet_manager.cc:1434] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.030s	user 0.010s	sys 0.005s
I20260709 13:01:40.419062  8874 tablet_bootstrap.cc:492] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:40.419656  9236 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.420226  9236 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.422462  9236 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.424341  8871 tablet_bootstrap.cc:654] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:40.424396  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:40.426131  8645 leader_election.cc:343] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:40.432546  8874 tablet_bootstrap.cc:654] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:40.438366  8871 tablet_bootstrap.cc:492] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: No bootstrap required, opened a new log
I20260709 13:01:40.438957  8871 ts_tablet_manager.cc:1403] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.011s
I20260709 13:01:40.442174  8871 raft_consensus.cc:359] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.442992  8871 raft_consensus.cc:385] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:40.443289  8871 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Initialized, Role: FOLLOWER
I20260709 13:01:40.444145  8871 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.446384  8871 ts_tablet_manager.cc:1434] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:01:40.454034  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:40.455125  8829 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:40.459342  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:40.460597  9182 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:40.461056  9182 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.461407  9182 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.502898  9182 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.513757  8952 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.514354  8952 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.515031  8874 tablet_bootstrap.cc:492] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:40.515566  8874 ts_tablet_manager.cc:1403] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.097s	user 0.019s	sys 0.012s
I20260709 13:01:40.515828  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:40.516396  8952 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.520503  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:01:40.522555  8645 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:40.527348  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:40.528060  8827 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.528837  9140 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:40.532948  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:01:40.536726  8645 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:40.541030  8719 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:40.543419  8952 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:40.543574  9182 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.518606  8874 raft_consensus.cc:359] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.548064  8874 raft_consensus.cc:385] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:40.548338  8874 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:40.549089  8874 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.550777  8752 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.551498  8874 ts_tablet_manager.cc:1434] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 13:01:40.552591  8874 tablet_bootstrap.cc:492] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:40.541397  8827 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:40.567458  8874 tablet_bootstrap.cc:654] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:40.579831  8874 tablet_bootstrap.cc:492] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:40.580034  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:40.580394  8874 ts_tablet_manager.cc:1403] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.007s
I20260709 13:01:40.581238  9140 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:40.581734  9140 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:40.582403  9140 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.588754  8874 raft_consensus.cc:359] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.589617  8874 raft_consensus.cc:385] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:40.589960  8874 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:40.591445  8874 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.593746  8874 ts_tablet_manager.cc:1434] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.004s
I20260709 13:01:40.593299  8752 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:40.595243  8874 tablet_bootstrap.cc:492] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
W20260709 13:01:40.603251  8645 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:40.607069  8874 tablet_bootstrap.cc:654] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:40.615689  9140 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.616323  9140 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.619074  9140 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.621677  8578 catalog_manager.cc:5697] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:40.622296  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:01:40.625586  8645 leader_election.cc:343] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:40.627564  8874 tablet_bootstrap.cc:492] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:40.628136  8874 ts_tablet_manager.cc:1403] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.033s	user 0.006s	sys 0.012s
I20260709 13:01:40.628649  9140 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:40.631721  8874 raft_consensus.cc:359] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.632578  8874 raft_consensus.cc:385] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:40.632867  8874 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:40.634418  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:40.635159  8827 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:40.633574  8874 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.637050  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:40.637519  8874 ts_tablet_manager.cc:1434] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 13:01:40.638237  9182 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:40.638666  8874 tablet_bootstrap.cc:492] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:40.638754  9182 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.639245  9182 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.646270  8874 tablet_bootstrap.cc:654] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:40.649042  9140 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:40.665954  9182 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.672550  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:40.673383  8829 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.673374  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:01:40.680497  8645 leader_election.cc:343] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:01:40.683529  8829 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:40.685593  9182 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.686944  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:01:40.687783  9182 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:40.688272  9182 raft_consensus.cc:697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:40.689114  9182 consensus_queue.cc:237] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.694356  9186 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.694988  9186 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.697263  9186 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.699352  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:40.699595  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:40.701663  8719 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:40.703708  8952 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:40.718564  8756 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.722452  9236 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:40.731618  8576 catalog_manager.cc:5697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:40.740731  8874 tablet_bootstrap.cc:492] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:40.741451  8874 ts_tablet_manager.cc:1403] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.103s	user 0.022s	sys 0.009s
I20260709 13:01:40.750592  9237 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:40.751683  8874 raft_consensus.cc:359] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.752527  8874 raft_consensus.cc:385] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:40.752856  8874 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:40.753716  8874 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.757037  8874 ts_tablet_manager.cc:1434] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:01:40.762202  8874 tablet_bootstrap.cc:492] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:40.782938  8874 tablet_bootstrap.cc:654] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:40.797669  8874 tablet_bootstrap.cc:492] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:40.798347  8874 ts_tablet_manager.cc:1403] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.005s
I20260709 13:01:40.800860  8874 raft_consensus.cc:359] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.801471  8874 raft_consensus.cc:385] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:40.801729  8874 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:40.802510  8874 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.805485  8874 ts_tablet_manager.cc:1434] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:01:40.805405  9182 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.806083  9182 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.808631  9182 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.810572  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:40.811816  8827 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:40.812813  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:40.813612  9182 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:01:40.814177  9182 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.814585  9182 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.814301  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:01:40.815393  8752 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 0.
I20260709 13:01:40.816644  8874 tablet_bootstrap.cc:492] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:40.821575  9186 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.822448  9186 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.825301  9182 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.828639  9182 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.829053  8874 tablet_bootstrap.cc:654] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:40.830158  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:01:40.830866  8756 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.831519  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:01:40.832129  8827 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.831914  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:40.835538  9186 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.839833  8827 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:40.841153  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:40.843034  8719 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:40.843605  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:01:40.846374  9182 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:01:40.847190  9182 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:01:40.849355  9186 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:40.850176  9182 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.842258  8756 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:01:40.860561  8874 tablet_bootstrap.cc:492] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:40.861207  8874 ts_tablet_manager.cc:1403] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.045s	user 0.019s	sys 0.015s
I20260709 13:01:40.865654  8874 raft_consensus.cc:359] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.866472  8874 raft_consensus.cc:385] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:40.870361  8874 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:40.871153  8874 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.873034  8874 ts_tablet_manager.cc:1434] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:01:40.873638  8576 catalog_manager.cc:5697] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:40.874964  8874 tablet_bootstrap.cc:492] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:40.881985  8874 tablet_bootstrap.cc:654] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:40.893532  8874 tablet_bootstrap.cc:492] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:40.894034  8874 ts_tablet_manager.cc:1403] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.007s
I20260709 13:01:40.896834  8874 raft_consensus.cc:359] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.898310  8874 raft_consensus.cc:385] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:40.898663  8874 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:40.900482  8874 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.904650  8874 ts_tablet_manager.cc:1434] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:01:40.905884  8874 tablet_bootstrap.cc:492] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:40.906169  8752 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.907325  9186 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.908157  9186 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.911875  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:40.912775  9114 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.913344  9114 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.914682  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:40.915712  8827 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 1: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:01:40.917014  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:40.918805  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:40.918795  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:40.920524  8952 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:40.920449  9114 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.924600  8874 tablet_bootstrap.cc:654] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:40.928000  9186 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.930276  8952 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:40.931485  9186 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.931598  8722 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:40.932085  9186 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.933410  8952 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:40.934574  9186 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:40.935432  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:01:40.937788  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:01:40.939946  8722 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:01:40.938088  8874 tablet_bootstrap.cc:492] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:40.940959  8874 ts_tablet_manager.cc:1403] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.001s
I20260709 13:01:40.941283  8952 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:01:40.946085  8874 raft_consensus.cc:359] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.946877  8874 raft_consensus.cc:385] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:40.947646  8874 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:40.949051  8874 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.952597  8874 ts_tablet_manager.cc:1434] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:01:40.954514  8874 tablet_bootstrap.cc:492] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: Bootstrap starting.
I20260709 13:01:40.962702  8874 tablet_bootstrap.cc:654] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: Neither blocks nor log segments found. Creating new log.
I20260709 13:01:40.978564  8874 tablet_bootstrap.cc:492] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: No bootstrap required, opened a new log
I20260709 13:01:40.979709  8874 ts_tablet_manager.cc:1403] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent bootstrapping tablet: real 0.025s	user 0.005s	sys 0.014s
I20260709 13:01:40.983342  8874 raft_consensus.cc:359] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.984128  8874 raft_consensus.cc:385] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:01:40.984449  8874 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Initialized, Role: FOLLOWER
I20260709 13:01:40.985275  8874 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:01:40.989871  8874 ts_tablet_manager.cc:1434] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:01:41.005734  8752 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:41.016268  8753 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:41.046762  8753 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:41.131445  9237 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:41.144768  9237 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:41.158175  8752 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:41.212025  9237 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:41.214718  8753 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:01:41.227244  9182 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:41.385068  9236 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:01:41.403383  9140 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:01:41.621855  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:01:41.665501  9346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:41.666054  9344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:01:41.667273  9343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:01:41.670833  7526 server_base.cc:1061] running on GCE node
I20260709 13:01:41.673106  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:01:41.673466  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:01:41.673710  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602101673690 us; error 0 us; skew 500 ppm
I20260709 13:01:41.674681  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:01:41.684885  7526 webserver.cc:533] Webserver started at http://127.7.89.132:39809/ using document root <none> and password file <none>
I20260709 13:01:41.685702  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:01:41.685989  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:01:41.686415  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:01:41.688336  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-3-root/instance:
uuid: "7aff04f53d4845a6a4fa491fd2a06145"
format_stamp: "Formatted at 2026-07-09 13:01:41 on dist-test-slave-h0rz"
I20260709 13:01:41.696206  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:01:41.702412  9351 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:41.703576  7526 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.001s
I20260709 13:01:41.704011  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-3-root
uuid: "7aff04f53d4845a6a4fa491fd2a06145"
format_stamp: "Formatted at 2026-07-09 13:01:41 on dist-test-slave-h0rz"
I20260709 13:01:41.704475  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:01:41.723879  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:01:41.726166  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:01:41.728375  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:01:41.731575  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:01:41.731830  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:41.732048  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:01:41.732193  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:01:41.835971  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.132:41749
I20260709 13:01:41.836103  9413 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.132:41749 every 8 connection(s)
I20260709 13:01:41.900106  7526 auto_leader_rebalancer-test.cc:384] add a tserver: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:41.920089  9414 heartbeater.cc:344] Connected to a master server at 127.7.89.190:45099
I20260709 13:01:41.920689  9414 heartbeater.cc:461] Registering TS with master...
I20260709 13:01:41.921723  9414 heartbeater.cc:507] Master 127.7.89.190:45099 requested a full tablet report, sending...
I20260709 13:01:41.924160  8576 ts_manager.cc:194] Registered new tserver with Master: 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:01:41.925998  8576 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40096
I20260709 13:01:42.126921  8829 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:42.134303  8756 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:42.137494  9190 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:42.139823  8677 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:42.141304  9190 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:42.151041  8796 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 20fcd6d0c3304abda83f416721e807c5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:42.152420  9190 raft_consensus.cc:2955] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:42.159914  8681 raft_consensus.cc:2955] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:42.153795  8756 raft_consensus.cc:2955] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:42.183558  8576 catalog_manager.cc:5697] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:42.226660  8681 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:42.255868  8756 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:42.258776  9236 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:42.263293  8827 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:42.265398  9236 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:01:42.277508  9427 raft_consensus.cc:2955] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:42.279279  8756 raft_consensus.cc:2955] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
W20260709 13:01:42.294685  8647 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac4bab9153b047238bffb6abbeb83601. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:42.296100  8827 raft_consensus.cc:2955] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:42.299427  8829 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:42.310155  8578 catalog_manager.cc:5697] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:42.317271  8756 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:42.318996  9340 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:01:42.320598  8676 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:42.322706  9428 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:42.323977  8796 consensus_peers.cc:597] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1e6257b62c5c415aad31f5fc646ce4d4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:42.334650  9340 raft_consensus.cc:2955] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:42.337359  8681 raft_consensus.cc:2955] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:42.339838  8756 raft_consensus.cc:2955] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:42.368415  8575 catalog_manager.cc:5697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:42.369310  8752 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:42.391176  8681 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:42.400175  8829 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:42.394055  9327 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:01:42.403168  9327 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:42.419790  8722 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 389e9e63581e4d1b941ba995f8cdbfc4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:42.423353  9327 raft_consensus.cc:2955] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:42.430711  8676 raft_consensus.cc:2955] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:42.434048  8829 raft_consensus.cc:2955] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:42.444775  8576 catalog_manager.cc:5697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:42.752821  9449 ts_tablet_manager.cc:933] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:42.765676  9449 tablet_copy_client.cc:323] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:01:42.852787  9452 ts_tablet_manager.cc:933] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221)
I20260709 13:01:42.865522  9452 tablet_copy_client.cc:323] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.129:43221
I20260709 13:01:42.929486  9414 heartbeater.cc:499] Master 127.7.89.190:45099 was elected leader, sending a full tablet report...
I20260709 13:01:42.944597  8840 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet 20fcd6d0c3304abda83f416721e807c5 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
I20260709 13:01:42.945459  8840 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet 20fcd6d0c3304abda83f416721e807c5 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-20fcd6d0c3304abda83f416721e807c5
I20260709 13:01:42.968863  9458 ts_tablet_manager.cc:933] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:42.981084  8691 tablet_copy_service.cc:140] P 0e8c59a1f0394eae8ce33b99f10efadc: Received BeginTabletCopySession request for tablet ac4bab9153b047238bffb6abbeb83601 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55676)
I20260709 13:01:42.981992  8691 tablet_copy_service.cc:161] P 0e8c59a1f0394eae8ce33b99f10efadc: Beginning new tablet copy session on tablet ac4bab9153b047238bffb6abbeb83601 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55676: session id = 7aff04f53d4845a6a4fa491fd2a06145-ac4bab9153b047238bffb6abbeb83601
I20260709 13:01:42.983250  9458 tablet_copy_client.cc:323] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:01:42.985605  8839 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet 1e6257b62c5c415aad31f5fc646ce4d4 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
I20260709 13:01:42.986347  8839 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet 1e6257b62c5c415aad31f5fc646ce4d4 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-1e6257b62c5c415aad31f5fc646ce4d4
I20260709 13:01:42.998333  8840 tablet_copy_source_session.cc:215] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:43.002072  8839 tablet_copy_source_session.cc:215] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:43.003475  8691 tablet_copy_source_session.cc:215] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:43.004904  9449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20fcd6d0c3304abda83f416721e807c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:43.013101  9452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac4bab9153b047238bffb6abbeb83601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:43.029592  9458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e6257b62c5c415aad31f5fc646ce4d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:43.012996  9460 ts_tablet_manager.cc:933] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:43.041240  9460 tablet_copy_client.cc:323] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.130:42321
I20260709 13:01:43.077160  9449 tablet_copy_client.cc:806] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:43.069545  9458 tablet_copy_client.cc:806] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:43.079005  9449 tablet_copy_client.cc:670] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:43.079589  9458 tablet_copy_client.cc:670] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:43.085842  9449 tablet_copy_client.cc:538] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:43.085664  9452 tablet_copy_client.cc:806] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:43.087024  9452 tablet_copy_client.cc:670] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:43.085770  9458 tablet_copy_client.cc:538] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:43.099532  9452 tablet_copy_client.cc:538] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:43.100478  9449 tablet_bootstrap.cc:492] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:43.103973  8766 tablet_copy_service.cc:140] P 0e9b0850145a4c66bfe983ffdfa93a57: Received BeginTabletCopySession request for tablet 389e9e63581e4d1b941ba995f8cdbfc4 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:35702)
I20260709 13:01:43.104765  8766 tablet_copy_service.cc:161] P 0e9b0850145a4c66bfe983ffdfa93a57: Beginning new tablet copy session on tablet 389e9e63581e4d1b941ba995f8cdbfc4 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:35702: session id = 7aff04f53d4845a6a4fa491fd2a06145-389e9e63581e4d1b941ba995f8cdbfc4
I20260709 13:01:43.109611  9458 tablet_bootstrap.cc:492] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:43.129658  9452 tablet_bootstrap.cc:492] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:43.151835  8766 tablet_copy_source_session.cc:215] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:43.163621  9460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389e9e63581e4d1b941ba995f8cdbfc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:43.198694  9458 tablet_bootstrap.cc:492] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:43.202961  9458 tablet_bootstrap.cc:492] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:43.205760  9458 ts_tablet_manager.cc:1403] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.096s	user 0.029s	sys 0.016s
I20260709 13:01:43.202986  9449 tablet_bootstrap.cc:492] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:43.217872  9449 tablet_bootstrap.cc:492] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:43.218834  9449 ts_tablet_manager.cc:1403] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.119s	user 0.029s	sys 0.011s
I20260709 13:01:43.222569  9449 raft_consensus.cc:359] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:43.223554  9449 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:43.213768  9458 raft_consensus.cc:359] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:43.225581  9458 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:43.226366  9458 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:43.224334  9449 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:43.228709  9452 tablet_bootstrap.cc:492] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:43.231595  9452 tablet_bootstrap.cc:492] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:43.232663  9452 ts_tablet_manager.cc:1403] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.103s	user 0.029s	sys 0.014s
I20260709 13:01:43.260936  9452 raft_consensus.cc:359] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:43.262074  9452 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:43.272686  9452 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:43.286366  9458 ts_tablet_manager.cc:1434] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.080s	user 0.036s	sys 0.017s
I20260709 13:01:43.293701  8840 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-1e6257b62c5c415aad31f5fc646ce4d4 received from {username='slave'} at 127.0.0.1:55168
I20260709 13:01:43.294397  8840 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-1e6257b62c5c415aad31f5fc646ce4d4 on tablet 1e6257b62c5c415aad31f5fc646ce4d4 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:43.335068  9452 ts_tablet_manager.cc:1434] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.102s	user 0.010s	sys 0.003s
I20260709 13:01:43.337692  8691 tablet_copy_service.cc:342] P 0e8c59a1f0394eae8ce33b99f10efadc: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-ac4bab9153b047238bffb6abbeb83601 received from {username='slave'} at 127.0.0.1:55676
I20260709 13:01:43.338220  8691 tablet_copy_service.cc:434] P 0e8c59a1f0394eae8ce33b99f10efadc: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-ac4bab9153b047238bffb6abbeb83601 on tablet ac4bab9153b047238bffb6abbeb83601 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:43.357935  9449 ts_tablet_manager.cc:1434] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.139s	user 0.013s	sys 0.040s
I20260709 13:01:43.360715  8840 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-20fcd6d0c3304abda83f416721e807c5 received from {username='slave'} at 127.0.0.1:55168
I20260709 13:01:43.361400  8840 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-20fcd6d0c3304abda83f416721e807c5 on tablet 20fcd6d0c3304abda83f416721e807c5 with peer 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:01:43.363370  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 48806.6us
I20260709 13:01:43.389050  9460 tablet_copy_client.cc:806] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:43.389967  9460 tablet_copy_client.cc:670] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:43.394930  9460 tablet_copy_client.cc:538] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:43.407359  9460 tablet_bootstrap.cc:492] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:43.437506  9460 tablet_bootstrap.cc:492] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:43.438706  9460 tablet_bootstrap.cc:492] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:43.439500  9460 ts_tablet_manager.cc:1403] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.033s	user 0.021s	sys 0.010s
I20260709 13:01:43.442721  9460 raft_consensus.cc:359] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:43.445040  9460 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:43.447156  9460 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:43.450227  9460 ts_tablet_manager.cc:1434] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:01:43.452212  8766 tablet_copy_service.cc:342] P 0e9b0850145a4c66bfe983ffdfa93a57: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-389e9e63581e4d1b941ba995f8cdbfc4 received from {username='slave'} at 127.0.0.1:35702
I20260709 13:01:43.452744  8766 tablet_copy_service.cc:434] P 0e9b0850145a4c66bfe983ffdfa93a57: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-389e9e63581e4d1b941ba995f8cdbfc4 on tablet 389e9e63581e4d1b941ba995f8cdbfc4 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:43.512006  9389 raft_consensus.cc:1217] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:43.740475  9389 raft_consensus.cc:1217] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:43.833066  9389 raft_consensus.cc:1217] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:43.920122  9428 raft_consensus.cc:1064] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:43.923048  9428 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:43.930886  9389 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:43.932435  8751 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:43.933229  8677 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:43.935189  9448 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:43.936265  9450 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:43.940644  9190 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:43.952657  9448 raft_consensus.cc:2955] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:43.954792  8677 raft_consensus.cc:2955] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:43.977349  8756 raft_consensus.cc:2955] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:43.981215  9389 raft_consensus.cc:2955] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:43.990834  8576 catalog_manager.cc:5697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.020208  9388 raft_consensus.cc:1217] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:44.060628  8824 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:44.067468  9389 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:44.067955  8756 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:44.069195  9190 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:44.072711  9450 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:44.082696  9340 raft_consensus.cc:2955] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.085295  9388 raft_consensus.cc:2955] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.090531  8748 raft_consensus.cc:2955] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.101372  8566 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1e6257b62c5c415aad31f5fc646ce4d4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:44.122015  8575 catalog_manager.cc:5697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.138046  8665 tablet_service.cc:1558] Processing DeleteTablet for tablet 1e6257b62c5c415aad31f5fc646ce4d4 with delete_type TABLET_DATA_TOMBSTONED (TS 0e8c59a1f0394eae8ce33b99f10efadc not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53334
I20260709 13:01:44.145577  9482 tablet_replica.cc:333] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:01:44.154316  9482 raft_consensus.cc:2243] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:44.160338  9482 raft_consensus.cc:2272] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:44.169893  9482 ts_tablet_manager.cc:1916] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:44.186939  9428 raft_consensus.cc:1064] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:44.190028  9428 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:44.210566  9388 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.212560  8748 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.212566  9482 ts_tablet_manager.cc:1929] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:44.214553  9482 log.cc:1199] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-0-root/wals/1e6257b62c5c415aad31f5fc646ce4d4
I20260709 13:01:44.219159  9340 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:44.222913  8563 catalog_manager.cc:5028] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): tablet 1e6257b62c5c415aad31f5fc646ce4d4 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:44.225826  8677 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.238158  9190 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:44.252430  9340 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:44.263868  9340 raft_consensus.cc:2955] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.273708  8748 raft_consensus.cc:2955] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.279709  8677 raft_consensus.cc:2955] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.288226  9388 raft_consensus.cc:2955] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.312675  8578 catalog_manager.cc:5697] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.332028  8824 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:44.339383  9388 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:44.344064  9428 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:44.351982  8677 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:44.358373  9428 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:44.360611  9190 raft_consensus.cc:2955] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.362320  9389 raft_consensus.cc:2955] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.386402  8566 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 20fcd6d0c3304abda83f416721e807c5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:44.400888  8677 raft_consensus.cc:2955] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.415902  8577 catalog_manager.cc:5697] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:44.419358  9237 raft_consensus.cc:1064] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:44.425246  9237 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:44.430817  9186 raft_consensus.cc:1064] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:44.433799  9186 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:44.441423  8738 tablet_service.cc:1558] Processing DeleteTablet for tablet 20fcd6d0c3304abda83f416721e807c5 with delete_type TABLET_DATA_TOMBSTONED (TS 0e9b0850145a4c66bfe983ffdfa93a57 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39018
I20260709 13:01:44.467796  9389 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.469842  8756 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.476593  8824 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.479908  9236 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:44.494748  9237 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:44.495971  9490 tablet_replica.cc:333] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:01:44.513456  9389 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.516371  8827 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.541798  9490 raft_consensus.cc:2243] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:44.543844  9490 raft_consensus.cc:2272] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:44.545840  9478 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:44.548849  9490 ts_tablet_manager.cc:1916] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:44.552316  9478 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:44.558485  8677 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:44.591950  9490 ts_tablet_manager.cc:1929] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:44.592723  9490 log.cc:1199] T 20fcd6d0c3304abda83f416721e807c5 P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-1-root/wals/20fcd6d0c3304abda83f416721e807c5
I20260709 13:01:44.600432  9186 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:44.620565  8564 catalog_manager.cc:5028] TS 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): tablet 20fcd6d0c3304abda83f416721e807c5 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:44.680783  9237 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:01:44.695058  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 71813.3us
I20260709 13:01:44.699363  9469 raft_consensus.cc:2955] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:01:44.712810  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 47450.1us
I20260709 13:01:44.723345  9236 raft_consensus.cc:2955] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.728091  8676 raft_consensus.cc:2955] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.733965  8756 raft_consensus.cc:2955] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.742079  8827 raft_consensus.cc:2955] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.757972  8575 catalog_manager.cc:5697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:44.764856  9389 raft_consensus.cc:2955] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.816320  8748 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:44.841892  8575 catalog_manager.cc:5697] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.855587  8676 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:44.857792  9478 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:44.860266  9388 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:01:44.863448  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 92770.3us
I20260709 13:01:44.868402  9186 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:44.877421  9478 raft_consensus.cc:2955] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.880187  9389 raft_consensus.cc:2955] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.886405  9388 raft_consensus.cc:2955] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.890275  8830 raft_consensus.cc:2955] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.908159  8676 raft_consensus.cc:2955] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.933220  8564 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 389e9e63581e4d1b941ba995f8cdbfc4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:44.935048  8676 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:44.945392  8748 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:44.946357  9389 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:44.947872  9502 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:44.949151  8578 catalog_manager.cc:5697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.962949  9493 raft_consensus.cc:2955] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:44.974887  8563 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ac4bab9153b047238bffb6abbeb83601 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:44.976508  9236 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:44.976902  9389 raft_consensus.cc:2955] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:01:44.986845  8563 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet ac4bab9153b047238bffb6abbeb83601 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260709 13:01:45.000644  8814 tablet_service.cc:1558] Processing DeleteTablet for tablet 389e9e63581e4d1b941ba995f8cdbfc4 with delete_type TABLET_DATA_TOMBSTONED (TS e814ed2013244fa6be42623ea5c0c7b9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60414
I20260709 13:01:45.003252  9503 tablet_replica.cc:333] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:01:45.004673  9503 raft_consensus.cc:2243] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.005595  9503 raft_consensus.cc:2272] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.005786  8748 raft_consensus.cc:2955] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:45.009652  9503 ts_tablet_manager.cc:1916] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:45.010627  8578 catalog_manager.cc:5697] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:45.029067  8814 tablet_service.cc:1558] Processing DeleteTablet for tablet ac4bab9153b047238bffb6abbeb83601 with delete_type TABLET_DATA_TOMBSTONED (TS e814ed2013244fa6be42623ea5c0c7b9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60414
I20260709 13:01:45.036619  9503 ts_tablet_manager.cc:1929] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:45.037453  9503 log.cc:1199] T 389e9e63581e4d1b941ba995f8cdbfc4 P e814ed2013244fa6be42623ea5c0c7b9: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root/wals/389e9e63581e4d1b941ba995f8cdbfc4
I20260709 13:01:45.039618  8566 catalog_manager.cc:5028] TS e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): tablet 389e9e63581e4d1b941ba995f8cdbfc4 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:45.040987  9503 tablet_replica.cc:333] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:01:45.051842  9503 raft_consensus.cc:2243] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:45.052589  9503 raft_consensus.cc:2272] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:45.056049  9503 ts_tablet_manager.cc:1916] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:45.074755  9503 ts_tablet_manager.cc:1929] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:45.075234  9503 log.cc:1199] T ac4bab9153b047238bffb6abbeb83601 P e814ed2013244fa6be42623ea5c0c7b9: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root/wals/ac4bab9153b047238bffb6abbeb83601
I20260709 13:01:45.076817  8566 catalog_manager.cc:5028] TS e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): tablet ac4bab9153b047238bffb6abbeb83601 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:46.104554  8750 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:46.114300  8677 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:46.116199  8722 consensus_peers.cc:597] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 295d39c61dd3489e98d9393a2da9c062. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:46.116390  8830 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:46.118129  9500 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:46.122529  9186 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:46.129185  9186 raft_consensus.cc:2955] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:46.131541  8676 raft_consensus.cc:2955] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:46.136490  8830 raft_consensus.cc:2955] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:46.147320  8676 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:46.157294  8577 catalog_manager.cc:5697] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:46.166337  8750 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:46.169234  9493 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:46.172104  8830 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:46.172477  8647 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1c963cd681ab4f45a5c339f9fd6e2687. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:46.175487  9493 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:01:46.198182  9502 raft_consensus.cc:2955] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:46.200896  8750 raft_consensus.cc:2955] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:46.204281  8830 raft_consensus.cc:2955] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:46.243675  8830 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:46.245769  8575 catalog_manager.cc:5697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:46.258769  8796 consensus_peers.cc:597] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 396dbfb60a3644b9b28c39c3dd36b322. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:46.260926  8748 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:46.262652  9190 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:46.265563  8677 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:46.268210  9190 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:46.281615  9499 raft_consensus.cc:2955] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:46.283356  8677 raft_consensus.cc:2955] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:46.291148  8748 raft_consensus.cc:2955] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:46.305845  8676 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:46.308844  8575 catalog_manager.cc:5697] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
W20260709 13:01:46.325910  8647 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4de6a8f976f5458d807b777f3ac94a23. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:46.327788  8830 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:46.330475  9236 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:46.332726  8748 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:46.334348  9236 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:46.340952  9502 raft_consensus.cc:2955] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:46.343312  8830 raft_consensus.cc:2955] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:46.354048  8748 raft_consensus.cc:2955] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:46.362864  8575 catalog_manager.cc:5697] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:46.646690  9521 ts_tablet_manager.cc:933] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221)
I20260709 13:01:46.647078  9522 ts_tablet_manager.cc:933] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:46.650012  9521 tablet_copy_client.cc:323] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.129:43221
I20260709 13:01:46.650131  9522 tablet_copy_client.cc:323] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.130:42321
I20260709 13:01:46.652316  8691 tablet_copy_service.cc:140] P 0e8c59a1f0394eae8ce33b99f10efadc: Received BeginTabletCopySession request for tablet 1c963cd681ab4f45a5c339f9fd6e2687 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55676)
I20260709 13:01:46.652905  8691 tablet_copy_service.cc:161] P 0e8c59a1f0394eae8ce33b99f10efadc: Beginning new tablet copy session on tablet 1c963cd681ab4f45a5c339f9fd6e2687 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55676: session id = 7aff04f53d4845a6a4fa491fd2a06145-1c963cd681ab4f45a5c339f9fd6e2687
I20260709 13:01:46.653438  8766 tablet_copy_service.cc:140] P 0e9b0850145a4c66bfe983ffdfa93a57: Received BeginTabletCopySession request for tablet 295d39c61dd3489e98d9393a2da9c062 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:35702)
I20260709 13:01:46.654021  8766 tablet_copy_service.cc:161] P 0e9b0850145a4c66bfe983ffdfa93a57: Beginning new tablet copy session on tablet 295d39c61dd3489e98d9393a2da9c062 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:35702: session id = 7aff04f53d4845a6a4fa491fd2a06145-295d39c61dd3489e98d9393a2da9c062
I20260709 13:01:46.661589  8691 tablet_copy_source_session.cc:215] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:46.663569  8766 tablet_copy_source_session.cc:215] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:46.665984  9521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c963cd681ab4f45a5c339f9fd6e2687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:46.682660  9522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 295d39c61dd3489e98d9393a2da9c062. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:46.685904  9521 tablet_copy_client.cc:806] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:46.686730  9521 tablet_copy_client.cc:670] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:46.693207  9521 tablet_copy_client.cc:538] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:46.721144  9522 tablet_copy_client.cc:806] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:46.722291  9522 tablet_copy_client.cc:670] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:46.725538  9521 tablet_bootstrap.cc:492] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:46.730734  9522 tablet_copy_client.cc:538] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:46.758297  9522 tablet_bootstrap.cc:492] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:46.771945  9521 tablet_bootstrap.cc:492] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:46.773658  9521 tablet_bootstrap.cc:492] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:46.774449  9521 ts_tablet_manager.cc:1403] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.049s	user 0.023s	sys 0.005s
I20260709 13:01:46.777855  9521 raft_consensus.cc:359] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:46.778790  9521 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:46.779721  9521 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:46.791757  9521 ts_tablet_manager.cc:1434] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.003s
I20260709 13:01:46.795257  8691 tablet_copy_service.cc:342] P 0e8c59a1f0394eae8ce33b99f10efadc: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-1c963cd681ab4f45a5c339f9fd6e2687 received from {username='slave'} at 127.0.0.1:55676
I20260709 13:01:46.795820  8691 tablet_copy_service.cc:434] P 0e8c59a1f0394eae8ce33b99f10efadc: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-1c963cd681ab4f45a5c339f9fd6e2687 on tablet 1c963cd681ab4f45a5c339f9fd6e2687 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:46.822109  9528 ts_tablet_manager.cc:933] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221)
I20260709 13:01:46.829813  9528 tablet_copy_client.cc:323] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.129:43221
I20260709 13:01:46.831782  8691 tablet_copy_service.cc:140] P 0e8c59a1f0394eae8ce33b99f10efadc: Received BeginTabletCopySession request for tablet 4de6a8f976f5458d807b777f3ac94a23 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55676)
I20260709 13:01:46.832489  8691 tablet_copy_service.cc:161] P 0e8c59a1f0394eae8ce33b99f10efadc: Beginning new tablet copy session on tablet 4de6a8f976f5458d807b777f3ac94a23 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55676: session id = 7aff04f53d4845a6a4fa491fd2a06145-4de6a8f976f5458d807b777f3ac94a23
I20260709 13:01:46.839478  9522 tablet_bootstrap.cc:492] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:46.840610  9522 tablet_bootstrap.cc:492] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:46.841409  9522 ts_tablet_manager.cc:1403] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.083s	user 0.034s	sys 0.019s
I20260709 13:01:46.844941  9522 raft_consensus.cc:359] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:46.845897  9522 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:46.846654  9522 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:46.847847  8691 tablet_copy_source_session.cc:215] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:46.850121  9522 ts_tablet_manager.cc:1434] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.001s
I20260709 13:01:46.852341  9528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4de6a8f976f5458d807b777f3ac94a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:46.853379  8766 tablet_copy_service.cc:342] P 0e9b0850145a4c66bfe983ffdfa93a57: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-295d39c61dd3489e98d9393a2da9c062 received from {username='slave'} at 127.0.0.1:35702
I20260709 13:01:46.853902  8766 tablet_copy_service.cc:434] P 0e9b0850145a4c66bfe983ffdfa93a57: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-295d39c61dd3489e98d9393a2da9c062 on tablet 295d39c61dd3489e98d9393a2da9c062 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:46.866321  9521 ts_tablet_manager.cc:933] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:46.875649  9528 tablet_copy_client.cc:806] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:46.876487  9528 tablet_copy_client.cc:670] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:46.878021  9521 tablet_copy_client.cc:323] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:01:46.880561  8840 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet 396dbfb60a3644b9b28c39c3dd36b322 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
I20260709 13:01:46.881275  8840 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet 396dbfb60a3644b9b28c39c3dd36b322 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-396dbfb60a3644b9b28c39c3dd36b322
I20260709 13:01:46.884768  9528 tablet_copy_client.cc:538] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:46.893066  8840 tablet_copy_source_session.cc:215] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:46.895664  9528 tablet_bootstrap.cc:492] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:46.897306  9521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 396dbfb60a3644b9b28c39c3dd36b322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:46.917960  9521 tablet_copy_client.cc:806] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:46.918594  9521 tablet_copy_client.cc:670] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:46.923728  9521 tablet_copy_client.cc:538] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:46.924559  9528 tablet_bootstrap.cc:492] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:46.925578  9528 tablet_bootstrap.cc:492] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:46.926335  9528 ts_tablet_manager.cc:1403] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.031s	user 0.019s	sys 0.008s
I20260709 13:01:46.941795  9528 raft_consensus.cc:359] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:46.942849  9528 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:46.943534  9528 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:46.950059  9528 ts_tablet_manager.cc:1434] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.023s	user 0.014s	sys 0.005s
I20260709 13:01:46.952025  8691 tablet_copy_service.cc:342] P 0e8c59a1f0394eae8ce33b99f10efadc: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-4de6a8f976f5458d807b777f3ac94a23 received from {username='slave'} at 127.0.0.1:55676
I20260709 13:01:46.952544  8691 tablet_copy_service.cc:434] P 0e8c59a1f0394eae8ce33b99f10efadc: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-4de6a8f976f5458d807b777f3ac94a23 on tablet 4de6a8f976f5458d807b777f3ac94a23 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:46.953104  9521 tablet_bootstrap.cc:492] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:46.979526  9521 tablet_bootstrap.cc:492] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:46.980523  9521 tablet_bootstrap.cc:492] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:46.981237  9521 ts_tablet_manager.cc:1403] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.008s
I20260709 13:01:46.983891  9521 raft_consensus.cc:359] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:46.984651  9521 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:46.985352  9521 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:46.987841  9521 ts_tablet_manager.cc:1434] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:01:46.991191  8840 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-396dbfb60a3644b9b28c39c3dd36b322 received from {username='slave'} at 127.0.0.1:55168
I20260709 13:01:46.991725  8840 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-396dbfb60a3644b9b28c39c3dd36b322 on tablet 396dbfb60a3644b9b28c39c3dd36b322 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:47.147902  9388 raft_consensus.cc:1217] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:47.238574  9388 raft_consensus.cc:1217] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:47.288722  9388 raft_consensus.cc:1217] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:47.362401  9388 raft_consensus.cc:1217] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:47.554407  9500 raft_consensus.cc:1064] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:47.556819  9500 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:47.567210  9388 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.568868  9186 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:47.573125  8676 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.573125  8827 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.577961  9186 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:47.580360  9186 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:47.602041  9500 raft_consensus.cc:2955] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.625342  8830 raft_consensus.cc:2955] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.625342  9388 raft_consensus.cc:2955] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.628767  8676 raft_consensus.cc:2955] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.644647  8578 catalog_manager.cc:5697] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.657932  9502 raft_consensus.cc:1064] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:47.660655  9502 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:47.678299  9388 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.679175  8756 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.680598  8829 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.680585  9502 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:47.682412  9533 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:47.683554  9236 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:47.709070  9502 raft_consensus.cc:2955] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.714329  8830 raft_consensus.cc:2955] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.731328  9389 raft_consensus.cc:2955] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.732162  8750 raft_consensus.cc:2955] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.758921  8577 catalog_manager.cc:5697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.729740  8756 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:47.777284  9190 raft_consensus.cc:1064] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:47.777287  9389 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:47.782588  8830 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:47.779991  9190 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:47.784960  9542 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:47.789619  9500 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:47.801221  9388 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.803099  9186 raft_consensus.cc:2955] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.803190  9448 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:01:47.807763  8677 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.809614  8756 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.811513  9448 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:47.827525  8564 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 295d39c61dd3489e98d9393a2da9c062 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:47.828658  8676 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:47.831013  9190 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:47.835631  9387 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:47.837809  9236 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:47.837195  8575 catalog_manager.cc:5697] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.843580  8830 raft_consensus.cc:2955] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.853471  8665 tablet_service.cc:1558] Processing DeleteTablet for tablet 295d39c61dd3489e98d9393a2da9c062 with delete_type TABLET_DATA_TOMBSTONED (TS 0e8c59a1f0394eae8ce33b99f10efadc not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53334
I20260709 13:01:47.829890  9389 raft_consensus.cc:2955] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.858378  9533 raft_consensus.cc:2955] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.870505  9387 raft_consensus.cc:2955] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.884256  9190 raft_consensus.cc:2955] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.897390  9502 raft_consensus.cc:1064] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:47.896399  8674 raft_consensus.cc:2955] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.900410  9502 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:47.920183  9389 raft_consensus.cc:2955] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.931737  8563 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1c963cd681ab4f45a5c339f9fd6e2687 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:47.937973  8576 catalog_manager.cc:5697] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.943210  8827 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:47.981947  9236 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:47.984383  9555 tablet_replica.cc:333] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:01:47.985671  9555 raft_consensus.cc:2243] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:47.987707  9389 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.989857  9534 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:47.992136  8747 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:47.993521  9555 raft_consensus.cc:2272] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:47.994380  9534 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:47.996425  8575 catalog_manager.cc:5697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:47.997112  8827 raft_consensus.cc:2955] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:48.004614  9555 ts_tablet_manager.cc:1916] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:48.010039  8830 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:48.021317  8756 raft_consensus.cc:2955] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:48.029278  9534 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:48.030378  8738 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c963cd681ab4f45a5c339f9fd6e2687 with delete_type TABLET_DATA_TOMBSTONED (TS 0e9b0850145a4c66bfe983ffdfa93a57 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39018
I20260709 13:01:48.032878  9560 tablet_replica.cc:333] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:01:48.036547  9560 raft_consensus.cc:2243] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:48.037703  9560 raft_consensus.cc:2272] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:48.047711  9560 ts_tablet_manager.cc:1916] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:48.053428  9534 raft_consensus.cc:2955] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:48.058139  8827 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:48.063911  9555 ts_tablet_manager.cc:1929] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:48.064599  9555 log.cc:1199] T 295d39c61dd3489e98d9393a2da9c062 P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-0-root/wals/295d39c61dd3489e98d9393a2da9c062
I20260709 13:01:48.066916  8563 catalog_manager.cc:5028] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): tablet 295d39c61dd3489e98d9393a2da9c062 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:48.069870  9560 ts_tablet_manager.cc:1929] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:48.070423  9560 log.cc:1199] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-1-root/wals/1c963cd681ab4f45a5c339f9fd6e2687
I20260709 13:01:48.072383  8564 catalog_manager.cc:5028] TS 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): tablet 1c963cd681ab4f45a5c339f9fd6e2687 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:48.086400  9389 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:48.090541  9499 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:01:48.091459  9387 raft_consensus.cc:2955] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:48.097749  8747 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:48.106782  9190 raft_consensus.cc:2955] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:48.110481  9448 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:48.115804  8830 raft_consensus.cc:2955] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:48.114173  9389 raft_consensus.cc:2955] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:48.121897  8756 raft_consensus.cc:2955] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:48.136550  8575 catalog_manager.cc:5697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:48.139083  8750 raft_consensus.cc:2955] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:48.159905  8575 catalog_manager.cc:5697] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:01:48.168686  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 53616.2us
I20260709 13:01:48.173694  8566 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 396dbfb60a3644b9b28c39c3dd36b322 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:48.186168  8674 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:48.200709  9389 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:48.202255  9236 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:48.214864  9236 raft_consensus.cc:2955] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:48.224283  8665 tablet_service.cc:1558] Processing DeleteTablet for tablet 396dbfb60a3644b9b28c39c3dd36b322 with delete_type TABLET_DATA_TOMBSTONED (TS 0e8c59a1f0394eae8ce33b99f10efadc not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53334
I20260709 13:01:48.225462  9555 tablet_replica.cc:333] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:01:48.232501  8756 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:48.238351  9502 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:01:48.247527  9389 raft_consensus.cc:2955] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:48.252458  9555 raft_consensus.cc:2243] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:48.253351  9555 raft_consensus.cc:2272] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:48.254699  8563 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4de6a8f976f5458d807b777f3ac94a23 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:48.262205  8575 catalog_manager.cc:5697] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:48.255690  8756 raft_consensus.cc:2955] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:48.262092  9555 ts_tablet_manager.cc:1916] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:48.286309  8814 tablet_service.cc:1558] Processing DeleteTablet for tablet 4de6a8f976f5458d807b777f3ac94a23 with delete_type TABLET_DATA_TOMBSTONED (TS e814ed2013244fa6be42623ea5c0c7b9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60414
I20260709 13:01:48.303172  9563 tablet_replica.cc:333] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:01:48.304684  9563 raft_consensus.cc:2243] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:48.305794  9555 ts_tablet_manager.cc:1929] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:48.305974  9563 raft_consensus.cc:2272] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:48.306288  9555 log.cc:1199] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-0-root/wals/396dbfb60a3644b9b28c39c3dd36b322
I20260709 13:01:48.308226  8563 catalog_manager.cc:5028] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): tablet 396dbfb60a3644b9b28c39c3dd36b322 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:48.309214  9563 ts_tablet_manager.cc:1916] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:48.327494  9563 ts_tablet_manager.cc:1929] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:48.327977  9563 log.cc:1199] T 4de6a8f976f5458d807b777f3ac94a23 P e814ed2013244fa6be42623ea5c0c7b9: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root/wals/4de6a8f976f5458d807b777f3ac94a23
I20260709 13:01:48.329615  8566 catalog_manager.cc:5028] TS e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): tablet 4de6a8f976f5458d807b777f3ac94a23 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:49.390663  8827 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
W20260709 13:01:49.398586  8796 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8acd9840694e4f62803bfe9810425202. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:49.400764  8756 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:49.403590  9448 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:49.405880  8676 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:49.408097  9448 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:49.419433  9448 raft_consensus.cc:2955] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:49.421284  8676 raft_consensus.cc:2955] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:49.425154  8750 raft_consensus.cc:2955] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:49.442013  8827 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:49.454989  8575 catalog_manager.cc:5697] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 2 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
W20260709 13:01:49.478916  8796 consensus_peers.cc:597] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8611228be8dd4f08b2b609e202974a6a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:49.486023  8676 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:49.488376  9448 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:49.493278  8747 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:49.495165  9567 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:49.502068  9448 raft_consensus.cc:2955] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:49.519201  8677 raft_consensus.cc:2955] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:49.528861  8756 raft_consensus.cc:2955] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:49.540642  8747 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:49.546298  8576 catalog_manager.cc:5697] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
W20260709 13:01:49.556558  8722 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 987153bedd0c4f86b726db1edfd1eec6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:49.559173  8827 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:49.559906  8677 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:49.561836  9564 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:49.567989  9537 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:49.584162  9536 raft_consensus.cc:2955] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:49.586760  8676 raft_consensus.cc:2955] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:49.586696  8827 raft_consensus.cc:2955] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:49.604766  8575 catalog_manager.cc:5697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:49.606998  8677 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
W20260709 13:01:49.622588  8647 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3169f7f9f5a7440398ae5ebe01fc3c3f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:49.628545  8756 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:49.628545  8827 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:49.630344  9502 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:49.631280  9565 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:49.646726  9533 raft_consensus.cc:2955] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:49.648317  8756 raft_consensus.cc:2955] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:49.649443  8829 raft_consensus.cc:2955] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:49.668752  8576 catalog_manager.cc:5697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:49.850070  9586 ts_tablet_manager.cc:933] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:49.852315  9586 tablet_copy_client.cc:323] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:01:49.854367  8840 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet 8acd9840694e4f62803bfe9810425202 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
I20260709 13:01:49.855054  8840 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet 8acd9840694e4f62803bfe9810425202 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-8acd9840694e4f62803bfe9810425202
I20260709 13:01:49.862587  8840 tablet_copy_source_session.cc:215] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:49.865798  9586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8acd9840694e4f62803bfe9810425202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.883605  9586 tablet_copy_client.cc:806] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:49.884392  9586 tablet_copy_client.cc:670] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:49.893752  9586 tablet_copy_client.cc:538] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:49.894870  9588 ts_tablet_manager.cc:933] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:49.910756  9586 tablet_bootstrap.cc:492] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:49.914723  9588 tablet_copy_client.cc:323] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:01:49.917093  8840 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet 8611228be8dd4f08b2b609e202974a6a from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
I20260709 13:01:49.917775  8840 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet 8611228be8dd4f08b2b609e202974a6a from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-8611228be8dd4f08b2b609e202974a6a
I20260709 13:01:49.935410  8840 tablet_copy_source_session.cc:215] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:49.941704  9588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8611228be8dd4f08b2b609e202974a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:49.958351  9586 tablet_bootstrap.cc:492] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:49.959580  9586 tablet_bootstrap.cc:492] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:49.960410  9586 ts_tablet_manager.cc:1403] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.050s	user 0.028s	sys 0.018s
I20260709 13:01:49.963868  9586 raft_consensus.cc:359] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:49.964797  9586 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:49.965476  9586 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:49.968250  9588 tablet_copy_client.cc:806] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:49.969281  9588 tablet_copy_client.cc:670] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:49.972896  9586 ts_tablet_manager.cc:1434] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.005s
I20260709 13:01:49.977339  9588 tablet_copy_client.cc:538] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:49.982903  8840 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-8acd9840694e4f62803bfe9810425202 received from {username='slave'} at 127.0.0.1:55168
I20260709 13:01:49.983582  8840 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-8acd9840694e4f62803bfe9810425202 on tablet 8acd9840694e4f62803bfe9810425202 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:49.992682  9588 tablet_bootstrap.cc:492] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:50.038071  9588 tablet_bootstrap.cc:492] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:50.039228  9588 tablet_bootstrap.cc:492] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:50.040022  9588 ts_tablet_manager.cc:1403] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.048s	user 0.026s	sys 0.018s
I20260709 13:01:50.043272  9588 raft_consensus.cc:359] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:50.044159  9588 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:50.044888  9588 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:50.047228  9588 ts_tablet_manager.cc:1434] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:01:50.053478  8840 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-8611228be8dd4f08b2b609e202974a6a received from {username='slave'} at 127.0.0.1:55168
I20260709 13:01:50.054016  8840 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-8611228be8dd4f08b2b609e202974a6a on tablet 8611228be8dd4f08b2b609e202974a6a with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:50.084226  9588 ts_tablet_manager.cc:933] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221)
I20260709 13:01:50.086058  9588 tablet_copy_client.cc:323] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.129:43221
I20260709 13:01:50.087882  8691 tablet_copy_service.cc:140] P 0e8c59a1f0394eae8ce33b99f10efadc: Received BeginTabletCopySession request for tablet 3169f7f9f5a7440398ae5ebe01fc3c3f from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55676)
I20260709 13:01:50.088418  8691 tablet_copy_service.cc:161] P 0e8c59a1f0394eae8ce33b99f10efadc: Beginning new tablet copy session on tablet 3169f7f9f5a7440398ae5ebe01fc3c3f from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55676: session id = 7aff04f53d4845a6a4fa491fd2a06145-3169f7f9f5a7440398ae5ebe01fc3c3f
I20260709 13:01:50.096891  8691 tablet_copy_source_session.cc:215] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:50.100986  9588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3169f7f9f5a7440398ae5ebe01fc3c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:50.117631  9588 tablet_copy_client.cc:806] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:50.118319  9588 tablet_copy_client.cc:670] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:50.122566  9588 tablet_copy_client.cc:538] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:50.134143  9588 tablet_bootstrap.cc:492] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:50.155359  9586 ts_tablet_manager.cc:933] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:50.159549  9586 tablet_copy_client.cc:323] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.130:42321
I20260709 13:01:50.161624  8766 tablet_copy_service.cc:140] P 0e9b0850145a4c66bfe983ffdfa93a57: Received BeginTabletCopySession request for tablet 987153bedd0c4f86b726db1edfd1eec6 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:35702)
I20260709 13:01:50.162241  8766 tablet_copy_service.cc:161] P 0e9b0850145a4c66bfe983ffdfa93a57: Beginning new tablet copy session on tablet 987153bedd0c4f86b726db1edfd1eec6 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:35702: session id = 7aff04f53d4845a6a4fa491fd2a06145-987153bedd0c4f86b726db1edfd1eec6
I20260709 13:01:50.170033  9588 tablet_bootstrap.cc:492] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:50.171591  9588 tablet_bootstrap.cc:492] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:50.172329  9588 ts_tablet_manager.cc:1403] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.038s	user 0.031s	sys 0.005s
I20260709 13:01:50.177495  9588 raft_consensus.cc:359] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:50.178406  9588 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:50.179133  9588 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:50.180264  8766 tablet_copy_source_session.cc:215] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:50.184073  9586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 987153bedd0c4f86b726db1edfd1eec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:50.184783  9588 ts_tablet_manager.cc:1434] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:01:50.200074  8691 tablet_copy_service.cc:342] P 0e8c59a1f0394eae8ce33b99f10efadc: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-3169f7f9f5a7440398ae5ebe01fc3c3f received from {username='slave'} at 127.0.0.1:55676
I20260709 13:01:50.200613  8691 tablet_copy_service.cc:434] P 0e8c59a1f0394eae8ce33b99f10efadc: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-3169f7f9f5a7440398ae5ebe01fc3c3f on tablet 3169f7f9f5a7440398ae5ebe01fc3c3f with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:50.214363  9586 tablet_copy_client.cc:806] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:50.215101  9586 tablet_copy_client.cc:670] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:50.220539  9586 tablet_copy_client.cc:538] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:50.232069  9586 tablet_bootstrap.cc:492] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:50.267211  9586 tablet_bootstrap.cc:492] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:50.268425  9586 tablet_bootstrap.cc:492] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:50.269253  9586 ts_tablet_manager.cc:1403] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.013s
I20260709 13:01:50.283427  9586 raft_consensus.cc:359] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:50.285579  9586 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:50.286525  9586 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:50.290555  9586 ts_tablet_manager.cc:1434] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.005s
I20260709 13:01:50.295691  8766 tablet_copy_service.cc:342] P 0e9b0850145a4c66bfe983ffdfa93a57: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-987153bedd0c4f86b726db1edfd1eec6 received from {username='slave'} at 127.0.0.1:35702
I20260709 13:01:50.296252  8766 tablet_copy_service.cc:434] P 0e9b0850145a4c66bfe983ffdfa93a57: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-987153bedd0c4f86b726db1edfd1eec6 on tablet 987153bedd0c4f86b726db1edfd1eec6 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:50.300256  9387 raft_consensus.cc:1217] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:01:50.449891  9387 raft_consensus.cc:1217] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:50.609467  9387 raft_consensus.cc:1217] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:50.694279  9448 raft_consensus.cc:1064] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:50.696424  9448 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:50.703573  8756 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:50.705855  9387 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:50.706190  9567 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:50.708202  9577 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:50.725164  8674 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:50.747247  9448 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:50.759181  9567 raft_consensus.cc:2955] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:50.776013  9389 raft_consensus.cc:2955] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:50.762310  8756 raft_consensus.cc:2955] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:50.801687  8576 catalog_manager.cc:5697] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:50.833495  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 50699us
I20260709 13:01:50.841377  8677 raft_consensus.cc:2955] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:50.850275  9389 raft_consensus.cc:1217] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:50.930296  8830 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:50.937925  9389 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:50.938642  8677 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:50.939901  9567 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:50.940989  9577 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:50.950253  9577 raft_consensus.cc:2955] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:50.954838  9389 raft_consensus.cc:2955] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:50.960523  8677 raft_consensus.cc:2955] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:50.973767  8566 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8acd9840694e4f62803bfe9810425202 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:50.982417  8577 catalog_manager.cc:5697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New cstate: current_term: 2 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:50.999624  8738 tablet_service.cc:1558] Processing DeleteTablet for tablet 8acd9840694e4f62803bfe9810425202 with delete_type TABLET_DATA_TOMBSTONED (TS 0e9b0850145a4c66bfe983ffdfa93a57 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39018
I20260709 13:01:51.002135  9448 raft_consensus.cc:1064] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:51.007323  9448 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:51.029337  9610 tablet_replica.cc:333] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:01:51.031512  8750 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:51.032195  8674 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:51.032951  9389 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:51.030969  9610 raft_consensus.cc:2243] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:51.034080  9577 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:51.035084  9610 raft_consensus.cc:2272] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:51.035594  9448 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:51.040177  9577 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:51.055613  9610 ts_tablet_manager.cc:1916] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:51.072805  9567 raft_consensus.cc:2955] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.077702  8750 raft_consensus.cc:2955] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.091333  9610 ts_tablet_manager.cc:1929] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:01:51.092031  9610 log.cc:1199] T 8acd9840694e4f62803bfe9810425202 P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-1-root/wals/8acd9840694e4f62803bfe9810425202
I20260709 13:01:51.077446  8674 raft_consensus.cc:2955] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.096541  9533 raft_consensus.cc:1064] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:51.095763  9389 raft_consensus.cc:2955] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.105240  8578 catalog_manager.cc:5697] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:51.109380  9533 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:51.120039  8564 catalog_manager.cc:5028] TS 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): tablet 8acd9840694e4f62803bfe9810425202 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:51.145823  9389 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:51.145915  8750 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:51.149253  9585 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:51.151330  8827 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:51.144116  8830 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:51.155202  9502 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:51.159070  9502 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:51.164856  9387 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:51.166742  9567 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:51.171322  8756 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:51.190424  9578 raft_consensus.cc:2955] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.198513  9387 raft_consensus.cc:2955] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.200325  9567 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:51.216516  8756 raft_consensus.cc:2955] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.225958  8575 catalog_manager.cc:5697] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.246114  8665 tablet_service.cc:1558] Processing DeleteTablet for tablet 8611228be8dd4f08b2b609e202974a6a with delete_type TABLET_DATA_TOMBSTONED (TS 0e8c59a1f0394eae8ce33b99f10efadc not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53334
I20260709 13:01:51.249257  9620 tablet_replica.cc:333] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:01:51.251652  9620 raft_consensus.cc:2243] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:51.252506  9620 raft_consensus.cc:2272] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:51.340951  9620 ts_tablet_manager.cc:1916] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:51.372363  8566 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8611228be8dd4f08b2b609e202974a6a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:51.373317  9616 raft_consensus.cc:2955] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.379256  8829 raft_consensus.cc:2955] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.399457  9389 raft_consensus.cc:2955] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.424729  8575 catalog_manager.cc:5697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:51.444479  8674 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:51.445073  9620 ts_tablet_manager.cc:1929] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:51.450070  9601 raft_consensus.cc:1064] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:51.447768  8750 raft_consensus.cc:2955] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.455593  9387 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:51.445699  9620 log.cc:1199] T 8611228be8dd4f08b2b609e202974a6a P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-0-root/wals/8611228be8dd4f08b2b609e202974a6a
I20260709 13:01:51.457109  9601 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:51.461369  9585 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:51.462862  8830 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:51.468209  9533 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:51.474238  9502 raft_consensus.cc:2955] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.477461  9387 raft_consensus.cc:2955] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.479074  8563 catalog_manager.cc:5028] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): tablet 8611228be8dd4f08b2b609e202974a6a (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:51.488090  8829 raft_consensus.cc:2955] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.501748  9388 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:51.504189  9602 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:51.512147  8563 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3169f7f9f5a7440398ae5ebe01fc3c3f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:51.519354  8674 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:51.522850  8829 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:51.522349  9604 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:01:51.532227  9602 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:51.527212  8575 catalog_manager.cc:5697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.554672  9604 raft_consensus.cc:2955] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.562326  8674 raft_consensus.cc:2955] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:01:51.570279  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 45539us
I20260709 13:01:51.582974  8738 tablet_service.cc:1558] Processing DeleteTablet for tablet 3169f7f9f5a7440398ae5ebe01fc3c3f with delete_type TABLET_DATA_TOMBSTONED (TS 0e9b0850145a4c66bfe983ffdfa93a57 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39018
I20260709 13:01:51.584283  9610 tablet_replica.cc:333] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:01:51.585510  9610 raft_consensus.cc:2243] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:51.586249  9610 raft_consensus.cc:2272] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:51.585111  9387 raft_consensus.cc:2955] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.591202  9610 ts_tablet_manager.cc:1916] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:51.595795  8578 catalog_manager.cc:5697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:51.619083  8829 raft_consensus.cc:2955] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.628612  9610 ts_tablet_manager.cc:1929] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:51.629173  9610 log.cc:1199] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-1-root/wals/3169f7f9f5a7440398ae5ebe01fc3c3f
I20260709 13:01:51.628793  8750 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:51.634900  8564 catalog_manager.cc:5028] TS 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): tablet 3169f7f9f5a7440398ae5ebe01fc3c3f (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:51.649411  9387 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:51.651953  8681 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:51.651716  9604 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:51.655067  9625 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:51.662865  9625 raft_consensus.cc:2955] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.673139  8681 raft_consensus.cc:2955] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.674281  9387 raft_consensus.cc:2955] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:51.678205  8564 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 987153bedd0c4f86b726db1edfd1eec6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:51.689985  8575 catalog_manager.cc:5697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:51.703440  8814 tablet_service.cc:1558] Processing DeleteTablet for tablet 987153bedd0c4f86b726db1edfd1eec6 with delete_type TABLET_DATA_TOMBSTONED (TS e814ed2013244fa6be42623ea5c0c7b9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60414
I20260709 13:01:51.706549  9628 tablet_replica.cc:333] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:01:51.708169  9628 raft_consensus.cc:2243] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:51.709060  9628 raft_consensus.cc:2272] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:51.713493  9628 ts_tablet_manager.cc:1916] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:51.735801  9628 ts_tablet_manager.cc:1929] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:51.736203  9628 log.cc:1199] T 987153bedd0c4f86b726db1edfd1eec6 P e814ed2013244fa6be42623ea5c0c7b9: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root/wals/987153bedd0c4f86b726db1edfd1eec6
I20260709 13:01:51.738056  8566 catalog_manager.cc:5028] TS e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): tablet 987153bedd0c4f86b726db1edfd1eec6 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
W20260709 13:01:52.803274  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 47792us
I20260709 13:01:52.934634  8750 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
W20260709 13:01:52.956113  8722 consensus_peers.cc:597] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f9ddea0a887446f910d38f8302e8172. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:52.957320  8824 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:52.958962  8674 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:52.960616  9625 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:52.963219  9629 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:52.977133  9604 raft_consensus.cc:2955] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:52.979202  8674 raft_consensus.cc:2955] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:52.984062  8829 raft_consensus.cc:2955] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:53.003477  8747 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:53.003875  8576 catalog_manager.cc:5697] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:53.026252  8722 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c01533e2a5204b1ebccac1133e495542. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:53.026885  8824 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:53.028330  9629 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:53.029546  8677 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:53.031064  9629 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:53.051457  9604 raft_consensus.cc:2955] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:53.057726  8677 raft_consensus.cc:2955] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:53.064299  8829 raft_consensus.cc:2955] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:53.084786  8824 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:53.093186  8578 catalog_manager.cc:5697] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:53.100934  8796 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ee1e688ccd74392befec90535c1efa4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:53.103102  8677 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:53.105158  9635 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:53.105953  8751 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:53.108395  9635 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:53.119988  9635 raft_consensus.cc:2955] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:53.121862  8751 raft_consensus.cc:2955] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:53.125434  8677 raft_consensus.cc:2955] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:53.146744  8576 catalog_manager.cc:5697] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:53.149281  8747 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
W20260709 13:01:53.161998  8722 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 142891c077a34ba289adf90a99aee612. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:53.164366  8824 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:53.166211  8676 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:53.168547  9604 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:53.169703  9625 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:53.190923  9604 raft_consensus.cc:2955] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:53.195550  8829 raft_consensus.cc:2955] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:53.193516  8674 raft_consensus.cc:2955] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:53.258271  8576 catalog_manager.cc:5697] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:53.409770  9652 ts_tablet_manager.cc:933] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:53.412061  9652 tablet_copy_client.cc:323] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.130:42321
I20260709 13:01:53.414354  8766 tablet_copy_service.cc:140] P 0e9b0850145a4c66bfe983ffdfa93a57: Received BeginTabletCopySession request for tablet c01533e2a5204b1ebccac1133e495542 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:35702)
I20260709 13:01:53.415084  8766 tablet_copy_service.cc:161] P 0e9b0850145a4c66bfe983ffdfa93a57: Beginning new tablet copy session on tablet c01533e2a5204b1ebccac1133e495542 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:35702: session id = 7aff04f53d4845a6a4fa491fd2a06145-c01533e2a5204b1ebccac1133e495542
I20260709 13:01:53.426894  8766 tablet_copy_source_session.cc:215] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:53.431994  9652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c01533e2a5204b1ebccac1133e495542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:53.457626  9652 tablet_copy_client.cc:806] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:53.458433  9652 tablet_copy_client.cc:670] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:53.463866  9652 tablet_copy_client.cc:538] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:53.473894  9652 tablet_bootstrap.cc:492] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:53.496971  9652 tablet_bootstrap.cc:492] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:53.498158  9652 tablet_bootstrap.cc:492] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:53.498982  9652 ts_tablet_manager.cc:1403] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.025s	user 0.027s	sys 0.000s
I20260709 13:01:53.503078  9652 raft_consensus.cc:359] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:53.524073  9652 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:53.525002  9652 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:53.530277  9655 ts_tablet_manager.cc:933] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:53.536906  9655 tablet_copy_client.cc:323] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:01:53.539390  9652 ts_tablet_manager.cc:1434] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.004s
I20260709 13:01:53.540915  8840 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet 5ee1e688ccd74392befec90535c1efa4 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
I20260709 13:01:53.541590  8840 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet 5ee1e688ccd74392befec90535c1efa4 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-5ee1e688ccd74392befec90535c1efa4
I20260709 13:01:53.543565  8766 tablet_copy_service.cc:342] P 0e9b0850145a4c66bfe983ffdfa93a57: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-c01533e2a5204b1ebccac1133e495542 received from {username='slave'} at 127.0.0.1:35702
I20260709 13:01:53.544142  8766 tablet_copy_service.cc:434] P 0e9b0850145a4c66bfe983ffdfa93a57: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-c01533e2a5204b1ebccac1133e495542 on tablet c01533e2a5204b1ebccac1133e495542 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:53.551481  8840 tablet_copy_source_session.cc:215] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:53.556442  9655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ee1e688ccd74392befec90535c1efa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:53.566304  9652 ts_tablet_manager.cc:933] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:53.577361  9655 tablet_copy_client.cc:806] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:53.578143  9655 tablet_copy_client.cc:670] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:53.584839  9655 tablet_copy_client.cc:538] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:53.589059  9658 ts_tablet_manager.cc:933] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:01:53.596840  9655 tablet_bootstrap.cc:492] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:53.599591  9652 tablet_copy_client.cc:323] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.130:42321
I20260709 13:01:53.602097  8766 tablet_copy_service.cc:140] P 0e9b0850145a4c66bfe983ffdfa93a57: Received BeginTabletCopySession request for tablet 1f9ddea0a887446f910d38f8302e8172 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:35702)
I20260709 13:01:53.602933  8766 tablet_copy_service.cc:161] P 0e9b0850145a4c66bfe983ffdfa93a57: Beginning new tablet copy session on tablet 1f9ddea0a887446f910d38f8302e8172 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:35702: session id = 7aff04f53d4845a6a4fa491fd2a06145-1f9ddea0a887446f910d38f8302e8172
I20260709 13:01:53.609581  9658 tablet_copy_client.cc:323] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.130:42321
I20260709 13:01:53.613395  8765 tablet_copy_service.cc:140] P 0e9b0850145a4c66bfe983ffdfa93a57: Received BeginTabletCopySession request for tablet 142891c077a34ba289adf90a99aee612 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:35702)
I20260709 13:01:53.614037  8765 tablet_copy_service.cc:161] P 0e9b0850145a4c66bfe983ffdfa93a57: Beginning new tablet copy session on tablet 142891c077a34ba289adf90a99aee612 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:35702: session id = 7aff04f53d4845a6a4fa491fd2a06145-142891c077a34ba289adf90a99aee612
I20260709 13:01:53.619508  8766 tablet_copy_source_session.cc:215] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:53.623281  9652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f9ddea0a887446f910d38f8302e8172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:53.623421  8765 tablet_copy_source_session.cc:215] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:53.631108  9658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142891c077a34ba289adf90a99aee612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:53.639243  9655 tablet_bootstrap.cc:492] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:53.640515  9655 tablet_bootstrap.cc:492] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:53.643034  9655 ts_tablet_manager.cc:1403] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.046s	user 0.033s	sys 0.001s
I20260709 13:01:53.653100  9655 raft_consensus.cc:359] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:53.654062  9655 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:53.654848  9655 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:53.658200  9655 ts_tablet_manager.cc:1434] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:01:53.660678  8840 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-5ee1e688ccd74392befec90535c1efa4 received from {username='slave'} at 127.0.0.1:55168
I20260709 13:01:53.661168  8840 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-5ee1e688ccd74392befec90535c1efa4 on tablet 5ee1e688ccd74392befec90535c1efa4 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:53.691679  9652 tablet_copy_client.cc:806] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:53.692438  9652 tablet_copy_client.cc:670] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:53.702960  9658 tablet_copy_client.cc:806] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:53.704797  9658 tablet_copy_client.cc:670] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:53.705750  9652 tablet_copy_client.cc:538] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:53.715792  9658 tablet_copy_client.cc:538] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:53.727033  9652 tablet_bootstrap.cc:492] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:53.735368  9658 tablet_bootstrap.cc:492] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:53.769132  9658 tablet_bootstrap.cc:492] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:53.770238  9658 tablet_bootstrap.cc:492] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:53.771042  9658 ts_tablet_manager.cc:1403] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.012s
I20260709 13:01:53.771622  9652 tablet_bootstrap.cc:492] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:53.772656  9652 tablet_bootstrap.cc:492] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:53.773404  9652 ts_tablet_manager.cc:1403] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.047s	user 0.026s	sys 0.007s
I20260709 13:01:53.774377  9658 raft_consensus.cc:359] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:53.775481  9658 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:53.779618  9652 raft_consensus.cc:359] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:53.780697  9652 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:53.781806  9652 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:53.785192  9652 ts_tablet_manager.cc:1434] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 13:01:53.787771  8766 tablet_copy_service.cc:342] P 0e9b0850145a4c66bfe983ffdfa93a57: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-1f9ddea0a887446f910d38f8302e8172 received from {username='slave'} at 127.0.0.1:35702
I20260709 13:01:53.788235  8766 tablet_copy_service.cc:434] P 0e9b0850145a4c66bfe983ffdfa93a57: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-1f9ddea0a887446f910d38f8302e8172 on tablet 1f9ddea0a887446f910d38f8302e8172 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:53.776180  9658 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:53.799219  9658 ts_tablet_manager.cc:1434] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.001s
I20260709 13:01:53.801107  8766 tablet_copy_service.cc:342] P 0e9b0850145a4c66bfe983ffdfa93a57: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-142891c077a34ba289adf90a99aee612 received from {username='slave'} at 127.0.0.1:35702
I20260709 13:01:53.801600  8766 tablet_copy_service.cc:434] P 0e9b0850145a4c66bfe983ffdfa93a57: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-142891c077a34ba289adf90a99aee612 on tablet 142891c077a34ba289adf90a99aee612 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:53.940171  9389 raft_consensus.cc:1217] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:54.001133  9389 raft_consensus.cc:1217] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:54.039837  9389 raft_consensus.cc:1217] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:54.171418  9389 raft_consensus.cc:1217] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:54.400578  9664 raft_consensus.cc:1064] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:54.403314  9664 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:54.411991  9389 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:54.414664  8676 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:54.417467  8824 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:54.416388  9604 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:01:54.419247  9625 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:54.422406  9625 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:01:54.474174  9666 raft_consensus.cc:2955] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.483311  8676 raft_consensus.cc:2955] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.483310  8824 raft_consensus.cc:2955] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.491544  9389 raft_consensus.cc:2955] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.563877  9664 raft_consensus.cc:1064] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:54.566771  9664 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:54.575389  8578 catalog_manager.cc:5697] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:54.593242  9671 raft_consensus.cc:1064] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:54.597972  9671 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:54.628094  9389 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:54.629540  8676 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:54.631683  9625 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:54.644703  8829 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:54.639482  9664 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:01:54.657130  9388 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:54.659453  9578 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:54.663297  9604 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:54.665406  8674 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:54.668745  9578 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:54.672515  8747 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:54.677340  9675 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:54.727993  9604 raft_consensus.cc:2955] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.735759  9387 raft_consensus.cc:2955] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.742133  8824 raft_consensus.cc:2955] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.749311  9671 raft_consensus.cc:2955] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.752225  9388 raft_consensus.cc:2955] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.793768  8677 raft_consensus.cc:2955] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.809538  9625 raft_consensus.cc:1064] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:54.809273  8674 raft_consensus.cc:2955] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.812731  9625 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:54.854939  8756 raft_consensus.cc:2955] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.866657  8747 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:54.879200  8576 catalog_manager.cc:5697] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.905124  8829 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:54.906149  9387 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:54.906347  9388 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:54.913374  8677 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:54.914482  8679 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:54.916002  9664 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:54.916249  9688 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:54.917721  8577 catalog_manager.cc:5697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.919363  8756 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:54.918524  9688 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:01:54.928453  9687 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:54.928035  9688 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:01:54.937059  8676 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:54.947295  9666 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:54.952409  9604 raft_consensus.cc:2955] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.970461  8829 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:54.980464  9389 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:54.984238  8680 raft_consensus.cc:2955] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:54.989957  9388 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:54.992393  8674 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:54.993250  9683 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:55.006717  9389 raft_consensus.cc:2955] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.008620  9687 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:55.012480  9632 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:55.004735  9635 raft_consensus.cc:2955] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.018887  9386 raft_consensus.cc:2955] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.021623  8681 raft_consensus.cc:2955] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.035508  8578 catalog_manager.cc:5697] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.043263  8564 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c01533e2a5204b1ebccac1133e495542 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:55.046047  9687 raft_consensus.cc:2955] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.046653  9625 raft_consensus.cc:2955] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.052098  8674 raft_consensus.cc:2955] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.052677  8679 raft_consensus.cc:2955] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.054883  8814 tablet_service.cc:1558] Processing DeleteTablet for tablet c01533e2a5204b1ebccac1133e495542 with delete_type TABLET_DATA_TOMBSTONED (TS e814ed2013244fa6be42623ea5c0c7b9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60414
I20260709 13:01:55.063027  9693 tablet_replica.cc:333] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:01:55.064718  9693 raft_consensus.cc:2243] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:55.064849  8578 catalog_manager.cc:5697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.065765  9693 raft_consensus.cc:2272] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:55.077387  9693 ts_tablet_manager.cc:1916] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:55.102313  9693 ts_tablet_manager.cc:1929] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:55.102952  9693 log.cc:1199] T c01533e2a5204b1ebccac1133e495542 P e814ed2013244fa6be42623ea5c0c7b9: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root/wals/c01533e2a5204b1ebccac1133e495542
I20260709 13:01:55.105119  8566 catalog_manager.cc:5028] TS e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): tablet c01533e2a5204b1ebccac1133e495542 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:55.106530  9386 raft_consensus.cc:2955] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.109345  8738 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ee1e688ccd74392befec90535c1efa4 with delete_type TABLET_DATA_TOMBSTONED (TS 0e9b0850145a4c66bfe983ffdfa93a57 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39018
I20260709 13:01:55.121687  8566 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5ee1e688ccd74392befec90535c1efa4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:55.122304  9389 raft_consensus.cc:2955] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.125201  8564 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 142891c077a34ba289adf90a99aee612 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:55.130086  9694 tablet_replica.cc:333] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:01:55.131470  9694 raft_consensus.cc:2243] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:55.132319  9694 raft_consensus.cc:2272] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:55.126315  8578 catalog_manager.cc:5697] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.144081  9694 ts_tablet_manager.cc:1916] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:55.150591  8829 raft_consensus.cc:2955] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.151526  8579 catalog_manager.cc:5697] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:55.206382  9694 ts_tablet_manager.cc:1929] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:55.206691  8814 tablet_service.cc:1558] Processing DeleteTablet for tablet 142891c077a34ba289adf90a99aee612 with delete_type TABLET_DATA_TOMBSTONED (TS e814ed2013244fa6be42623ea5c0c7b9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60414
I20260709 13:01:55.207873  9694 log.cc:1199] T 5ee1e688ccd74392befec90535c1efa4 P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-1-root/wals/5ee1e688ccd74392befec90535c1efa4
I20260709 13:01:55.208496  9693 tablet_replica.cc:333] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:01:55.208573  8747 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:55.209748  9693 raft_consensus.cc:2243] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:55.210057  8564 catalog_manager.cc:5028] TS 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): tablet 5ee1e688ccd74392befec90535c1efa4 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:55.210645  9693 raft_consensus.cc:2272] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:55.215252  9693 ts_tablet_manager.cc:1916] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:55.215308  9389 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:55.216522  8829 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:55.217247  9687 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:55.224154  9687 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:55.228574  9625 raft_consensus.cc:2955] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.232355  9389 raft_consensus.cc:2955] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.238550  8829 raft_consensus.cc:2955] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.240015  9693 ts_tablet_manager.cc:1929] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:55.241578  9693 log.cc:1199] T 142891c077a34ba289adf90a99aee612 P e814ed2013244fa6be42623ea5c0c7b9: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root/wals/142891c077a34ba289adf90a99aee612
I20260709 13:01:55.244149  8566 catalog_manager.cc:5028] TS e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): tablet 142891c077a34ba289adf90a99aee612 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:55.248960  8564 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1f9ddea0a887446f910d38f8302e8172 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:55.262890  8579 catalog_manager.cc:5697] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:55.284438  8665 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f9ddea0a887446f910d38f8302e8172 with delete_type TABLET_DATA_TOMBSTONED (TS 0e8c59a1f0394eae8ce33b99f10efadc not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53334
I20260709 13:01:55.304045  9695 tablet_replica.cc:333] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:01:55.306435  9695 raft_consensus.cc:2243] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:55.307523  9695 raft_consensus.cc:2272] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:55.312273  9695 ts_tablet_manager.cc:1916] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:55.328382  9695 ts_tablet_manager.cc:1929] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:55.328869  9695 log.cc:1199] T 1f9ddea0a887446f910d38f8302e8172 P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-0-root/wals/1f9ddea0a887446f910d38f8302e8172
I20260709 13:01:55.330852  8563 catalog_manager.cc:5028] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): tablet 1f9ddea0a887446f910d38f8302e8172 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:56.384460  8829 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:56.392196  8756 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:56.392401  8679 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:56.394094  8796 consensus_peers.cc:597] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41d15a8ebfe8480d9861c1285faaf7b5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:56.394116  9635 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:56.395301  9632 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:56.406039  9632 raft_consensus.cc:2955] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:56.408018  8756 raft_consensus.cc:2955] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:56.415593  8679 raft_consensus.cc:2955] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:56.428444  8576 catalog_manager.cc:5697] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:56.430181  8829 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
W20260709 13:01:56.456959  8796 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 05559e602d4b4ea3986701fa621706a0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:56.458678  8747 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:56.460588  9632 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:56.461016  8679 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:56.464349  9632 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:56.477438  9632 raft_consensus.cc:2955] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:56.480468  8747 raft_consensus.cc:2955] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:56.482836  8679 raft_consensus.cc:2955] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:56.512295  8679 consensus_queue.cc:237] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:56.524545  8579 catalog_manager.cc:5697] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:56.534842  8647 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a7eadf5c488447886a90612fbf1a2bb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:56.541203  8829 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:56.544783  8756 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:56.546262  9709 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:56.547601  9672 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:56.571581  9672 raft_consensus.cc:2955] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:56.573518  8756 raft_consensus.cc:2955] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:56.574347  8829 raft_consensus.cc:2955] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:56.594237  8578 catalog_manager.cc:5697] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:56.595371  8679 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
W20260709 13:01:56.613283  8647 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9f81d7aa5ed04c68b603d272f24760e2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:56.613586  8829 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:56.615690  9533 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:56.628916  8747 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:56.631407  9672 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:56.639215  9672 raft_consensus.cc:2955] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:56.648934  8756 raft_consensus.cc:2955] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:56.657255  8824 raft_consensus.cc:2955] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:56.667554  8579 catalog_manager.cc:5697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:01:56.986929  9716 ts_tablet_manager.cc:933] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:56.989837  9716 tablet_copy_client.cc:323] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:01:56.992442  8840 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet 41d15a8ebfe8480d9861c1285faaf7b5 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
I20260709 13:01:56.993083  8840 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet 41d15a8ebfe8480d9861c1285faaf7b5 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-41d15a8ebfe8480d9861c1285faaf7b5
I20260709 13:01:57.005234  8840 tablet_copy_source_session.cc:215] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:57.009334  9716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41d15a8ebfe8480d9861c1285faaf7b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:57.029752  9718 ts_tablet_manager.cc:933] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:01:57.030562  9716 tablet_copy_client.cc:806] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:57.031563  9716 tablet_copy_client.cc:670] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:57.035727  9718 tablet_copy_client.cc:323] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:01:57.036806  9716 tablet_copy_client.cc:538] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:57.039036  8840 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet 05559e602d4b4ea3986701fa621706a0 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
I20260709 13:01:57.039716  8840 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet 05559e602d4b4ea3986701fa621706a0 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-05559e602d4b4ea3986701fa621706a0
I20260709 13:01:57.049494  9716 tablet_bootstrap.cc:492] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:57.052628  8840 tablet_copy_source_session.cc:215] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:57.062139  9720 ts_tablet_manager.cc:933] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221)
I20260709 13:01:57.075590  9718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05559e602d4b4ea3986701fa621706a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:57.088260  9720 tablet_copy_client.cc:323] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.129:43221
I20260709 13:01:57.091601  8691 tablet_copy_service.cc:140] P 0e8c59a1f0394eae8ce33b99f10efadc: Received BeginTabletCopySession request for tablet 3a7eadf5c488447886a90612fbf1a2bb from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55676)
I20260709 13:01:57.092213  8691 tablet_copy_service.cc:161] P 0e8c59a1f0394eae8ce33b99f10efadc: Beginning new tablet copy session on tablet 3a7eadf5c488447886a90612fbf1a2bb from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55676: session id = 7aff04f53d4845a6a4fa491fd2a06145-3a7eadf5c488447886a90612fbf1a2bb
I20260709 13:01:57.103556  8691 tablet_copy_source_session.cc:215] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:57.111053  9720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a7eadf5c488447886a90612fbf1a2bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:57.152372  9720 tablet_copy_client.cc:806] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:57.153209  9720 tablet_copy_client.cc:670] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:57.158845  9720 tablet_copy_client.cc:538] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:57.180804  9718 tablet_copy_client.cc:806] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:57.181558  9718 tablet_copy_client.cc:670] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:57.193496  9720 tablet_bootstrap.cc:492] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:57.204442  9716 tablet_bootstrap.cc:492] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:57.205509  9716 tablet_bootstrap.cc:492] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:57.206297  9716 ts_tablet_manager.cc:1403] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.157s	user 0.047s	sys 0.018s
I20260709 13:01:57.209595  9716 raft_consensus.cc:359] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:57.210539  9716 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:57.211277  9716 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:57.223228  9718 tablet_copy_client.cc:538] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:57.224179  9716 ts_tablet_manager.cc:1434] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.004s
I20260709 13:01:57.226526  8840 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-41d15a8ebfe8480d9861c1285faaf7b5 received from {username='slave'} at 127.0.0.1:55168
I20260709 13:01:57.227397  8840 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-41d15a8ebfe8480d9861c1285faaf7b5 on tablet 41d15a8ebfe8480d9861c1285faaf7b5 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:57.251405  9716 ts_tablet_manager.cc:933] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221)
W20260709 13:01:57.285104  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 108726us
I20260709 13:01:57.291795  9718 tablet_bootstrap.cc:492] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:57.303264  9716 tablet_copy_client.cc:323] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.129:43221
I20260709 13:01:57.318681  8691 tablet_copy_service.cc:140] P 0e8c59a1f0394eae8ce33b99f10efadc: Received BeginTabletCopySession request for tablet 9f81d7aa5ed04c68b603d272f24760e2 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55676)
I20260709 13:01:57.366413  8691 tablet_copy_service.cc:161] P 0e8c59a1f0394eae8ce33b99f10efadc: Beginning new tablet copy session on tablet 9f81d7aa5ed04c68b603d272f24760e2 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55676: session id = 7aff04f53d4845a6a4fa491fd2a06145-9f81d7aa5ed04c68b603d272f24760e2
I20260709 13:01:57.426954  8691 tablet_copy_source_session.cc:215] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:01:57.438374  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 144339us
I20260709 13:01:57.461678  9720 tablet_bootstrap.cc:492] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:57.463999  9386 raft_consensus.cc:1217] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:57.476259  9716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f81d7aa5ed04c68b603d272f24760e2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:01:57.492027  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 52488.1us
I20260709 13:01:57.493796  9720 tablet_bootstrap.cc:492] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:57.531479  9720 ts_tablet_manager.cc:1403] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.338s	user 0.049s	sys 0.034s
I20260709 13:01:57.603304  9718 tablet_bootstrap.cc:492] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:57.626863  9718 tablet_bootstrap.cc:492] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:57.632445  9718 ts_tablet_manager.cc:1403] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.390s	user 0.021s	sys 0.020s
I20260709 13:01:57.641167  9718 raft_consensus.cc:359] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:57.622578  9720 raft_consensus.cc:359] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:57.695057  9720 raft_consensus.cc:740] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:57.695945  9720 consensus_queue.cc:260] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:57.699225  9720 ts_tablet_manager.cc:1434] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.167s	user 0.013s	sys 0.000s
I20260709 13:01:57.702512  8691 tablet_copy_service.cc:342] P 0e8c59a1f0394eae8ce33b99f10efadc: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-3a7eadf5c488447886a90612fbf1a2bb received from {username='slave'} at 127.0.0.1:55676
I20260709 13:01:57.703203  8691 tablet_copy_service.cc:434] P 0e8c59a1f0394eae8ce33b99f10efadc: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-3a7eadf5c488447886a90612fbf1a2bb on tablet 3a7eadf5c488447886a90612fbf1a2bb with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:57.700333  9718 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:57.791689  9718 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:57.794448  9718 ts_tablet_manager.cc:1434] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.157s	user 0.010s	sys 0.001s
I20260709 13:01:57.799804  8840 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-05559e602d4b4ea3986701fa621706a0 received from {username='slave'} at 127.0.0.1:55168
I20260709 13:01:57.800356  8840 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-05559e602d4b4ea3986701fa621706a0 on tablet 05559e602d4b4ea3986701fa621706a0 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:57.809578  9716 tablet_copy_client.cc:806] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:57.810366  9716 tablet_copy_client.cc:670] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:57.821835  9716 tablet_copy_client.cc:538] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:57.864331  9716 tablet_bootstrap.cc:492] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:01:57.901945  9716 tablet_bootstrap.cc:492] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:57.903529  9716 tablet_bootstrap.cc:492] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:01:57.904409  9716 ts_tablet_manager.cc:1403] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.011s
I20260709 13:01:57.908094  9716 raft_consensus.cc:359] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:57.909157  9716 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:01:57.910284  9716 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:01:57.919198  9716 ts_tablet_manager.cc:1434] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:01:57.922420  8691 tablet_copy_service.cc:342] P 0e8c59a1f0394eae8ce33b99f10efadc: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-9f81d7aa5ed04c68b603d272f24760e2 received from {username='slave'} at 127.0.0.1:55676
I20260709 13:01:57.923019  8691 tablet_copy_service.cc:434] P 0e8c59a1f0394eae8ce33b99f10efadc: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-9f81d7aa5ed04c68b603d272f24760e2 on tablet 9f81d7aa5ed04c68b603d272f24760e2 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:01:57.925916  9742 raft_consensus.cc:1064] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:57.929056  9742 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:57.957453  9389 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:57.960505  9734 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:57.975904  8747 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:57.979411  8680 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:57.984187  9741 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:57.986723  9737 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:58.013196  9739 raft_consensus.cc:2955] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.022037  9389 raft_consensus.cc:2955] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.022773  8680 raft_consensus.cc:2955] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.051729  8579 catalog_manager.cc:5697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.074987  8747 raft_consensus.cc:2955] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.145637  8829 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:58.156260  9389 raft_consensus.cc:1217] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:58.158674  9386 raft_consensus.cc:1217] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:58.174726  9386 raft_consensus.cc:1217] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:58.193786  9389 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:58.199491  9742 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:58.203653  8748 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:58.207922  9742 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:58.214183  9741 raft_consensus.cc:2955] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.217835  9389 raft_consensus.cc:2955] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.249322  8747 raft_consensus.cc:2955] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.273303  8566 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 41d15a8ebfe8480d9861c1285faaf7b5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:58.299626  8579 catalog_manager.cc:5697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.314088  8665 tablet_service.cc:1558] Processing DeleteTablet for tablet 41d15a8ebfe8480d9861c1285faaf7b5 with delete_type TABLET_DATA_TOMBSTONED (TS 0e8c59a1f0394eae8ce33b99f10efadc not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53334
I20260709 13:01:58.316495  9754 tablet_replica.cc:333] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:01:58.317881  9754 raft_consensus.cc:2243] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:58.318833  9754 raft_consensus.cc:2272] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:58.322124  9754 ts_tablet_manager.cc:1916] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:58.339236  9754 ts_tablet_manager.cc:1929] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:58.339871  9754 log.cc:1199] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-0-root/wals/41d15a8ebfe8480d9861c1285faaf7b5
I20260709 13:01:58.343061  8563 catalog_manager.cc:5028] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): tablet 41d15a8ebfe8480d9861c1285faaf7b5 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:58.540997  9752 raft_consensus.cc:1064] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:58.544030  9752 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:58.553115  9753 raft_consensus.cc:1064] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:58.555966  9753 consensus_queue.cc:237] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:58.567376  9386 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:58.569484  9709 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:58.573843  8824 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:58.577965  9735 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:58.592455  8829 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:58.592512  9389 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:58.594712  9753 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:58.597030  9753 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:58.598744  8756 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:58.601248  8756 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:58.613404  9735 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:58.619015  9752 raft_consensus.cc:2955] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.616381  9753 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:01:58.638329  9386 raft_consensus.cc:2955] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.665581  8578 catalog_manager.cc:5697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:58.623080  8824 raft_consensus.cc:2955] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.772980  9742 raft_consensus.cc:1064] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:01:58.776218  9742 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:01:58.834596  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 87092.4us
I20260709 13:01:58.845183  9753 raft_consensus.cc:2955] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.852072  8680 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:58.857903  8747 raft_consensus.cc:2955] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.861399  8748 raft_consensus.cc:2955] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.878731  9386 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:58.880406  9752 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:58.888849  9709 raft_consensus.cc:2955] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.891103  9387 raft_consensus.cc:2955] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.905145  8827 raft_consensus.cc:2955] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.927416  8751 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:58.933531  9752 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:58.947342  9389 raft_consensus.cc:2955] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.955132  8756 raft_consensus.cc:2955] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:58.966553  8578 catalog_manager.cc:5697] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:58.974372  8563 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9f81d7aa5ed04c68b603d272f24760e2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:59.021420  9386 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:59.024309  9765 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:59.021752  8578 catalog_manager.cc:5697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:59.035153  8679 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:59.038709  9724 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:59.053879  8679 consensus_queue.cc:237] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:59.054549  8814 tablet_service.cc:1558] Processing DeleteTablet for tablet 9f81d7aa5ed04c68b603d272f24760e2 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 4)) from {username='slave'} at 127.0.0.1:60414
I20260709 13:01:59.083060  9386 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:59.087648  9709 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:59.092458  8824 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:59.099431  9768 tablet_replica.cc:333] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:01:59.104736  9768 raft_consensus.cc:2243] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.105967  9765 raft_consensus.cc:2955] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:59.107987  9768 raft_consensus.cc:2272] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.112958  8747 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:59.114115  8679 raft_consensus.cc:2955] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:59.126734  9768 ts_tablet_manager.cc:1916] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:59.132495  9709 raft_consensus.cc:2955] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:59.134582  9389 raft_consensus.cc:2955] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:59.147423  8576 catalog_manager.cc:5697] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:59.164969  9386 raft_consensus.cc:2955] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:59.184407  9724 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:59.186036  9753 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:59.189074  9768 ts_tablet_manager.cc:1929] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:59.192693  9768 log.cc:1199] T 9f81d7aa5ed04c68b603d272f24760e2 P e814ed2013244fa6be42623ea5c0c7b9: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root/wals/9f81d7aa5ed04c68b603d272f24760e2
I20260709 13:01:59.195667  8566 catalog_manager.cc:5028] TS e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): tablet 9f81d7aa5ed04c68b603d272f24760e2 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:59.195979  8824 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:01:59.203490  8674 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:59.217821  9742 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:59.218505  8576 catalog_manager.cc:5697] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:59.249575  8830 raft_consensus.cc:2955] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:59.267875  9764 raft_consensus.cc:2955] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:59.274505  8738 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a7eadf5c488447886a90612fbf1a2bb with delete_type TABLET_DATA_TOMBSTONED (TS 0e9b0850145a4c66bfe983ffdfa93a57 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39018
I20260709 13:01:59.275398  8681 raft_consensus.cc:2955] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:59.292201  8566 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 05559e602d4b4ea3986701fa621706a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:59.304107  9771 tablet_replica.cc:333] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:01:59.305951  9771 raft_consensus.cc:2243] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.311488  9771 raft_consensus.cc:2272] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.308920  8756 raft_consensus.cc:2955] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:59.305451  8576 catalog_manager.cc:5697] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:59.329099  9771 ts_tablet_manager.cc:1916] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:59.338083  8738 tablet_service.cc:1558] Processing DeleteTablet for tablet 05559e602d4b4ea3986701fa621706a0 with delete_type TABLET_DATA_TOMBSTONED (TS 0e9b0850145a4c66bfe983ffdfa93a57 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39018
I20260709 13:01:59.346819  8563 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3a7eadf5c488447886a90612fbf1a2bb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:59.403127  9387 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:59.407534  9771 ts_tablet_manager.cc:1929] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:59.408207  9771 log.cc:1199] T 3a7eadf5c488447886a90612fbf1a2bb P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-1-root/wals/3a7eadf5c488447886a90612fbf1a2bb
I20260709 13:01:59.410313  9771 tablet_replica.cc:333] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:01:59.411476  8564 catalog_manager.cc:5028] TS 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): tablet 3a7eadf5c488447886a90612fbf1a2bb (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:01:59.411521  9771 raft_consensus.cc:2243] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.412648  9771 raft_consensus.cc:2272] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.416569  9741 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:59.417855  9771 ts_tablet_manager.cc:1916] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:59.426347  9386 raft_consensus.cc:2955] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:01:59.441758  9771 ts_tablet_manager.cc:1929] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:59.442348  9771 log.cc:1199] T 05559e602d4b4ea3986701fa621706a0 P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-1-root/wals/05559e602d4b4ea3986701fa621706a0
I20260709 13:01:59.443789  8564 catalog_manager.cc:5028] TS 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): tablet 05559e602d4b4ea3986701fa621706a0 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:02:00.463450  8756 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:00.472651  8828 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:00.472867  8722 consensus_peers.cc:597] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b50b38d6e5314ca190c9a14a5e0f655d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:00.473912  8681 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:00.474751  9777 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:00.475857  9770 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:00.484934  9725 raft_consensus.cc:2955] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.486788  8828 raft_consensus.cc:2955] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.491158  8681 raft_consensus.cc:2955] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.505630  8578 catalog_manager.cc:5697] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.531409  8828 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:00.540720  8756 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:00.541219  8681 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:00.541779  8796 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 95fdf24d372f40b8877c2792102f34cf. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:00.542462  9774 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:00.543516  9739 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:00.554278  9774 raft_consensus.cc:2955] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.556411  8681 raft_consensus.cc:2955] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.563428  8756 raft_consensus.cc:2955] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.574769  8579 catalog_manager.cc:5697] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.595474  8828 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
W20260709 13:02:00.655750  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 70460.6us
I20260709 13:02:00.659605  8748 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:02:00.660111  8796 consensus_peers.cc:597] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 293a9dab5be3479a90d5803585682a00. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:00.663252  9766 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:00.666131  8679 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:00.667761  9790 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:00.674778  9789 raft_consensus.cc:2955] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.676286  8748 raft_consensus.cc:2955] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.697919  8681 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:00.699950  8579 catalog_manager.cc:5697] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:00.691787  8679 raft_consensus.cc:2955] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.734690  8747 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:00.736660  9792 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:00.734690  8824 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:00.739210  9773 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:02:00.741436  8647 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2217efcaa5fe4b729f1beef2f96e4189. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:00.755208  9709 raft_consensus.cc:2955] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.769979  8824 raft_consensus.cc:2955] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.772400  8748 raft_consensus.cc:2955] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.798336  8576 catalog_manager.cc:5697] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:00.987305  9800 ts_tablet_manager.cc:933] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:00.999085  9800 tablet_copy_client.cc:323] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:02:01.001197  8840 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet 95fdf24d372f40b8877c2792102f34cf from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
I20260709 13:02:01.001902  8840 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet 95fdf24d372f40b8877c2792102f34cf from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-95fdf24d372f40b8877c2792102f34cf
I20260709 13:02:01.011606  8840 tablet_copy_source_session.cc:215] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:01.025292  9800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95fdf24d372f40b8877c2792102f34cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.043761  9800 tablet_copy_client.cc:806] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:01.044425  9800 tablet_copy_client.cc:670] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:01.048856  9800 tablet_copy_client.cc:538] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:01.058863  9800 tablet_bootstrap.cc:492] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:02:01.072435  9802 ts_tablet_manager.cc:933] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:01.074385  9802 tablet_copy_client.cc:323] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.130:42321
I20260709 13:02:01.076334  8766 tablet_copy_service.cc:140] P 0e9b0850145a4c66bfe983ffdfa93a57: Received BeginTabletCopySession request for tablet b50b38d6e5314ca190c9a14a5e0f655d from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:35702)
I20260709 13:02:01.076952  8766 tablet_copy_service.cc:161] P 0e9b0850145a4c66bfe983ffdfa93a57: Beginning new tablet copy session on tablet b50b38d6e5314ca190c9a14a5e0f655d from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:35702: session id = 7aff04f53d4845a6a4fa491fd2a06145-b50b38d6e5314ca190c9a14a5e0f655d
I20260709 13:02:01.085837  8766 tablet_copy_source_session.cc:215] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:01.087200  9800 tablet_bootstrap.cc:492] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:01.088701  9800 tablet_bootstrap.cc:492] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:02:01.089568  9800 ts_tablet_manager.cc:1403] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.015s
I20260709 13:02:01.089986  9802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b50b38d6e5314ca190c9a14a5e0f655d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.092911  9800 raft_consensus.cc:359] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:01.093855  9800 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:02:01.094506  9800 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:01.098392  9800 ts_tablet_manager.cc:1434] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.003s
I20260709 13:02:01.108978  8840 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-95fdf24d372f40b8877c2792102f34cf received from {username='slave'} at 127.0.0.1:55168
I20260709 13:02:01.109488  8840 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-95fdf24d372f40b8877c2792102f34cf on tablet 95fdf24d372f40b8877c2792102f34cf with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:01.118312  9802 tablet_copy_client.cc:806] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:01.119124  9802 tablet_copy_client.cc:670] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:01.123490  9802 tablet_copy_client.cc:538] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:01.135020  9802 tablet_bootstrap.cc:492] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:02:01.157996  9802 tablet_bootstrap.cc:492] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:01.159055  9802 tablet_bootstrap.cc:492] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:02:01.159869  9802 ts_tablet_manager.cc:1403] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.025s	user 0.005s	sys 0.022s
I20260709 13:02:01.163362  9802 raft_consensus.cc:359] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:01.164343  9802 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:02:01.165056  9802 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:01.171187  9802 ts_tablet_manager.cc:1434] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.005s
I20260709 13:02:01.173054  8766 tablet_copy_service.cc:342] P 0e9b0850145a4c66bfe983ffdfa93a57: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-b50b38d6e5314ca190c9a14a5e0f655d received from {username='slave'} at 127.0.0.1:35702
I20260709 13:02:01.173539  8766 tablet_copy_service.cc:434] P 0e9b0850145a4c66bfe983ffdfa93a57: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-b50b38d6e5314ca190c9a14a5e0f655d on tablet b50b38d6e5314ca190c9a14a5e0f655d with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:01.179061  9802 ts_tablet_manager.cc:933] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:01.185631  9802 tablet_copy_client.cc:323] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:02:01.192072  8840 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet 293a9dab5be3479a90d5803585682a00 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
I20260709 13:02:01.193001  8840 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet 293a9dab5be3479a90d5803585682a00 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-293a9dab5be3479a90d5803585682a00
I20260709 13:02:01.200133  9800 ts_tablet_manager.cc:933] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221)
I20260709 13:02:01.226799  9800 tablet_copy_client.cc:323] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.129:43221
I20260709 13:02:01.238481  8691 tablet_copy_service.cc:140] P 0e8c59a1f0394eae8ce33b99f10efadc: Received BeginTabletCopySession request for tablet 2217efcaa5fe4b729f1beef2f96e4189 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55676)
I20260709 13:02:01.239162  8691 tablet_copy_service.cc:161] P 0e8c59a1f0394eae8ce33b99f10efadc: Beginning new tablet copy session on tablet 2217efcaa5fe4b729f1beef2f96e4189 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55676: session id = 7aff04f53d4845a6a4fa491fd2a06145-2217efcaa5fe4b729f1beef2f96e4189
I20260709 13:02:01.250818  8840 tablet_copy_source_session.cc:215] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:01.254499  9802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293a9dab5be3479a90d5803585682a00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.260545  8691 tablet_copy_source_session.cc:215] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:01.272648  9800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2217efcaa5fe4b729f1beef2f96e4189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:01.282207  9802 tablet_copy_client.cc:806] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:01.283037  9802 tablet_copy_client.cc:670] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:01.305989  9800 tablet_copy_client.cc:806] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:01.306479  9802 tablet_copy_client.cc:538] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:01.318101  9802 tablet_bootstrap.cc:492] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:02:01.318725  9800 tablet_copy_client.cc:670] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:01.324271  9800 tablet_copy_client.cc:538] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:01.334329  9800 tablet_bootstrap.cc:492] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:02:01.354372  9802 tablet_bootstrap.cc:492] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:01.355543  9802 tablet_bootstrap.cc:492] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:02:01.356338  9802 ts_tablet_manager.cc:1403] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.039s	user 0.036s	sys 0.001s
I20260709 13:02:01.360232  9802 raft_consensus.cc:359] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:01.361218  9802 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:02:01.361967  9802 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:01.369109  9802 ts_tablet_manager.cc:1434] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:02:01.371033  8840 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-293a9dab5be3479a90d5803585682a00 received from {username='slave'} at 127.0.0.1:55168
I20260709 13:02:01.371536  8840 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-293a9dab5be3479a90d5803585682a00 on tablet 293a9dab5be3479a90d5803585682a00 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:01.380743  9389 raft_consensus.cc:1217] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:01.388783  9800 tablet_bootstrap.cc:492] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:01.389986  9800 tablet_bootstrap.cc:492] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:02:01.390872  9800 ts_tablet_manager.cc:1403] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.057s	user 0.019s	sys 0.019s
I20260709 13:02:01.394588  9800 raft_consensus.cc:359] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:01.395574  9800 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:02:01.396252  9800 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:01.399255  9800 ts_tablet_manager.cc:1434] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:02:01.400992  8691 tablet_copy_service.cc:342] P 0e8c59a1f0394eae8ce33b99f10efadc: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-2217efcaa5fe4b729f1beef2f96e4189 received from {username='slave'} at 127.0.0.1:55676
I20260709 13:02:01.401448  8691 tablet_copy_service.cc:434] P 0e8c59a1f0394eae8ce33b99f10efadc: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-2217efcaa5fe4b729f1beef2f96e4189 on tablet 2217efcaa5fe4b729f1beef2f96e4189 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:01.473671  9389 raft_consensus.cc:1217] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:01.583953  9386 raft_consensus.cc:1217] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:01.640682  9386 raft_consensus.cc:1217] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:01.901664  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:01.919641  9766 raft_consensus.cc:1064] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:02:01.923164  9766 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:01.932710  9386 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:01.935499  8679 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:01.937350  9789 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:01.939760  8756 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:01.952286  9790 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:01.954246  9739 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:02.056867  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 61081.7us
I20260709 13:02:02.076924  9790 raft_consensus.cc:1064] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:02:02.086087  9789 raft_consensus.cc:2955] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:02.091949  9386 raft_consensus.cc:2955] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:02.083350  9790 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:02.161099  8576 catalog_manager.cc:5697] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:02.202534  9770 raft_consensus.cc:1064] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
W20260709 13:02:02.247659  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 120279us
W20260709 13:02:02.251034  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 133732us
I20260709 13:02:02.254343  8679 raft_consensus.cc:2955] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:02.268393  8756 raft_consensus.cc:2955] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:02:02.335709  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 64739.8us
W20260709 13:02:02.364063  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 140984us
I20260709 13:02:02.334105  9770 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:02.437153  9386 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:02.485778  9847 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:02:02.543071  9773 raft_consensus.cc:1064] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:02:02.564095  8679 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:02.574461  9833 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:02.550032  9773 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:02.680918  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.692569  9389 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:02.696985  8826 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:02.741909  8756 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:02.747933  9389 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:02:02.748869  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 266947us
I20260709 13:02:02.781564  8756 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:02.788359  9709 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:02.700013  9826 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
I20260709 13:02:02.795131  9798 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:02:02.842765  9837 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:02:02.921718  8679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:02.924433  8679 raft_consensus.cc:606] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:02.933535  8680 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:02.940470  8680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:02.941466  8680 raft_consensus.cc:606] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:02.960067  8680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:02.961594  8680 raft_consensus.cc:606] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Received request to transfer leadership to TS 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:02.985157  9822 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:03.036968  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 208906us
I20260709 13:02:03.065110  9773 raft_consensus.cc:993] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc: : Instructing follower 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:03.065979  9773 raft_consensus.cc:1081] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Signalling peer 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:03.069156  9386 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:34746
I20260709 13:02:03.070158  9386 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:03.070756  9386 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:03.092084  9828 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:03.124017  9844 raft_consensus.cc:2955] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.138510  8826 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:03.151829  9837 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:03.163733  9389 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:03.164001  8826 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:03.152846  9837 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:03.168646  9837 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:03.168557  9828 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:03.182092  9385 raft_consensus.cc:2955] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.187441  9773 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:03.189505  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:03.208060  9386 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:03.258298  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:03.259604  8828 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:03.263208  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:03.264031  8674 raft_consensus.cc:3055] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:03.264376  8674 raft_consensus.cc:740] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:03.238054  9766 raft_consensus.cc:2955] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.268517  8756 raft_consensus.cc:2955] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.272739  9386 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:03.265340  8674 consensus_queue.cc:260] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:03.275539  8674 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:03.282516  8751 raft_consensus.cc:2955] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.291522  8576 catalog_manager.cc:5697] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:03.295429  9844 raft_consensus.cc:993] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: : Instructing follower 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:03.295732  9385 raft_consensus.cc:2955] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.301793  9389 raft_consensus.cc:2955] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.304363  9739 raft_consensus.cc:2955] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.333881  8674 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:03.332288  8828 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:03.332294  9854 raft_consensus.cc:1081] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Signalling peer 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:03.332471  9826 raft_consensus.cc:2955] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.329439  8826 raft_consensus.cc:2955] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.361239  9352 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:03.368052  9872 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:03.372869  9844 raft_consensus.cc:993] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc: : Instructing follower 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:03.414947  9854 raft_consensus.cc:1081] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Signalling peer 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:03.436583  8748 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:39032
I20260709 13:02:03.439931  8748 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:03.443985  8748 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:03.437796  9872 raft_consensus.cc:697] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:03.442930  8680 raft_consensus.cc:2955] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:02:03.526537  9873 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:02:03.536417  9389 raft_consensus.cc:2955] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.545846  8756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:39032
I20260709 13:02:03.552852  8756 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:03.553532  8756 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:03.563488  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:03.538493  9872 consensus_queue.cc:237] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:03.588056  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.590313  8566 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 95fdf24d372f40b8877c2792102f34cf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:03.609738  8756 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:03.649987  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:03.651283  8674 raft_consensus.cc:3055] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:03.651659  8674 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:03.655959  8756 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:03.659085  8674 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:03.687320  8674 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:03.713092  8748 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:03.723496  8677 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:03.737160  8679 raft_consensus.cc:2955] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.753181  8753 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:03.754755  8748 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:03.759629  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:03.762868  9837 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:03.769239  9826 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:03.769953  9826 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:03.774665  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:03.788535  8829 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:03.789608  9839 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:03.791194  9825 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:03.791074  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:03.791677  9825 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:03.792579  9825 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:03.794374  9825 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:03.775128  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:03.795418  8824 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:03.795289  9766 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:03.798494  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:03.775383  8826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:58932
I20260709 13:02:03.805152  8826 raft_consensus.cc:606] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:03.784477  8674 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:03.772946  9826 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:03.814296  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:03.815811  8824 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:03.816006  8826 raft_consensus.cc:2955] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.817312  8674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:03.817965  8674 raft_consensus.cc:606] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Received request to transfer leadership to TS 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:03.818552  8829 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:03.819787  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:03.815969  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:03.826340  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:03.828655  8719 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:03.833714  8677 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:03.834954  9826 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:03.840969  8722 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:03.846182  9385 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:03.855254  9825 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:03.860365  8577 catalog_manager.cc:5697] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:03.848006  9837 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:03.847437  8674 raft_consensus.cc:2955] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.865643  8752 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:03.846840  9826 raft_consensus.cc:697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:03.885047  9826 consensus_queue.cc:237] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:03.885200  9854 raft_consensus.cc:2955] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.898444  8679 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:03.907270  8738 tablet_service.cc:1558] Processing DeleteTablet for tablet 95fdf24d372f40b8877c2792102f34cf with delete_type TABLET_DATA_TOMBSTONED (TS 0e9b0850145a4c66bfe983ffdfa93a57 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39018
I20260709 13:02:03.907756  9385 raft_consensus.cc:2955] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:03.909328  9825 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:03.911197  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:03.912158  9837 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:03.912916  9837 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:03.913967  9837 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:03.931388  8563 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2217efcaa5fe4b729f1beef2f96e4189 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:03.926270  8752 raft_consensus.cc:2955] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:04.024969  9854 raft_consensus.cc:993] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: : Instructing follower 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:04.027496  9854 raft_consensus.cc:1081] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Signalling peer 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:04.031629  8576 catalog_manager.cc:5697] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 2, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 2 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:04.035686  9389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:34746
I20260709 13:02:04.036391  9389 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:04.036808  9389 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:04.050761  9389 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:04.061128  9885 tablet_replica.cc:333] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:02:04.070232  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:04.071033  8679 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:04.071347  8679 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:04.072219  8679 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:04.073473  8679 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:04.073645  9389 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:04.087211  8679 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:04.089275  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:04.090466  9872 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:04.091123  9872 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:04.092346  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:04.092190  9872 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:04.093187  8748 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:04.034088  8576 catalog_manager.cc:5697] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:04.103770  8576 catalog_manager.cc:5697] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:04.122385  8748 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
W20260709 13:02:04.126920  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46545.6us
I20260709 13:02:04.140899  9885 raft_consensus.cc:2243] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.142277  9885 raft_consensus.cc:2272] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.146194  8578 catalog_manager.cc:5697] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New cstate: current_term: 1 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:04.155527  8826 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:04.158119  8679 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:04.160069  9872 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:04.163447  9872 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:04.165061  9766 raft_consensus.cc:993] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: : Instructing follower 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:02:04.165634  9766 raft_consensus.cc:1081] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Signalling peer 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:02:04.170794  8577 catalog_manager.cc:5697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 1 to 2, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:04.177326  8814 tablet_service.cc:1558] Processing DeleteTablet for tablet 2217efcaa5fe4b729f1beef2f96e4189 with delete_type TABLET_DATA_TOMBSTONED (TS e814ed2013244fa6be42623ea5c0c7b9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60414
I20260709 13:02:04.191603  9885 ts_tablet_manager.cc:1916] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:04.207095  8575 catalog_manager.cc:5697] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 2, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 2 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:04.228969  8824 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:04.217764  8753 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:04.237884  9389 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:04.244993  9386 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:04.248714  9837 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:04.254580  9832 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:04.259773  8752 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:04.260972  8677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:53342
I20260709 13:02:04.261624  8677 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:04.261996  8677 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:04.273463  9893 tablet_replica.cc:333] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:02:04.272761  9838 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:04.275175  9893 raft_consensus.cc:2243] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.278018  9838 raft_consensus.cc:2955] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:04.281229  9389 raft_consensus.cc:2955] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:04.295681  9883 raft_consensus.cc:2955] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:04.297133  8826 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:04.300014  9386 raft_consensus.cc:2955] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:04.302587  8748 raft_consensus.cc:2955] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:04.298363  9893 raft_consensus.cc:2272] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.311064  9826 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:04.269979  8577 catalog_manager.cc:5697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 1 to 2, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:04.340708  8824 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:04.344244  9830 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:04.346385  8677 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:04.349815  8677 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:04.352398  8828 raft_consensus.cc:2955] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:04.355824  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:04.356576  8756 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:04.361799  8566 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 293a9dab5be3479a90d5803585682a00 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:04.368238  8756 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:04.375648  8676 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:04.380623  8645 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:04.381637  9823 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:04.382400  9823 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:04.383504  9823 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:04.392409  8756 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:04.394145  9893 ts_tablet_manager.cc:1916] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:04.393968  8575 catalog_manager.cc:5697] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:04.402505  9837 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:04.407367  9828 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:04.408668  9838 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:04.409072  9838 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:04.409981  9838 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:04.411413  9838 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:04.411512  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:04.434989  8564 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b50b38d6e5314ca190c9a14a5e0f655d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:04.438201  8575 catalog_manager.cc:5697] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 3 to 4, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:04.444873  9885 ts_tablet_manager.cc:1929] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:04.453236  8577 catalog_manager.cc:5697] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 2 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:04.455854  8665 tablet_service.cc:1558] Processing DeleteTablet for tablet 293a9dab5be3479a90d5803585682a00 with delete_type TABLET_DATA_TOMBSTONED (TS 0e8c59a1f0394eae8ce33b99f10efadc not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53334
I20260709 13:02:04.462568  8826 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:04.467494  8828 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:04.469357  9885 log.cc:1199] T 95fdf24d372f40b8877c2792102f34cf P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-1-root/wals/95fdf24d372f40b8877c2792102f34cf
I20260709 13:02:04.473292  9878 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:02:04.483734  9900 tablet_replica.cc:333] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:02:04.484042  8564 catalog_manager.cc:5028] TS 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): tablet 95fdf24d372f40b8877c2792102f34cf (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:02:04.485471  9900 raft_consensus.cc:2243] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.491528  9893 ts_tablet_manager.cc:1929] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:04.492125  9893 log.cc:1199] T 2217efcaa5fe4b729f1beef2f96e4189 P e814ed2013244fa6be42623ea5c0c7b9: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root/wals/2217efcaa5fe4b729f1beef2f96e4189
I20260709 13:02:04.488113  9900 raft_consensus.cc:2272] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.497179  8676 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:04.498899  8566 catalog_manager.cc:5028] TS e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): tablet 2217efcaa5fe4b729f1beef2f96e4189 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:02:04.502964  9900 ts_tablet_manager.cc:1916] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:04.510746  9878 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:02:04.517814  8665 tablet_service.cc:1558] Processing DeleteTablet for tablet b50b38d6e5314ca190c9a14a5e0f655d with delete_type TABLET_DATA_TOMBSTONED (TS 0e8c59a1f0394eae8ce33b99f10efadc not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53334
I20260709 13:02:04.537194  9900 ts_tablet_manager.cc:1929] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:04.538101  9900 log.cc:1199] T 293a9dab5be3479a90d5803585682a00 P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-0-root/wals/293a9dab5be3479a90d5803585682a00
I20260709 13:02:04.541142  9900 tablet_replica.cc:333] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:02:04.541767  8563 catalog_manager.cc:5028] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): tablet 293a9dab5be3479a90d5803585682a00 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:02:04.542680  9900 raft_consensus.cc:2243] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.544104  9900 raft_consensus.cc:2272] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.555886  8748 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:04.556460  9900 ts_tablet_manager.cc:1916] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:02:04.557921  9872 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:04.584246  9900 ts_tablet_manager.cc:1929] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:02:04.584833  9900 log.cc:1199] T b50b38d6e5314ca190c9a14a5e0f655d P 0e8c59a1f0394eae8ce33b99f10efadc: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-0-root/wals/b50b38d6e5314ca190c9a14a5e0f655d
I20260709 13:02:04.587728  8563 catalog_manager.cc:5028] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): tablet b50b38d6e5314ca190c9a14a5e0f655d (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:02:04.601442  8679 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:04.604368  9876 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:04.776242  8748 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:04.777679  9823 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:04.790307  8828 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:04.792155  9839 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:04.827862  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:04.881029  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:04.935679  8748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:02:04.936286  8748 raft_consensus.cc:606] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:04.939189  8679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:04.939805  8679 raft_consensus.cc:606] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Received request to transfer leadership to TS 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:04.942720  8748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:02:04.943275  8748 raft_consensus.cc:606] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:04.947101  8826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
mode: GRACEFUL
new_leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:58932
I20260709 13:02:04.947664  8826 raft_consensus.cc:606] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Received request to transfer leadership to TS 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:04.950790  8826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:58932
I20260709 13:02:04.951472  8826 raft_consensus.cc:606] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:04.952696  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:04.953368  9830 raft_consensus.cc:993] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: : Instructing follower 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:02:04.953977  9878 raft_consensus.cc:1081] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Signalling peer 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:02:04.954897  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:04.955873  8677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:53352
I20260709 13:02:04.956588  8677 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:04.957082  8677 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.967057  8677 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:04.969152  8677 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:04.970808  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:04.970860  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:04.971976  8748 raft_consensus.cc:3055] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:04.972214  8826 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.972313  8748 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:04.973285  8748 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:04.974335  8748 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:04.980773  8826 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:04.980773  8748 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:04.982244  8645 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:04.982992  9839 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:04.983497  9839 raft_consensus.cc:697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:04.984506  9839 consensus_queue.cc:237] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:05.005316  8575 catalog_manager.cc:5697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 2 to 3, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 3 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:05.033015  9823 raft_consensus.cc:993] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: : Instructing follower 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:05.033596  9839 raft_consensus.cc:1081] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Signalling peer 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:05.035667  9389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:34746
I20260709 13:02:05.036204  9389 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:05.036482  9389 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:05.044328  9389 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:05.046396  9389 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:05.047430  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:05.047760  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:05.048161  8677 raft_consensus.cc:3055] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:05.048385  8826 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:05.048520  8677 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:05.049894  8677 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:05.051235  8677 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:05.061368  8677 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:05.063390  9352 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:05.064054  8826 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:05.064388  9872 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:05.065838  9872 raft_consensus.cc:697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:05.068015  9872 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:05.092504  8575 catalog_manager.cc:5697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 2, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 2 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:05.143723  9826 raft_consensus.cc:993] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: : Instructing follower 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:02:05.144444  9878 raft_consensus.cc:1081] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Signalling peer 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:02:05.146095  8677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:53352
I20260709 13:02:05.146742  8677 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:05.147110  8677 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:05.149214  9889 raft_consensus.cc:993] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: : Instructing follower 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:05.149824  9889 raft_consensus.cc:1081] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Signalling peer 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:05.152092  9389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:34734
I20260709 13:02:05.152817  9389 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:05.153182  9389 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:05.156157  8677 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:05.158393  8677 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:05.159461  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:05.160182  8826 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:05.162992  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:05.163736  8753 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:05.164053  8753 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:05.164875  8753 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:05.166249  8753 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:05.167120  9389 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:05.170352  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:05.171089  8756 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:05.171371  9389 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:05.173583  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:05.174407  8828 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:05.174837  8828 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:05.175868  8828 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:05.178256  8828 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:05.179186  8753 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:05.187774  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:05.188792  9839 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:05.188635  8826 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:05.189615  9839 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:05.188344  8756 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:05.191504  9839 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:05.187781  8828 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:05.192950  9353 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:05.194098  9872 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:05.194708  9872 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:05.195703  9872 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:05.215886  8575 catalog_manager.cc:5697] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 2, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 2 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:05.218595  8577 catalog_manager.cc:5697] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 2 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:05.231624  9838 raft_consensus.cc:993] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: : Instructing follower 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:05.232208  9838 raft_consensus.cc:1081] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Signalling peer 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:05.235867  8753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:39024
I20260709 13:02:05.236706  8753 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:05.237090  8753 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:05.246954  8753 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:05.249482  8753 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:05.250191  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:05.250932  8677 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:05.253849  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:05.254575  8828 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:05.254912  8828 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:05.255707  8828 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:05.256834  8828 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:05.258889  8677 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:05.260371  8719 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:05.261159  9826 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:05.261787  9826 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:05.263868  8828 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:05.263582  9826 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:05.277699  8575 catalog_manager.cc:5697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:05.501097  8828 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:05.502655  9839 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:05.522502  8679 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:05.525533  8828 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:05.525854  9872 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:05.527215  9876 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:05.532758  8753 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:05.536558  9920 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:05.654209  8748 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:05.655721  9823 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:05.658358  8753 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:05.666464  8748 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:05.669694  9872 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:05.690053  8826 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:05.692660  8828 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:02:05.693720  8722 consensus_peers.cc:597] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2147fd3c99a94f0c8e9c1a914bdb79c3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:05.694868  9876 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:05.696452  8677 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:05.698762  8826 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:05.701472  9878 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:05.703675  9826 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:05.719650  8824 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:05.721249  9823 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:05.725297  9826 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:05.771768  9921 raft_consensus.cc:2955] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:05.790441  8677 raft_consensus.cc:2955] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
W20260709 13:02:05.798807  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 67383.4us
I20260709 13:02:05.808204  8679 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:05.811395  9878 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:02:05.816179  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 71484.9us
I20260709 13:02:05.842805  8826 raft_consensus.cc:2955] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:05.879561  8575 catalog_manager.cc:5697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:05.899972  8756 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
W20260709 13:02:05.951711  8722 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 51388ea026cc4d098949a4a0327a35fc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:05.955966  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:05.961404  8677 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:05.961398  8826 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:05.963205  9945 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:05.972541  9945 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:06.005357  9945 raft_consensus.cc:2955] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:06.006750  8679 raft_consensus.cc:2955] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:06.015921  8826 raft_consensus.cc:2955] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:06.031452  8828 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:06.032270  8575 catalog_manager.cc:5697] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:06.062738  8796 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: abd86f4e9b544531a7cbfa0042b4fa6c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:06.064960  8753 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:06.066676  9917 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:06.068210  8679 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:06.070096  9943 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:06.140256  9942 raft_consensus.cc:2955] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:06.146051  8753 raft_consensus.cc:2955] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:06.177596  9959 ts_tablet_manager.cc:933] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:06.179970  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:06.204332  9959 tablet_copy_client.cc:323] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.130:42321
I20260709 13:02:06.205101  8579 catalog_manager.cc:5697] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:06.212702  8766 tablet_copy_service.cc:140] P 0e9b0850145a4c66bfe983ffdfa93a57: Received BeginTabletCopySession request for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:35702)
I20260709 13:02:06.213419  8766 tablet_copy_service.cc:161] P 0e9b0850145a4c66bfe983ffdfa93a57: Beginning new tablet copy session on tablet 2147fd3c99a94f0c8e9c1a914bdb79c3 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:35702: session id = 7aff04f53d4845a6a4fa491fd2a06145-2147fd3c99a94f0c8e9c1a914bdb79c3
I20260709 13:02:06.215191  8679 raft_consensus.cc:2955] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:06.229321  8753 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:06.253120  8828 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:06.253393  8766 tablet_copy_source_session.cc:215] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:06.257030  8677 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:02:06.255822  9948 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:02:06.260895  9948 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:02:06.264568  8722 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 364ad2097a2341cdb96752b9a370f9a9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:06.277046  9959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:06.307742  8677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:06.308514  8677 raft_consensus.cc:606] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:06.310986  9946 raft_consensus.cc:2955] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:06.312183  8753 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:02:06.313493  8753 raft_consensus.cc:606] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Received request to transfer leadership to TS 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:06.314232  8677 raft_consensus.cc:2955] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:06.318210  8826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
mode: GRACEFUL
new_leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:58932
I20260709 13:02:06.319038  8826 raft_consensus.cc:606] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Received request to transfer leadership to TS 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:06.323719  8676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:06.324565  8676 raft_consensus.cc:606] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Received request to transfer leadership to TS 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:06.328101  8826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
mode: GRACEFUL
new_leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:58932
I20260709 13:02:06.328783  8826 raft_consensus.cc:606] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Received request to transfer leadership to TS 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:06.330135  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:06.332268  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:06.337293  9959 tablet_copy_client.cc:806] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:06.338184  9959 tablet_copy_client.cc:670] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:06.337076  8828 raft_consensus.cc:2955] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:06.346666  9959 tablet_copy_client.cc:538] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:06.347721  8579 catalog_manager.cc:5697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: config changed from index -1 to 2, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:06.360737  9959 tablet_bootstrap.cc:492] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:02:06.394727  9959 tablet_bootstrap.cc:492] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:06.395925  9959 tablet_bootstrap.cc:492] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:02:06.396778  9959 ts_tablet_manager.cc:1403] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.016s
I20260709 13:02:06.400457  9959 raft_consensus.cc:359] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:06.401471  9959 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:02:06.402217  9959 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:06.405303  9959 ts_tablet_manager.cc:1434] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:02:06.414961  8766 tablet_copy_service.cc:342] P 0e9b0850145a4c66bfe983ffdfa93a57: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-2147fd3c99a94f0c8e9c1a914bdb79c3 received from {username='slave'} at 127.0.0.1:35702
I20260709 13:02:06.415561  8766 tablet_copy_service.cc:434] P 0e9b0850145a4c66bfe983ffdfa93a57: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-2147fd3c99a94f0c8e9c1a914bdb79c3 on tablet 2147fd3c99a94f0c8e9c1a914bdb79c3 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:06.506595  9389 raft_consensus.cc:1217] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:06.548446  9959 ts_tablet_manager.cc:933] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:06.554306  9959 tablet_copy_client.cc:323] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.130:42321
I20260709 13:02:06.556437  8766 tablet_copy_service.cc:140] P 0e9b0850145a4c66bfe983ffdfa93a57: Received BeginTabletCopySession request for tablet 51388ea026cc4d098949a4a0327a35fc from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:35702)
I20260709 13:02:06.557076  8766 tablet_copy_service.cc:161] P 0e9b0850145a4c66bfe983ffdfa93a57: Beginning new tablet copy session on tablet 51388ea026cc4d098949a4a0327a35fc from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:35702: session id = 7aff04f53d4845a6a4fa491fd2a06145-51388ea026cc4d098949a4a0327a35fc
I20260709 13:02:06.566159  8766 tablet_copy_source_session.cc:215] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:06.567431  9966 ts_tablet_manager.cc:933] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:06.569974  9959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51388ea026cc4d098949a4a0327a35fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:06.576186  9966 tablet_copy_client.cc:323] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:02:06.578804  8840 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet abd86f4e9b544531a7cbfa0042b4fa6c from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
I20260709 13:02:06.579454  8840 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet abd86f4e9b544531a7cbfa0042b4fa6c from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-abd86f4e9b544531a7cbfa0042b4fa6c
I20260709 13:02:06.588667  8840 tablet_copy_source_session.cc:215] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:06.591301  9959 tablet_copy_client.cc:806] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:06.592049  9959 tablet_copy_client.cc:670] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:06.593078  9966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd86f4e9b544531a7cbfa0042b4fa6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:06.596565  9959 tablet_copy_client.cc:538] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:06.608662  9959 tablet_bootstrap.cc:492] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:02:06.623323  9966 tablet_copy_client.cc:806] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:06.626678  9966 tablet_copy_client.cc:670] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:06.631762  9966 tablet_copy_client.cc:538] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:06.636488  9959 tablet_bootstrap.cc:492] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:06.638399  9959 tablet_bootstrap.cc:492] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:02:06.639832  9959 ts_tablet_manager.cc:1403] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.031s	user 0.019s	sys 0.009s
I20260709 13:02:06.642540  9945 raft_consensus.cc:993] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: : Instructing follower 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:06.643169  9945 raft_consensus.cc:1081] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Signalling peer 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:06.643814  9959 raft_consensus.cc:359] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:06.646848  9966 tablet_bootstrap.cc:492] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:02:06.647534  9959 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:02:06.648082  9389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:34754
I20260709 13:02:06.648661  9389 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:06.649128  9389 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.648553  9959 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:06.651856  9959 ts_tablet_manager.cc:1434] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:02:06.653139  9937 raft_consensus.cc:993] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: : Instructing follower 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:06.653750  9940 raft_consensus.cc:1081] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Signalling peer 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:06.656774  9386 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:34746
I20260709 13:02:06.657444  9386 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:06.657858  9386 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.663247  8766 tablet_copy_service.cc:342] P 0e9b0850145a4c66bfe983ffdfa93a57: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-51388ea026cc4d098949a4a0327a35fc received from {username='slave'} at 127.0.0.1:35702
I20260709 13:02:06.663950  8766 tablet_copy_service.cc:434] P 0e9b0850145a4c66bfe983ffdfa93a57: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-51388ea026cc4d098949a4a0327a35fc on tablet 51388ea026cc4d098949a4a0327a35fc with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:06.706355  9389 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:06.708828  9389 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:06.709681  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:06.710048  9386 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:06.710654  8677 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.712162  9386 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:06.714026  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:06.714818  8679 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:06.715178  8679 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:06.719141  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:06.719871  8753 raft_consensus.cc:3055] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:06.720201  8753 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:06.720671  9966 tablet_bootstrap.cc:492] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:06.721110  8753 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:06.721812  9966 tablet_bootstrap.cc:492] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:02:06.722121  8679 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:06.722688  8753 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.722813  9966 ts_tablet_manager.cc:1403] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.076s	user 0.022s	sys 0.004s
I20260709 13:02:06.726960  8679 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.730074  9966 raft_consensus.cc:359] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:06.731124  9966 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:02:06.731886  9966 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:06.724140  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:06.725684  8677 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:06.733311  8756 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.737620  8679 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:06.740630  9966 ts_tablet_manager.cc:1434] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:02:06.743065  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:06.744380  9352 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:06.745751  9913 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:06.749840  8840 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-abd86f4e9b544531a7cbfa0042b4fa6c received from {username='slave'} at 127.0.0.1:55168
I20260709 13:02:06.750463  8840 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-abd86f4e9b544531a7cbfa0042b4fa6c on tablet abd86f4e9b544531a7cbfa0042b4fa6c with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:06.752873  8753 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:06.755920  8756 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:06.770888  9971 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:06.771986  9913 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:06.773164  9913 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:06.780917  9839 raft_consensus.cc:993] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: : Instructing follower 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:06.782006  9839 raft_consensus.cc:1081] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Signalling peer 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:06.783119  9971 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:06.784224  9971 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:06.788509  8753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:39032
I20260709 13:02:06.789439  8753 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:06.789990  8753 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.810572  9942 raft_consensus.cc:993] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: : Instructing follower 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:06.811369  9889 raft_consensus.cc:1081] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Signalling peer 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:06.813917  9386 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:34734
I20260709 13:02:06.814538  9386 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:06.814924  9386 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.825445  9966 ts_tablet_manager.cc:933] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:06.826865  8753 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:06.830557  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:06.831786  8676 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:06.832158  8676 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:06.833045  8676 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:06.834299  8676 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.836809  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:06.837598  8828 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.842769  8576 catalog_manager.cc:5697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 2, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 2 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:06.844811  8576 catalog_manager.cc:5697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 2, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 2 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:06.859313  9966 tablet_copy_client.cc:323] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.130:42321
I20260709 13:02:06.861197  8766 tablet_copy_service.cc:140] P 0e9b0850145a4c66bfe983ffdfa93a57: Received BeginTabletCopySession request for tablet 364ad2097a2341cdb96752b9a370f9a9 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:35702)
I20260709 13:02:06.861858  8766 tablet_copy_service.cc:161] P 0e9b0850145a4c66bfe983ffdfa93a57: Beginning new tablet copy session on tablet 364ad2097a2341cdb96752b9a370f9a9 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:35702: session id = 7aff04f53d4845a6a4fa491fd2a06145-364ad2097a2341cdb96752b9a370f9a9
I20260709 13:02:06.862421  9386 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:06.869619  9386 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:06.871387  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:06.872172  8826 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:06.872493  8826 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:06.873363  8826 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:06.874500  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:06.875214  8677 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.874562  8826 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.883973  9942 raft_consensus.cc:993] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: : Instructing follower 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:06.884644  9943 raft_consensus.cc:1081] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Signalling peer 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:06.830778  8753 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:06.891872  9389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:34734
I20260709 13:02:06.899554  9389 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:06.900094  9389 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.920845  8828 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:06.924139  9389 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:06.929032  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:06.931536  8828 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:06.931999  8828 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:06.935227  8826 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:06.935104  8828 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:06.937170  8828 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.937904  9389 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:06.950702  8828 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:06.953369  9355 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:06.954768  9972 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:06.955531  9972 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:06.956908  9972 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:07.066576  9389 raft_consensus.cc:1217] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:07.156976  9355 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:07.158342  9971 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:07.159442  9971 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:07.160883  9971 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:07.341481  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 183026us
I20260709 13:02:07.490164  9943 raft_consensus.cc:1064] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:02:07.499370  9943 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:07.547667  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.006s	sys 0.001s
I20260709 13:02:07.548321  9389 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:02:07.549943  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.007s	sys 0.001s
I20260709 13:02:07.550638  9889 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:07.799793  9971 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.045s	sys 0.044s
W20260709 13:02:07.803798  9971 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.045s	sys 0.044s
I20260709 13:02:07.811498  9984 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:07.812373  9984 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:07.814851  9984 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:07.843575  8680 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:07.845978  9971 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:07.854934  9972 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:07.855731  9972 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:07.858850  9972 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:06.883850  8677 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:07.863300  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:07.866673  9988 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:07.867473  8576 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40096 (request call id 135) took 1054 ms. Trace:
I20260709 13:02:07.867403  9988 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:07.869916  9988 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:07.339409  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:07.887701  8681 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:02:07.887885  8676 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.036s	user 0.013s	sys 0.010s
I20260709 13:02:07.889704  8676 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:07.890293  9971 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:07.890808  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1014) took 1060 ms. Trace:
I20260709 13:02:07.867954  8576 rpcz_store.cc:276] 0709 13:02:06.813212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.818937 (+  5725us) service_pool.cc:224] Handling call
0709 13:02:06.915224 (+ 96287us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:06.915262 (+    38us) write_op.cc:454] Released schema lock
0709 13:02:07.867399 (+952137us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:06.864597 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:06.865293 (+   696us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:06.865345 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:02:06.865360 (+    15us) tablet.cc:662] Decoding operations
0709 13:02:06.865916 (+   556us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:06.865980 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:06.866031 (+    51us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:02:06.866481 (+   450us) tablet.cc:701] Row locks acquired
0709 13:02:06.866501 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:02:06.866719 (+   218us) write_op.cc:270] Start()
0709 13:02:06.866851 (+   132us) write_op.cc:276] Timestamp: P: 1783602126866654 usec, L: 0
0709 13:02:06.866873 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:02:06.868570 (+  1697us) log.cc:844] Serialized 913 byte log entry
0709 13:02:06.870728 (+  2158us) op_driver.cc:464] REPLICATION: finished
0709 13:02:06.871213 (+   485us) write_op.cc:301] APPLY: starting
0709 13:02:06.871302 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:06.871449 (+   147us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:06.871566 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:06.871935 (+   369us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:06.872124 (+   189us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:02:06.872176 (+    52us) write_op.cc:312] APPLY: finished
0709 13:02:06.872384 (+   208us) log.cc:844] Serialized 29 byte log entry
0709 13:02:06.872564 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:06.872668 (+   104us) write_op.cc:454] Released schema lock
0709 13:02:06.872738 (+    70us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":2,"spinlock_wait_cycles":9088,"child_traces":[["op",{"apply.queue_time_us":289,"apply.run_cpu_time_us":1828,"apply.run_wall_time_us":1827,"num_ops":2,"prepare.queue_time_us":6324,"prepare.run_cpu_time_us":3491,"prepare.run_wall_time_us":4336,"replication_time_us":3665}]]}
I20260709 13:02:07.891496  8676 rpcz_store.cc:276] 0709 13:02:06.830085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.830225 (+   140us) service_pool.cc:224] Handling call
0709 13:02:07.890688 (+1060463us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:07.923314  9993 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:07.923987  9993 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:07.926493  9993 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:07.942653  8722 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:07.943670  9999 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:07.944259  9999 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:07.945181  9999 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:07.946990  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.958056 10001 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:07.959198 10001 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:07.961889 10001 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:07.962774 10001 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:07.963351 10001 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:07.965412  8766 tablet_copy_source_session.cc:215] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:07.966337 10001 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:07.967931  9917 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:07.968498  9917 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:07.969694 10001 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:07.971022 10001 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:07.973105  8766 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:35702 (request call id 43) took 1112 ms. Trace:
I20260709 13:02:07.974335  9999 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:07.974665  9917 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:07.974961  9999 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:07.975963  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:07.973637  8766 rpcz_store.cc:276] 0709 13:02:06.860887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.861052 (+   165us) service_pool.cc:224] Handling call
0709 13:02:07.972998 (+1111946us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:07.978164  9999 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:07.978989  9993 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:07.979621  9993 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:07.981887  9993 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:07.982308 10001 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:07.984125  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:07.989645  9993 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:07.990689  9993 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:07.995237  9993 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.000080  9986 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:08.000803  9986 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.007074  9992 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:08.007691  9991 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:08.008067  9993 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:08.007794  9992 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:08.008298  9991 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:08.004889  9986 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.011288  9992 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:08.016590  9984 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:08.017628  9984 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.020088  9991 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:08.020862  9988 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:08.021416  9988 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.032876  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.035557  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:39032 (request call id 1128) took 1247 ms. Trace:
I20260709 13:02:08.035671  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.036235  8753 rpcz_store.cc:276] 0709 13:02:06.788160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.788364 (+   204us) service_pool.cc:224] Handling call
0709 13:02:08.035450 (+1247086us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.039263  9991 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:08.039968  9991 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.040444  9990 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:08.041018  9990 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:08.043390  9999 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:08.043803 10002 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:08.044101  9999 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.044399 10002 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:08.046867  9389 raft_consensus.cc:1217] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:02:08.046501  9999 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.012012  9993 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.050693  9999 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:08.051467  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:08.052829  9993 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.049901 10002 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:08.056293  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 1.18123e+06us
I20260709 13:02:08.063206  9990 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:08.063875  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.064172  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.068493  9937 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:08.068632  9992 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.069334 10001 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.069106  9937 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.069245  9992 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.069938 10001 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.057865  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.072053 10001 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:08.072053  9937 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.042412  9991 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.073797  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.075278  9989 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.075784  9989 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.077584  8824 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:08.078158  9989 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.081064  9983 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:08.087507  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.088033  8756 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:08.088249  8752 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:02:08.088763  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.095098  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.095269  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:08.044742  9984 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.097702  9992 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.051314  9999 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:08.112975  9990 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.113706  9990 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:08.116772  9990 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:08.117180  9980 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:08.117825  9980 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:08.124153  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:08.128244  9980 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.130008  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:08.134888 10002 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:08.114147  8576 catalog_manager.cc:5697] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 2 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:08.135550 10002 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.136684  8576 catalog_manager.cc:5697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 2 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:08.138896 10002 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.138260  9966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 364ad2097a2341cdb96752b9a370f9a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.146171  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.149432  9989 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:08.150209  9989 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:08.152850  9999 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:08.157428  9889 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:08.160992  9917 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.191787  9889 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.191900  9917 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.194017  9889 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:08.161753  9958 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.162278  9942 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.163576  9943 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:08.165126  9988 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.169800 10002 raft_consensus.cc:1064] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:02:08.170717  8681 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:08.176012  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.177937  9987 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:08.179847  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.194397  9917 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:08.180194  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.182039  9984 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:08.227175  9942 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:08.228199  9984 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.230827  9984 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.182825  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:08.185312  8747 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:08.186208  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.231931  9943 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.186558  8751 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:08.188668  8754 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.191071  9989 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:08.234834  9958 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.237411  9958 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:08.238114  9999 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:08.240470  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:08.241505  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:08.242835  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:08.213651  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.249670  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.243943  9987 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:08.253993  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.255613  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.256402  8824 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:08.257673  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.234434  9943 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:08.267896 10003 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.268573  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.257922  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:08.230370  9942 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:08.279994  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.281911  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.282178  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.273638  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.275465  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.285200  8828 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:08.286994  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.280882  8679 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:02:08.290535  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.271346  8754 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:08.294941  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.297555  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.297127  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.301996  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.300590  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.301668 10004 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.298058 10003 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.305408  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.267000  9993 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:08.263223  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 157791us
I20260709 13:02:08.306737 10004 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.309667  9942 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.315268  9987 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:08.316529  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:08.316071 10002 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:08.312134  9942 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:08.320063  9942 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:08.243945  9999 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:08.322373  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:08.322144  9993 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:08.323356 10003 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.328339  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.327348 10004 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:08.329591 10011 log.cc:927] Time spent T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:02:08.331738  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.332122  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.333990  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.335897  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.340692  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.341112  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.342409  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.342864  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.344036  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.343122  9988 raft_consensus.cc:1064] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:02:08.351851  8673 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:02:08.345870  9993 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.351200  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.354174  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:08.354562  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.357054  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:08.357753  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.355647  9353 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:08.359115 10010 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:02:08.362903  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.363440  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:08.364167  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.368175  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.360596  9989 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.369750  9943 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:08.370922  9913 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.367439  9917 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.355612  9992 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.373147  9993 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:08.372483  9917 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:08.374696  9972 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:08.375133  9917 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:08.375945 10010 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:08.376838  8647 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:08.379884  9989 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:08.380367  9989 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.380848  9989 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:08.370844  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.380618  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.384735  9999 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:08.385603  9999 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:08.386158  9999 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.388067  9942 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:08.388429  9999 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.388772  9942 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:08.392009  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.395069  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.398159  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.358687 10004 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.402025  9942 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:08.408222  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:08.407372  9352 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:08.411924  9913 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.417323  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.418828  9989 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.420485 10010 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:08.422092  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 89674.6us
I20260709 13:02:08.354552  9988 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:08.423250  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.410151  9942 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.424543  9942 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.410336 10004 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:08.431278  9942 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:08.432111  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:02:08.433736  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.434698  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.436790 10004 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.438802  9989 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.476384  9383 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:08.421895 10010 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:08.487286  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 123929us
I20260709 13:02:08.484242  9999 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:02:08.488652  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:08.511641  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:08.512135  9943 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:08.513137  9943 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:08.513235  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:08.512678  9958 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.514081  9942 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.514297  9943 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.516692  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.520958  8645 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:08.524642  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:08.525053  9917 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:08.525786  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:08.527067  9991 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.525821 10003 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:08.528321 10003 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.528751 10003 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:08.529270  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.530764  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.531127  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.534556  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.535372  8824 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:08.535604  8676 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:08.536732  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.537353  8677 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:08.539908 10001 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.530818  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:08.540972  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.536814  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.541620  8824 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:08.542045  8828 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:08.540607 10001 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:08.543514  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.544054  9987 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:08.544219  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:08.544926  8826 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:08.525313  9989 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.549974 10003 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.528801  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.544373 10001 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:08.563788  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.522296  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:08.553051  8826 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:08.557746  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:08.565466 10004 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.566826  9991 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:08.567267  9991 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.567721  9991 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.567905  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:08.568594  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:08.554761  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:08.566443  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:08.571650  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:08.571854  9989 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:08.572333  9989 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.572726  9989 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.572912  9937 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:08.573343  9937 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.573720  9937 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.575237  8647 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:08.576422  9993 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:08.576943  9993 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:08.575392  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.570751  8828 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:08.569389  9992 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.582765  9989 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:08.585475  9989 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.585848  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:08.586794  9958 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.587301  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:08.564031 10003 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.588553  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:08.589337  9992 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:08.592000  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:08.592231  8826 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:08.592653  8828 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.593545  9992 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:08.589792  9943 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.590636  8645 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:08.578070  9993 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:08.596053 10016 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:02:08.692111  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:08.704135  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:08.706791  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 112073us
I20260709 13:02:08.596163  9992 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.696954  8826 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:08.728101  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:08.609488 10003 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.612273  9987 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:08.745455 10002 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:08.746826 10021 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.117s	user 0.004s	sys 0.003s
I20260709 13:02:08.751659  9990 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:08.752457  9990 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.715116 10005 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:02:08.767097  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:08.773125  9943 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:08.812448  9381 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:08.778076  8828 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:08.715117 10013 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.766896 10010 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:08.783676  9991 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.780053  9889 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.735181  8825 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:08.809295  9966 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:34754 (request call id 259) took 1984 ms. Trace:
I20260709 13:02:08.789659  9937 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:08.825457  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.827302 10013 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:08.821424 10005 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:02:08.832931  9966 rpcz_store.cc:276] 0709 13:02:06.824543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.824682 (+   139us) service_pool.cc:224] Handling call
0709 13:02:06.825918 (+  1236us) ts_tablet_manager.cc:934] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
0709 13:02:08.099995 (+1274077us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:02:08.419261 (+319266us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:08.419392 (+   131us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:02:08.806797 (+387405us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:02:08.807165 (+   368us) tablet_replica.cc:206] Creating consensus instance
0709 13:02:08.809132 (+  1967us) ts_tablet_manager.cc:989] Replica registered.
0709 13:02:08.809198 (+    66us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":257,"spinlock_wait_cycles":1920,"tablet-copy.queue_time_us":182,"thread_start_us":32466,"threads_started":1}
I20260709 13:02:08.836440  9966 tablet_copy_client.cc:806] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:08.837119  9966 tablet_copy_client.cc:670] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
W20260709 13:02:08.848433  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 156443us
W20260709 13:02:08.851956  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.858983 10013 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:08.862047  9966 tablet_copy_client.cc:538] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:08.871766  9991 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.875759  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:08.882679  8681 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:08.889521  8675 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:02:08.904757  9986 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:08.907177 10002 consensus_queue.cc:1059] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:08.907810  9986 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:08.908279  9986 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:08.908721 10002 raft_consensus.cc:3055] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:08.909198  9986 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:08.919312  8851 debug-util.cc:398] Leaking SignalData structure 0x7b08001bcbe0 after lost signal to thread 7529
W20260709 13:02:08.920674  8851 debug-util.cc:398] Leaking SignalData structure 0x7b08000ceec0 after lost signal to thread 8613
I20260709 13:02:08.909228 10002 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:02:08.922508  8851 debug-util.cc:398] Leaking SignalData structure 0x7b0800360dc0 after lost signal to thread 8705
W20260709 13:02:08.930012  8851 debug-util.cc:398] Leaking SignalData structure 0x7b08002d0440 after lost signal to thread 8780
W20260709 13:02:08.932010  8851 debug-util.cc:398] Leaking SignalData structure 0x7b08002c2de0 after lost signal to thread 8854
W20260709 13:02:08.932487  8851 debug-util.cc:398] Leaking SignalData structure 0x7b08002cd8c0 after lost signal to thread 8855
W20260709 13:02:08.934064  8851 debug-util.cc:398] Leaking SignalData structure 0x7b08001f85e0 after lost signal to thread 9413
I20260709 13:02:08.910552  9986 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:08.950879  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:08.952158 10010 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:08.958474  9937 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:08.973341  8673 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:08.975270 10029 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:08.979382  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:08.993157  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:08.934499  8851 debug-util.cc:398] Leaking SignalData structure 0x7b080004cfe0 after lost signal to thread 9414
W20260709 13:02:09.011905  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 283965us
W20260709 13:02:09.013629  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 443508us
I20260709 13:02:09.016022  8793 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:09.016081  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:09.028448  8793 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:09.029992 10031 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:02:09.032073  8676 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:08.979874 10002 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:09.043702  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:09.044658  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:09.013736  9966 tablet_bootstrap.cc:492] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:02:09.086400  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:09.092569 10030 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:09.056130  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:09.013696 10033 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:09.078858  8827 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.049535  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:09.056437  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:09.026357 10003 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:09.078941  8824 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.043984  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:09.007686  9992 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
W20260709 13:02:09.119494  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 73179.3us
I20260709 13:02:09.113207  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:09.047237  9917 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:09.121584  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 93232.7us
I20260709 13:02:09.136061  8679 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:09.134130  8824 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:09.136505  8793 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:09.125471 10039 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:02:09.141117 10037 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:09.126250 10002 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.119900 10030 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:09.145059  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:09.198766  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55068us
I20260709 13:02:09.139070 10033 raft_consensus.cc:697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:02:09.212878  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.225354  8676 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:09.271736  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 46841.8us
W20260709 13:02:09.331614  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.343166 10029 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:09.344242 10029 consensus_queue.cc:1243] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:09.344833 10029 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:09.345209 10029 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:09.346645 10029 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.348884 10029 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.349391  8827 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
W20260709 13:02:09.333308  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 188386us
W20260709 13:02:09.337245  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 123018us
W20260709 13:02:09.354077  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.305622  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 105709us
I20260709 13:02:09.268833 10044 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.357388  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:09.329830 10037 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:09.351229 10030 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.373432  8753 raft_consensus.cc:2955] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:09.381673  8793 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:09.367044 10044 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:09.379127  9999 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
W20260709 13:02:09.394735 10042 log.cc:927] Time spent T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:02:09.398406  9988 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:09.398967  9988 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:09.399891  9988 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.401021  9988 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.409538 10035 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:09.412518 10035 raft_consensus.cc:2764] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:02:09.389655 10044 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.435137  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:09.383379  9987 consensus_queue.cc:1059] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:02:09.439292  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:09.440816  8827 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:09.453358  8828 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:09.460315  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:09.456988  8825 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:09.477227  8747 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:09.406914 10033 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:09.442909  9992 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:09.490291  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.490545  8828 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:09.484679  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 117241us
W20260709 13:02:09.503337  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.506408  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:09.509894  8675 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:09.535655  8722 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.589s (SENT) [suppressed 2 similar messages]
I20260709 13:02:09.543390  8748 raft_consensus.cc:2410] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:02:09.553399  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:09.554255  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:09.561987  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 194054us
I20260709 13:02:09.565093  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:09.576130  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 207737us
W20260709 13:02:09.579053  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 203568us
W20260709 13:02:09.586654  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51031.2us
I20260709 13:02:09.590762  9999 raft_consensus.cc:3055] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:09.591279  9999 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:09.592191  9999 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:09.594570 10002 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:09.596372 10002 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.599015 10002 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:09.595599  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:09.595078  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:09.592790  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:09.608804  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.609961  9989 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:09.610359  9989 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:09.611824 10034 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:09.612381 10034 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:09.612426  9989 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.610934  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:09.614233  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:09.618077  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.619864  8828 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:09.621066  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:09.622056  9991 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:09.622874  9991 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:02:09.624074 10043 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.290s	user 0.003s	sys 0.000s
I20260709 13:02:09.624277  9991 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.625423 10030 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:09.626022 10030 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.628932  8753 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:09.628436 10030 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:09.632977  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:09.613444 10034 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.634553  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:09.637737  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:09.645658  8824 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:09.647930  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:09.605142 10044 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.649602  8827 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:09.650738  8824 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:09.655596  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.662294 10046 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:09.666858 10046 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.670302 10030 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:09.675343 10030 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.677094 10031 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:09.643801 10033 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:09.685456  8827 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:02:09.680008  9971 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:09.688407  9971 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:09.693122  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:09.690753 10033 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.697674 10008 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:09.680539 10058 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:09.679303 10004 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:09.677781 10031 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.679684 10003 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:09.707500 10004 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.709282  9999 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.700135 10058 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:09.712450  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:09.720129  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:09.720885 10031 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:09.724119  8827 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:58932
I20260709 13:02:09.724810  8827 raft_consensus.cc:606] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:09.715664  8679 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:09.731202  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:09.735116  8826 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:09.737610  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:09.739084  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:02:09.749310  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.751272 10061 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:09.755672  9971 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:09.755820 10048 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:09.757305  8722 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:09.714408 10044 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:09.698357 10008 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:09.766028  8722 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:09.714501 10003 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.756726 10048 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:09.765383  9986 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:02:09.764861 10029 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:09.784616  9987 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:09.756263  9383 raft_consensus.cc:2955] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:09.802767 10048 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:09.806921 10033 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:09.820556  8824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:58932
I20260709 13:02:09.822033  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:09.823149 10058 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:09.821321  8824 raft_consensus.cc:606] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:09.832175  8576 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40096 (request call id 136) took 1928 ms. Trace:
I20260709 13:02:09.814095 10004 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:09.823184  9989 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.832774  8576 rpcz_store.cc:276] 0709 13:02:07.902571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.917876 (+ 15305us) service_pool.cc:224] Handling call
0709 13:02:08.022918 (+105042us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b5400002490. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d50b2 00007f32e834ea89 00007f32e82e6da6 00007f32e82b8088 00007f32e83b637e 00007f32e68772a4 00007f32e6877232 00007f32e6877161 00007f32e68770dc
0709 13:02:09.831305 (+1808387us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:08.269020 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:08.324577 (+ 55557us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:08.326327 (+  1750us) write_op.cc:435] Acquired schema lock
0709 13:02:08.329364 (+  3037us) tablet.cc:662] Decoding operations
0709 13:02:08.346470 (+ 17106us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:08.346579 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:08.346631 (+    52us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:02:08.346989 (+   358us) tablet.cc:701] Row locks acquired
0709 13:02:08.347007 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:02:08.347181 (+   174us) write_op.cc:270] Start()
0709 13:02:08.347306 (+   125us) write_op.cc:276] Timestamp: P: 1783602128347128 usec, L: 0
0709 13:02:08.347329 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:02:08.359104 (+ 11775us) log.cc:844] Serialized 913 byte log entry
0709 13:02:09.093050 (+733946us) op_driver.cc:464] REPLICATION: finished
0709 13:02:09.407639 (+314589us) write_op.cc:301] APPLY: starting
0709 13:02:09.432808 (+ 25169us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:09.442100 (+  9292us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:09.481399 (+ 39299us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:09.521011 (+ 39612us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:09.540523 (+ 19512us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:02:09.542834 (+  2311us) write_op.cc:312] APPLY: finished
0709 13:02:09.558981 (+ 16147us) log.cc:844] Serialized 29 byte log entry
0709 13:02:09.559254 (+   273us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:09.559386 (+   132us) write_op.cc:454] Released schema lock
0709 13:02:09.559458 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:09.559963 (+   505us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:09.560005 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":85888,"reported_tablets":4,"spinlock_wait_cycles":7733120,"child_traces":[["op",{"apply.queue_time_us":298646,"apply.run_cpu_time_us":3769,"apply.run_wall_time_us":155151,"num_ops":2,"prepare.queue_time_us":97108,"prepare.run_cpu_time_us":7604,"prepare.run_wall_time_us":216919,"raft.queue_time_us":288581,"raft.run_cpu_time_us":59965,"raft.run_wall_time_us":240001,"replication_time_us":745542,"spinlock_wait_cycles":67840,"thread_start_us":620750,"threads_started":4,"wal-append.queue_time_us":375011}]]}
I20260709 13:02:09.854754  8675 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:09.860605 10008 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:09.874605  8575 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43462 (request call id 262) took 1730 ms. Trace:
I20260709 13:02:09.875365  8575 rpcz_store.cc:276] 0709 13:02:08.143884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.144091 (+   207us) service_pool.cc:224] Handling call
0709 13:02:09.874492 (+1730401us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":2,"spinlock_wait_cycles":35584}
I20260709 13:02:09.874025  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:09.878120  8679 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:09.879770 10045 raft_consensus.cc:993] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: : Instructing follower 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:09.880363 10045 raft_consensus.cc:1081] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Signalling peer 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:09.874502  9352 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:09.883018  9980 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.873924  9966 tablet_bootstrap.cc:492] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:09.884119  9980 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:09.879822  8579 catalog_manager.cc:5697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 1 to 2, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:09.884618  9980 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.884177 10038 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:02:09.886874  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:09.886888 10029 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:09.889091 10038 consensus_queue.cc:1243] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:09.890585  9913 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:09.887888 10023 raft_consensus.cc:2955] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:09.891074  9913 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:09.894011 10029 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:09.901206  9913 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.893695  9966 tablet_bootstrap.cc:492] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:02:09.910562 10029 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:09.914989  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:09.920096  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:09.921942  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:09.905617 10003 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:09.927099  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:09.929185  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:09.929371  8747 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:39024
I20260709 13:02:09.930630  8747 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:09.931053  8747 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.925686  9966 ts_tablet_manager.cc:1403] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.955s	user 0.040s	sys 0.018s
I20260709 13:02:09.948335  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:09.955780  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.962116  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:09.974766  8748 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:09.975205  8748 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:09.975324  8747 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:09.979617  8748 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:10.040688  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.040992  8748 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:10.042484  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 276595us
I20260709 13:02:10.043558  8722 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.043650  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:10.044870  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.047945  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:10.049784  8747 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:10.051092  8722 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.052613  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.020756  8675 raft_consensus.cc:606] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:10.062526  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:10.062814  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:10.065900  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:10.069977  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:10.061275  8675 raft_consensus.cc:614] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:10.073590 10032 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:10.049412 10038 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.041924 10033 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:10.052069  9986 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.055192  9987 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.090883  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.094858  8722 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.086575 10031 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:10.098091  9913 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:10.098397 10031 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:10.099152 10030 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:10.102142  8748 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:10.085932 10033 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:10.106020  9987 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:10.108111 10033 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:10.109915  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:10.116775 10061 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:02:10.106103 10052 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:10.103937  9986 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.101853  9988 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.118026 10052 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:10.126065  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:10.109381  9999 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.074266 10032 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:10.137115  9966 raft_consensus.cc:359] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:10.139844  9984 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:10.119719 10029 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.134018  8722 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.158933  9913 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:10.161031  8754 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:10.148689  9986 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:02:10.161903  9988 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.164130 10046 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:10.166819 10031 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:10.170311  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:10.171554  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:10.138566  9966 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:02:10.175735  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:10.177402  8826 raft_consensus.cc:3055] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:10.177784  8826 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:10.178755  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:10.179739  8825 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 2: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:10.180192  8679 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:10.181058  8826 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:10.182026 10052 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:10.182410  8826 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.185586  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:10.186654  8748 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:10.190593 10032 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:10.207875  9966 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:10.210361  9966 ts_tablet_manager.cc:1434] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.267s	user 0.010s	sys 0.000s
I20260709 13:02:10.211985  9988 raft_consensus.cc:3055] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:10.212630  9988 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:10.213342  8766 tablet_copy_service.cc:342] P 0e9b0850145a4c66bfe983ffdfa93a57: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-364ad2097a2341cdb96752b9a370f9a9 received from {username='slave'} at 127.0.0.1:35702
I20260709 13:02:10.213893  8766 tablet_copy_service.cc:434] P 0e9b0850145a4c66bfe983ffdfa93a57: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-364ad2097a2341cdb96752b9a370f9a9 on tablet 364ad2097a2341cdb96752b9a370f9a9 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:10.221976  8826 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
W20260709 13:02:10.223837 10062 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.251s	user 0.005s	sys 0.000s
I20260709 13:02:10.226290  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1053) took 1246 ms. Trace:
I20260709 13:02:10.213670  9988 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:10.226904  8673 rpcz_store.cc:276] 0709 13:02:08.979759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.980067 (+   308us) service_pool.cc:224] Handling call
0709 13:02:09.010683 (+ 30616us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:09.085183 (+ 74500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.087525 (+  2342us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:09.571849 (+484324us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b2c000014c8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e0df2a11 00007f32e0e02372 00007f32e2534a01 00007f32e24f943a 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2
0709 13:02:09.593162 (+ 21313us) log.cc:844] Serialized 556 byte log entry
0709 13:02:09.835272 (+242110us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:09.835323 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.835423 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:10.218567 (+383144us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:10.225964 (+  7397us) raft_consensus.cc:1685] finished
0709 13:02:10.226008 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:10.226172 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12577280,"thread_start_us":219943,"threads_started":1,"wal-append.queue_time_us":365638}
I20260709 13:02:10.252219  9980 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:10.252771  9988 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.268043  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.275463  8722 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.277290  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:10.279405 10034 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.004s	sys 0.028s
W20260709 13:02:10.315471 10034 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.004s	sys 0.029s
I20260709 13:02:10.319232  9389 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:10.283007  8719 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.272423 10029 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.268836  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:10.310968  9986 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:10.369966  8645 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:10.371068  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:10.372658  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:10.380010  8674 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.287818  9999 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.386739  8829 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
mode: GRACEFUL
new_leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:58932
I20260709 13:02:10.388391  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:10.391599  8829 raft_consensus.cc:606] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Received request to transfer leadership to TS 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:10.375485  9384 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:10.399679  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:10.405761  8674 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:10.331979  9980 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:10.287532  9993 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.105s
I20260709 13:02:10.399884  9381 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.456113  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:10.401063  9991 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:10.408077 10005 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:10.406411  9986 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:10.460041  8829 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
mode: GRACEFUL
new_leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:58932
I20260709 13:02:10.468062  8829 raft_consensus.cc:606] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Received request to transfer leadership to TS 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:10.468627  8829 raft_consensus.cc:614] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:10.448313  9352 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:10.475382  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:10.478154  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:10.479619  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:10.398537 10034 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:02:10.451818 10066 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:10.483570  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:10.486358  8579 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.011s	sys 0.025s
W20260709 13:02:10.488592  8579 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.012s	sys 0.025s
I20260709 13:02:10.461808 10005 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:10.471411  9991 raft_consensus.cc:697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:10.487205  9913 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:10.491321 10066 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:10.496644 10005 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.550064  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:10.533355  8679 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:10.564597  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:10.566965  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:10.578068  8793 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:10.514521  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 413912us
I20260709 13:02:10.541376 10013 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:10.547458  9913 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:10.584700  8680 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:10.604507  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:10.583786  9993 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:10.591477 10036 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.571381  9986 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:10.600350  9889 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:10.615757 10049 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:10.661149  8647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.035s	sys 0.133s
I20260709 13:02:10.597136 10013 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:10.639240 10076 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.058s	user 0.001s	sys 0.001s
I20260709 13:02:10.633694  9991 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:10.656658 10066 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:10.625259  9993 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:10.647001 10069 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 13:02:10.661621  8647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.035s	sys 0.133s
W20260709 13:02:10.664639  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 352062us
I20260709 13:02:10.675079  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:10.660286  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 44718.7us
I20260709 13:02:10.680008  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.689424  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:10.703765  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 420845us
I20260709 13:02:10.703891  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.683285 10068 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:10.663193  9889 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:10.708015 10074 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.707067 10046 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.719726 10005 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:10.682303  9937 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.758670 10005 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:10.716074 10032 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.768357  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:10.765578  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 106598us
I20260709 13:02:10.670285  9913 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:10.766911  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51437.4us
W20260709 13:02:10.753597  9988 consensus_meta.cc:287] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.500s	user 0.008s	sys 0.001s
I20260709 13:02:10.704207 10073 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
W20260709 13:02:10.775032  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 164555us
I20260709 13:02:10.775324  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.779347  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:10.780963  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.782867  8824 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 2: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:10.785293  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:10.786842  8824 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 2: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:10.733799 10068 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:10.759696 10049 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
W20260709 13:02:10.802431  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.810935  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:10.811794 10061 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:10.812318  9355 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:10.812389 10061 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:10.814111  8748 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 3. (term mismatch)
I20260709 13:02:10.814530  8748 pending_rounds.cc:85] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 2
I20260709 13:02:10.814796  8748 pending_rounds.cc:107] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 13:02:10.815107  8748 raft_consensus.cc:2917] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 13:02:10.793459  9937 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.809530 10068 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:10.810803 10036 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:10.806378 10013 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:10.873226  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:10.816149  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:10.797609  9381 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:10.878316  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:10.799235 10032 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:10.869287  8748 raft_consensus.cc:2889] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:10.870855  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:10.875773  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 70593.9us
I20260709 13:02:10.892198  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:10.808183  9889 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:10.814235  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:10.874192  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 55388us
W20260709 13:02:10.900763  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 88544.7us
W20260709 13:02:10.902678  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 100542us
I20260709 13:02:10.902050  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:10.874421 10078 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:10.937366  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:10.939455 10084 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:10.814945 10082 raft_consensus.cc:2955] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:10.915768  8755 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:02:10.977927  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 85862.1us
I20260709 13:02:11.000658 10083 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:10.970039  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:10.979776  9889 raft_consensus.cc:3055] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:10.974961 10029 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:11.021111  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:10.979784 10084 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:11.046797  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:10.998883 10066 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:02:11.014858  9889 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:02:11.017422 10081 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
W20260709 13:02:11.067495 10091 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:02:11.031373 10002 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.049788 10096 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:11.053817 10010 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:11.093596 10045 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:11.094333 10045 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:11.096403  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 77888.4us
I20260709 13:02:11.096556 10045 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:11.056730  9993 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:11.070509 10090 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:02:11.065835 10061 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:11.096472 10013 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.101508 10010 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:11.108901 10013 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:11.109308  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 44480.4us
I20260709 13:02:11.110814 10010 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:11.111198 10013 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:11.053272 10029 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:11.112558  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 92513.9us
I20260709 13:02:11.113654  8793 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:11.050364 10032 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:02:11.099666 10002 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:11.114197 10013 raft_consensus.cc:993] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: : Instructing follower 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:02:11.066587 10084 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:11.117857  8793 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:11.121025  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:11.094022 10092 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:11.126492 10089 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.119400  9889 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:11.127188 10097 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.102756  9988 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:11.121996 10096 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:11.102090 10023 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:02:11.114097 10029 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:11.137025  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:11.157914  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:11.141094 10013 raft_consensus.cc:1081] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Signalling peer 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:02:11.141564 10045 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.166074 10071 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:11.174357  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.174700  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.177708  8793 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:11.183504 10095 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:11.186677 10053 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.187706  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:11.188763  8675 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 2: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:11.180413 10008 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:11.193595  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:11.193979 10099 log.cc:927] Time spent T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:02:11.191874 10008 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:11.198325  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.173229  9889 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:11.147785  9988 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:11.209410  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.152951 10089 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:11.202373 10102 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:11.228164  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.235121  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.244673  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:11.160899 10092 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:11.252768  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.246301  8680 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:53342
I20260709 13:02:11.210664 10008 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:11.206347 10104 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:11.221231 10097 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.269798  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:11.270644  8824 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:11.265239  8680 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:11.216295  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.214082  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:11.203646 10071 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:11.279495  8680 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:11.262217 10029 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.280020  8825 raft_consensus.cc:2955] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:11.283401 10092 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:11.266507 10053 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:11.308861  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:11.278095 10102 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:11.297598  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.332139 10102 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:11.309763 10002 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:11.341919 10048 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:11.342458 10048 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:11.343472 10048 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:11.325675 10098 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:11.353044  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 255553us
I20260709 13:02:11.344966 10048 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:11.317689 10096 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:11.280378 10104 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:11.288585 10097 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:11.346033 10098 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:11.326505 10053 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:11.377331 10053 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:11.383901 10104 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:11.384832  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:11.386842  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:11.388471  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:11.391898 10089 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:11.394239  8680 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:11.396945  8680 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:11.378264  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:11.400055 10097 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:11.372292  9988 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:11.406894 10071 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:11.412189  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:11.408185  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:11.414494  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:11.430559  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:11.434332  8829 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:11.436118  8826 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:11.441238 10078 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.448276  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:11.448340  8824 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:11.451473  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:11.453684  8647 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:11.457150 10051 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.461162  9382 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 3. (term mismatch)
I20260709 13:02:11.461663  9382 pending_rounds.cc:85] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting all ops after (but not including) 2
I20260709 13:02:11.461898  9382 pending_rounds.cc:107] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 13:02:11.462180  9382 raft_consensus.cc:2917] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 13:02:11.462677  9382 raft_consensus.cc:2889] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:11.464742  9355 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:11.484412  9971 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.457465 10098 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:11.486799 10005 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:11.487666  8828 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:11.487437 10005 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:11.492097  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:11.490461 10005 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:11.523633  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:11.524873  8827 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:11.525327  8827 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:11.527467  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:11.526497  8827 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:11.531697  8827 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:11.515811  9971 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:02:11.527724 10051 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:02:11.553376  8829 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:11.554258  8579 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43478 (request call id 252) took 2585 ms. Trace:
I20260709 13:02:11.555389  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:11.560675  9352 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:11.591873  8827 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:11.586458  8575 catalog_manager.cc:5697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 2, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 2 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:11.593170  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.593786  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:11.596321  8575 catalog_manager.cc:5697] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 2, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 2 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:11.598855  8575 catalog_manager.cc:5697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 2 to 3, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 3 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:11.601650  8575 catalog_manager.cc:5697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 2 to 3, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 3 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:11.604077  8575 catalog_manager.cc:5697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 2, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 2 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:11.614504  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.618394  8677 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:11.619287  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:11.575978  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:11.586364  8579 rpcz_store.cc:276] 0709 13:02:08.969017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.969172 (+   155us) service_pool.cc:224] Handling call
0709 13:02:11.554139 (+2584967us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:10.554697 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:10.555558 (+   861us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:10.555612 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:02:10.555632 (+    20us) tablet.cc:662] Decoding operations
0709 13:02:10.556135 (+   503us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:10.556261 (+   126us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:10.556432 (+   171us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:02:10.556906 (+   474us) tablet.cc:701] Row locks acquired
0709 13:02:10.556940 (+    34us) write_op.cc:260] PREPARE: finished
0709 13:02:10.557123 (+   183us) write_op.cc:270] Start()
0709 13:02:10.557263 (+   140us) write_op.cc:276] Timestamp: P: 1783602130557054 usec, L: 0
0709 13:02:10.557290 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:02:10.558240 (+   950us) log.cc:844] Serialized 538 byte log entry
0709 13:02:10.906906 (+348666us) op_driver.cc:464] REPLICATION: finished
0709 13:02:11.145400 (+238494us) write_op.cc:301] APPLY: starting
0709 13:02:11.145536 (+   136us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:11.145651 (+   115us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:11.145833 (+   182us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:11.155968 (+ 10135us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:11.184810 (+ 28842us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:02:11.202095 (+ 17285us) write_op.cc:312] APPLY: finished
0709 13:02:11.212765 (+ 10670us) log.cc:844] Serialized 23 byte log entry
0709 13:02:11.224398 (+ 11633us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:11.242060 (+ 17662us) write_op.cc:454] Released schema lock
0709 13:02:11.246889 (+  4829us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:11.287619 (+ 40730us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:11.290024 (+  2405us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6268,"reported_tablets":5,"spinlock_wait_cycles":1908992,"child_traces":[["op",{"apply.queue_time_us":238024,"apply.run_cpu_time_us":3015,"apply.run_wall_time_us":131704,"num_ops":1,"prepare.queue_time_us":661214,"prepare.run_cpu_time_us":4113,"prepare.run_wall_time_us":39645,"replication_time_us":349414,"thread_start_us":790698,"threads_started":2}]]}
I20260709 13:02:11.557436 10054 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:11.627795 10054 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:11.575788  9355 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:11.629904  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:11.558393  8825 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:02:11.630375  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 54741.7us
I20260709 13:02:11.631419 10071 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.653633  8750 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:11.633503  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:11.672744  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:11.632402  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:11.676962  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:11.669929  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.669160  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:11.675853  8680 raft_consensus.cc:2955] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:11.680122  8793 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:11.672197  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:11.682968  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:11.684394  8647 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:11.667241  9987 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.685490  9987 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:11.684973  9385 raft_consensus.cc:2955] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:11.617877  9971 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.690966 10084 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:11.692782  9384 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:02:11.693048  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 70532.4us
I20260709 13:02:11.699285  8751 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:11.630422  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:11.683066 10104 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:11.735821  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51520.4us
I20260709 13:02:11.739610  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:11.740993 10085 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.750320  9352 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:11.751549  8794 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:11.755427  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:11.759644  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:11.731163 10109 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:11.774732 10093 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:11.775484 10093 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:11.776005 10093 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:11.782393 10103 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.738260 10010 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
W20260709 13:02:11.784289  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46758.2us
I20260709 13:02:11.750949 10084 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.781648 10072 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.788905  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.764178 10078 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.736340 10061 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:02:11.801633  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.806213  8647 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:11.788973 10092 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.824205  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:11.823369  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:11.797966 10097 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.854012 10002 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:11.864043 10104 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:11.864554 10002 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:11.847149  9990 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.854401 10092 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:11.888284 10002 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:11.805640 10103 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:11.886276 10092 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:11.865448 10098 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.949033  8677 raft_consensus.cc:2955] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:11.955334  9987 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:11.977485  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:11.944662  9989 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:12.042649 10054 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:12.037312  9989 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:12.072845  9989 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:12.142836 10098 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:12.143548 10098 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:12.150110 10034 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:12.150990 10034 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:12.153723 10034 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:12.166395 10093 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:12.243695 10093 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:12.256002  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 98758.4us
W20260709 13:02:12.260710  8575 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.008s	sys 0.004s
W20260709 13:02:12.261186  8575 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.008s	sys 0.004s
I20260709 13:02:12.265662  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:12.257764  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:12.304852  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:12.306219  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:12.330251  8795 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.003s	sys 0.012s
I20260709 13:02:12.298381 10010 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:12.314841 10102 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.005s	sys 0.009s
W20260709 13:02:12.314172  8823 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.004s	sys 0.012s
I20260709 13:02:12.349164 10010 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:12.342336  8795 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.003s	sys 0.012s
W20260709 13:02:12.352437  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.009s	sys 0.009s
W20260709 13:02:12.382781  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.010s	sys 0.009s
W20260709 13:02:12.383250  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 533299us
I20260709 13:02:12.384690  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:12.358289 10010 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:12.355221  8823 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.004s	sys 0.012s
W20260709 13:02:12.348400 10102 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.005s	sys 0.009s
W20260709 13:02:12.393394  8681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 1568) took 1938 ms (client timeout 1617 ms). Trace:
I20260709 13:02:12.354542 10098 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:12.399467  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:12.422825  8794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.008s	sys 0.041s
W20260709 13:02:12.437057  8794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.011s	sys 0.041s
W20260709 13:02:12.436486  8647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.072s	sys 0.090s
W20260709 13:02:12.437563  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 639030us
W20260709 13:02:12.438563  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 53989.5us
W20260709 13:02:12.398077  8681 rpcz_store.cc:269] 0709 13:02:10.455107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.455733 (+   626us) service_pool.cc:224] Handling call
0709 13:02:12.393100 (+1937367us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54002576d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:12.393316 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3089193216}
W20260709 13:02:12.441418  8647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.072s	sys 0.090s
W20260709 13:02:12.425089  9381 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.014s	sys 0.037s
W20260709 13:02:12.442327  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 619084us
I20260709 13:02:12.443760  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:12.443563  8793 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.014s	sys 0.017s
I20260709 13:02:12.387861 10066 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:12.387621 10103 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:12.453191  8793 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.015s	sys 0.017s
W20260709 13:02:12.465916  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 686996us
I20260709 13:02:12.458514  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:12.423933 10010 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:12.479430 10061 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:12.479136 10010 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:12.480044 10061 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:12.481772 10010 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:12.482267 10129 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.073s	user 0.005s	sys 0.002s
W20260709 13:02:12.484251 10124 log.cc:927] Time spent T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:02:12.487061  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1191) took 1039 ms. Trace:
I20260709 13:02:12.487552  8826 rpcz_store.cc:276] 0709 13:02:11.447496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.447919 (+   423us) service_pool.cc:224] Handling call
0709 13:02:11.453638 (+  5719us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.454034 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.454066 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.454749 (+   683us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.790354 (+335605us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:11.790396 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.790555 (+   159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.486727 (+696172us) raft_consensus.cc:1685] finished
0709 13:02:12.486756 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.486973 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7244,"raft.queue_time_us":1514,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":2974,"thread_start_us":335250,"threads_started":1,"wal-append.queue_time_us":807941}
I20260709 13:02:12.397883 10096 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:12.451514  9381 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.015s	sys 0.038s
I20260709 13:02:12.497566  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:12.441767 10134 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:12.503651  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:12.493047 10074 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:12.516271  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:12.503091 10127 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.152s	user 0.001s	sys 0.001s
I20260709 13:02:12.479157 10093 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:12.480037  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:12.514482  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:12.482475 10061 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:12.541538  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:12.507180 10075 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.009s	sys 0.019s
I20260709 13:02:12.544792  8824 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:12.543542  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:12.542438  8677 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:12.549094  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:12.556640  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:12.515666 10098 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.563378  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:12.515197 10029 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:12.566376  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:12.572439  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:12.519488 10005 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:02:12.545296 10075 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.010s	sys 0.020s
I20260709 13:02:12.611267 10094 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.621567  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:12.622226 10083 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:12.508126 10096 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:12.650218  8830 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
W20260709 13:02:12.648691 10136 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:02:12.522882 10074 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:12.515496 10134 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:12.566066  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:12.566762 10061 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:02:12.688328 10138 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:02:12.688582  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:12.545595 10093 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:12.731283  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 272945us
I20260709 13:02:12.731621  9986 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.611801 10029 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:12.670534 10135 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:12.670217 10005 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:12.682550 10103 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:12.796151 10148 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:12.838052  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:12.795868 10132 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.828667  9988 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:12.678848 10083 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:12.744880 10096 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:12.840197 10142 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:02:12.839040  9986 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:12.841768 10083 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:12.841069  9986 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:12.843804 10029 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:12.692526  8822 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:12.798470  8677 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:12.772583 10075 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
I20260709 13:02:12.837844 10148 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:12.677717  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 93) took 1058 ms. Trace:
I20260709 13:02:12.781610  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:12.886391 10146 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:02:12.892203  9383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 261) took 1440 ms. Trace:
I20260709 13:02:12.720675  9987 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:02:12.894793  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 338313us
I20260709 13:02:12.839140  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 1603) took 1053 ms. Trace:
W20260709 13:02:12.802399 10145 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:02:12.910893  9987 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:12.899319 10092 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.839151 10132 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:12.912779 10092 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:12.914463  9987 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:12.782658 10152 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:12.823979 10074 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:12.766412 10094 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:12.825467 10134 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:12.813486 10151 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:12.839278 10005 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:12.737000 10048 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:12.934229 10048 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:12.906973  8675 rpcz_store.cc:276] 0709 13:02:11.785624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.788650 (+  3026us) service_pool.cc:224] Handling call
0709 13:02:12.837893 (+1049243us) spinlock_profiling.cc:243] Waited 5.62 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:12.839054 (+  1161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12928896}
W20260709 13:02:12.900270  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 118848us
I20260709 13:02:12.914710 10132 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:12.917716  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:12.860002  9986 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:12.903838  9383 rpcz_store.cc:276] 0709 13:02:11.451603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.452746 (+  1143us) service_pool.cc:224] Handling call
0709 13:02:11.566996 (+114250us) spinlock_profiling.cc:243] Waited 7.69 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:11.567260 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.685226 (+117966us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.685259 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:11.685320 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:11.685353 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.891722 (+1206369us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b540061c1d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:12.891807 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.892066 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288541312}
I20260709 13:02:12.919027  9355 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:12.900519 10103 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:12.900254 10135 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:12.763212 10098 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.144s
I20260709 13:02:12.923560 10153 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:02:12.735873 10141 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.182s	user 0.001s	sys 0.001s
I20260709 13:02:12.901476 10083 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:12.761487 10066 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.157s
I20260709 13:02:12.911057  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1605) took 1123 ms. Trace:
I20260709 13:02:12.969095 10157 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:12.898473  8823 rpcz_store.cc:276] 0709 13:02:11.591214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.625272 (+ 34058us) service_pool.cc:224] Handling call
0709 13:02:11.694211 (+ 68939us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.766959 (+ 72748us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.771482 (+  4523us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.781288 (+  9806us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.418254 (+636966us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:12.419688 (+  1434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.423970 (+  4282us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.481650 (+ 57680us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:12.552962 (+ 71312us) raft_consensus.cc:1685] finished
0709 13:02:12.563347 (+ 10385us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.628056 (+ 64709us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35232,"raft.queue_time_us":26494,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":15785,"spinlock_wait_cycles":15378176,"thread_start_us":612740,"threads_started":1,"wal-append.queue_time_us":627238}
I20260709 13:02:12.932310 10085 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.798509 10093 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:12.937577  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 178250us
I20260709 13:02:12.937703  8794 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:12.899746  9988 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:12.743462 10061 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:12.941664  8719 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:12.972180 10157 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:12.972795 10157 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:12.973281 10157 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:12.973665  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:12.941771  8722 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:12.941264 10029 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:12.928619 10094 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:12.948400  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:12.951337 10131 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:12.952006 10122 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.135s	user 0.001s	sys 0.015s
I20260709 13:02:12.953176 10048 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:12.940311 10109 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:12.953766 10153 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:12.955096  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:12.956431  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:12.957319  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:12.978900  8830 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:12.977792  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:12.980221  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:12.982472 10029 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:12.968773  8672 rpcz_store.cc:276] 0709 13:02:11.787382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.788698 (+  1316us) service_pool.cc:224] Handling call
0709 13:02:12.910089 (+1121391us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4e9c 00007f32df3d4a5b 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:12.910347 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.910588 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.910640 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.910691 (+    51us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:12.910744 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.910815 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.910951 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39769344}
I20260709 13:02:12.984380  9984 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:12.986719 10002 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:12.986858 10061 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:12.985827 10103 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:12.987352 10109 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.030684  8679 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
W20260709 13:02:13.033844  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 53694.5us
W20260709 13:02:13.032407  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 54732.3us
I20260709 13:02:13.035156  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:13.038503  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:13.038669  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:12.982892 10083 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:12.981575 10096 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:12.986308  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:13.045156  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:13.030882 10156 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:13.048454  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:13.051301  8722 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:13.051990 10159 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:13.052040  9355 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:13.052328 10135 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:13.052582 10161 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:13.053119 10074 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:13.054351  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1599) took 1270 ms. Trace:
I20260709 13:02:13.041725 10083 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.057336  8830 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 2: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:12.989609  9986 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:13.039887  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:13.036970 10112 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:13.062238  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:13.078425  8747 raft_consensus.cc:2410] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:13.063069  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:13.063079  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:13.033145 10132 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:12.992286 10048 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:12.936939 10151 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:12.982349 10085 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.063973 10093 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:13.042333 10066 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:13.040120 10131 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:13.055342 10109 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.062302  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:13.058251 10061 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:13.035126  9984 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.076706 10083 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:13.064059 10029 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.065783 10103 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:13.095428  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:13.075018  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:13.037412 10002 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:13.050345 10096 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.095695  8747 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:13.067318  8678 rpcz_store.cc:276] 0709 13:02:11.784420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.786001 (+  1581us) service_pool.cc:224] Handling call
0709 13:02:13.053622 (+1267621us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4e9c 00007f32df3d4a5b 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:13.053826 (+   204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.054035 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.054063 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.054108 (+    45us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:13.054137 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.054193 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.054290 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3991040}
I20260709 13:02:13.072865 10054 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:13.431816 10013 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:13.125864 10048 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.433319 10005 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:13.433468 10013 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.238654 10164 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:13.167253 10135 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:13.431114 10008 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:13.448571 10164 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:13.449316 10164 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:13.449117 10135 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:13.258013 10130 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.838s	user 0.005s	sys 0.014s
I20260709 13:02:13.089205 10152 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:13.135219 10159 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:13.080366  9986 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.096261 10112 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.104336  9999 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:13.468237  9999 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.470000 10112 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:13.471132 10148 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.472044  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:13.472502 10093 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:13.473122  8830 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:13.473548  9984 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.473472 10093 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.395690  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1233) took 1534 ms. Trace:
I20260709 13:02:13.455214  9937 raft_consensus.cc:1064] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:02:13.432057 10054 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:13.136273  9988 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:13.233330 10143 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.550s	user 0.000s	sys 0.001s
I20260709 13:02:13.456108 10082 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:13.456035 10103 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.405583  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:13.352725  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:13.457645 10089 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.078554 10156 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.496094  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:13.461752 10092 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:13.461642 10113 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:13.424418  9383 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 LEARNER]: Advancing to term 2
I20260709 13:02:13.502066 10161 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:13.502439  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:13.405046  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:13.427690  9971 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:13.477113 10150 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:13.402187  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:13.417384  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:13.492911 10134 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.493269  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:13.479187 10159 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.495602 10096 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:13.092700  9987 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:13.478479 10054 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:13.239038 10151 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.449322 10008 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.435968 10005 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.503063 10161 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.510023 10094 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:13.510695 10061 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.475379  8750 rpcz_store.cc:276] 0709 13:02:11.869740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.870015 (+   275us) service_pool.cc:224] Handling call
0709 13:02:13.393243 (+1523228us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b5800083198. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:13.393960 (+   717us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.394665 (+   705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.394712 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.394824 (+   112us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:13.394876 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.395022 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.395302 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3582848}
I20260709 13:02:13.516750  9971 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.385545  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:13.241181 10153 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:13.474480  9984 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:13.520712 10134 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.224171 10154 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.532047  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:13.510053 10150 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.537580 10113 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.544916 10089 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.547492  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:13.531697  9999 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.462846 10152 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.224444 10162 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.472s
I20260709 13:02:13.534283 10061 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:13.544726  9971 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.085229 10132 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.544762 10113 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:13.270846 10157 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.594313  8748 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:13.573953  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:13.542426 10131 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.600299 10170 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:13.579965 10112 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:13.604252  9988 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:13.633615  9987 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.653214  9988 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.656322  9987 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:13.656548  9988 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.591144 10094 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:13.657519 10170 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.605304 10134 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.575213 10082 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.675572  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1606) took 1817 ms. Trace:
I20260709 13:02:13.676105  8674 rpcz_store.cc:276] 0709 13:02:11.787694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.856558 (+ 68864us) service_pool.cc:224] Handling call
0709 13:02:11.940822 (+ 84264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.368779 (+1427957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.368911 (+   132us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.369054 (+   143us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:13.369137 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.591579 (+222442us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54003f43d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:13.591639 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.605457 (+ 13818us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3282193664}
I20260709 13:02:13.677510  8747 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:13.588888  9984 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.683072 10085 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.683759 10150 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.665848 10089 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:13.674115  9382 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:13.686769 10115 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:13.687578  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 85195us
I20260709 13:02:13.691692  9990 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:13.715044  9990 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.717836  9990 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:13.692440  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 287932us
I20260709 13:02:13.700243 10066 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.702051  9986 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:13.703056 10135 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:13.703644  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 147328us
I20260709 13:02:13.704084 10048 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:13.704704 10132 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:13.651024 10148 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:13.707336  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 125205us
I20260709 13:02:13.749788 10148 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.752671 10148 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.718042 10008 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.720484 10103 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:13.632727 10154 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.741829 10013 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:13.724339  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 104166us
I20260709 13:02:13.758968 10066 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.762223 10066 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.763785  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:13.763964  9937 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.768600 10002 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.773187  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:13.775233  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:13.777159  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:13.777987 10170 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.779884  9984 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:13.780769  9353 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:13.789168  9991 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:13.790016  9991 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.792552 10074 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:13.792943 10098 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:13.793377 10074 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.793746 10098 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.794016 10094 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:13.796363 10098 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:13.799326 10094 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:13.799134  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:13.800379  8676 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:13.796072 10074 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:13.756955 10093 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.757618  8754 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:13.806836  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:13.819958  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:13.821225  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:13.792603  9991 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:13.827013  9991 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:13.827791  9991 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.830636  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:13.831413  9386 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
W20260709 13:02:13.809273  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.812201  8747 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
W20260709 13:02:13.833161  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.784337 10115 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:13.806526 10154 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:13.801049 10008 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:13.850883  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:13.830452  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:13.840651  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:13.830420  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:13.839949  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:13.899744  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:13.901540  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 62805.4us
I20260709 13:02:13.831244  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:13.922055 10172 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:02:13.922466  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:13.924311  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:13.925767  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:13.927361  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:13.926522  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:13.923647  8579 catalog_manager.cc:5697] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:14.007856  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:13.959957  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:13.970255  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:13.998523  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 81917us
I20260709 13:02:14.014016  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.010255  8747 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:13.906484  9991 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:14.019985  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.020466  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.021134  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:13.967190  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:13.962841  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:13.942894  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.015126  9383 raft_consensus.cc:2410] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:02:13.970953 10083 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.510s	user 0.015s	sys 0.002s
I20260709 13:02:13.928098 10131 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:13.967520  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:13.919664 10115 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:14.030243  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.036254  9387 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:02:14.027717  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.037602  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.039397 10005 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.040659 10161 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:14.042847 10061 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.043596  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.045519 10061 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.046675  8750 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:14.046887  8679 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:13.991254 10082 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.031778  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:13.961731  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:13.962368  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:13.966446 10164 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.516s	user 0.006s	sys 0.009s
I20260709 13:02:13.964520  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.034404  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.034853  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 90) took 1046 ms. Trace:
I20260709 13:02:14.023388 10152 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:14.026074  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.051435  8681 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:14.056340  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1671) took 1004 ms. Trace:
I20260709 13:02:14.058293 10156 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.068334  8754 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:14.069557  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.075258  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.080732  9353 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:14.081844 10054 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.508s	user 0.013s	sys 0.000s
W20260709 13:02:14.070031 10178 log.cc:927] Time spent T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:02:14.091056 10179 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.084s	user 0.003s	sys 0.002s
I20260709 13:02:14.119794  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.119997  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.120965  9383 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:02:14.121763  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.093045  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.096241  8748 rpcz_store.cc:276] 0709 13:02:13.052305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.056516 (+  4211us) service_pool.cc:224] Handling call
0709 13:02:13.056808 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.594872 (+538064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.056059 (+461187us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54001577d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:14.056238 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235026176}
I20260709 13:02:14.099685  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:14.099740  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:14.105167  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.107590  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 94) took 1117 ms. Trace:
I20260709 13:02:14.062798 10083 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.117467  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.085148 10061 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.126345  8751 rpcz_store.cc:276] 0709 13:02:12.990273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.990831 (+   558us) service_pool.cc:224] Handling call
0709 13:02:12.991203 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.521152 (+529949us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.524793 (+  3641us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.529208 (+  4415us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:13.529247 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.106936 (+577689us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b5400157cd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:14.107014 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.107317 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1205973376}
I20260709 13:02:14.116350 10054 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.118939  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.122440  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.131945  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.095554  8755 rpcz_store.cc:276] 0709 13:02:12.988625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.992805 (+  4180us) service_pool.cc:224] Handling call
0709 13:02:14.027267 (+1034462us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b5800083198. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4e9c 00007f32df3d4a5b 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:14.029771 (+  2504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.034205 (+  4434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.034249 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.034319 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:14.034358 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.034441 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.034587 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13200640}
I20260709 13:02:14.095701 10164 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.130512  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.133448  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.138895  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.131987  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:14.143123  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 380398us
I20260709 13:02:14.144392  9384 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:14.144615 10157 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.145767  8827 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:02:14.149340  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 183218us
I20260709 13:02:14.152325  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.132588 10061 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.152043  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.138691 10053 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.161233  8756 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:14.158835  8747 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:14.144016 10156 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:14.143123 10082 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:14.162993  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1212) took 1080 ms. Trace:
W20260709 13:02:14.170796  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 107096us
I20260709 13:02:14.157130  9937 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:02:14.188344  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:14.191627  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.194344  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.197341 10083 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:14.166428 10089 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:14.198824  9937 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:14.199419  9937 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:14.199299 10089 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.200811  9937 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.168932 10104 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:14.203260  8796 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:14.203824 10054 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.205495 10082 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:02:14.208370 10082 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.208186  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:14.209533 10154 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.210018 10053 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:14.210449 10053 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:14.211086 10053 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.203354 10104 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.169268 10156 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.218562 10156 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.219650  8796 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:14.217207  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.210036 10154 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.251471  8645 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:14.256182 10104 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:14.250344  9937 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.259552  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.252231 10098 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.261138 10098 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.263779 10098 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.254914 10154 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.270453  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.167460  8828 rpcz_store.cc:276] 0709 13:02:13.059882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.478255 (+418373us) service_pool.cc:224] Handling call
0709 13:02:13.478898 (+   643us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.479782 (+   884us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.479821 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:13.542339 (+ 62518us) log.cc:844] Serialized 26 byte log entry
0709 13:02:13.564840 (+ 22501us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:13.671627 (+106787us) log.cc:844] Serialized 14 byte log entry
0709 13:02:14.132146 (+460519us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b180030aae0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:02:14.132384 (+   238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.132498 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:14.132543 (+    45us) raft_consensus.cc:1685] finished
0709 13:02:14.140272 (+  7729us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.140401 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62866,"prepare.queue_time_us":177643,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":466576,"raft.run_cpu_time_us":724,"raft.run_wall_time_us":46639,"spinlock_wait_cycles":14519552,"thread_start_us":128081,"threads_started":2,"wal-append.queue_time_us":169765}
I20260709 13:02:14.271328  8748 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
W20260709 13:02:14.278421  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 58924.6us
W20260709 13:02:14.278884  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 64015.9us
W20260709 13:02:14.280696  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 120622us
I20260709 13:02:14.284089  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:14.285889 10013 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:14.286607  9999 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:14.287192  9999 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.287616  9999 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.287812  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1210) took 1238 ms. Trace:
I20260709 13:02:14.288203 10115 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.286478 10013 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:14.288267  8827 rpcz_store.cc:276] 0709 13:02:13.049851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.050038 (+   187us) service_pool.cc:224] Handling call
0709 13:02:13.050490 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:14.146219 (+1095729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.287471 (+141252us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540041c154. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:14.287702 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2504475520}
I20260709 13:02:14.289430 10013 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.288267  9991 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.280683 10104 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:14.290149 10104 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:14.290153  9991 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.291580 10083 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.293426  9991 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.291184 10096 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.268080 10164 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.301944  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.315908  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:02:14.322762  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 159227us
I20260709 13:02:14.300882 10104 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.327613  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:14.332772  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.335098  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:14.336923 10148 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.337452 10148 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.337888 10148 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.341480  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:14.332052  8796 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:14.289047 10115 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.328737  8755 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:14.334335  9381 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEARNER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:14.357558  8793 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:14.360389  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.350275 10135 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:14.349037 10089 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:14.347239 10061 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.354086  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.372025 10051 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.372216  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.372421  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.368934  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:14.374333  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.375339  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.376905  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.377719  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:14.378733  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.379702  8679 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:14.381691  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.382938  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.383719 10115 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:14.384032  8675 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:53334
I20260709 13:02:14.384716  8678 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:14.385066  8675 raft_consensus.cc:606] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Received request to transfer leadership
W20260709 13:02:14.389290  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.352607 10053 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.391031  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.394996  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.353550 10164 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:02:14.392715  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.369308 10094 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:14.396565  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 59629.8us
I20260709 13:02:14.411147  9382 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.415081  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:14.415352  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.415836  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.389997 10051 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:14.411293 10074 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:14.420317  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.430711  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.419507 10089 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.408363 10083 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:14.409049 10094 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.420315  8828 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:14.411248 10135 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:14.442112  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 84676.4us
W20260709 13:02:14.449224  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 86363.9us
I20260709 13:02:14.448549  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1211) took 1384 ms. Trace:
I20260709 13:02:14.439121 10094 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.454396  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.443090 10074 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.463408  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.463888 10074 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:14.431980 10102 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:02:14.466801 10135 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.468307  8575 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43462 (request call id 263) took 3872 ms. Trace:
I20260709 13:02:14.469525 10135 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:14.472150  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.473126  8679 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:14.475690  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:14.476415  9999 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.478381  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.479305  8672 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:14.448671 10010 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.483917  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:14.442796 10164 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.492022 10053 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:14.494455  8719 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:14.487943  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.488988  9991 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.497076  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:14.466337  8675 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:53334
I20260709 13:02:14.495756  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.499815  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.501453  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:14.502000 10159 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:14.451228  8824 rpcz_store.cc:276] 0709 13:02:13.050972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.631829 (+580857us) service_pool.cc:224] Handling call
0709 13:02:13.724624 (+ 92795us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:14.141822 (+417198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.141858 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:14.165519 (+ 23661us) log.cc:844] Serialized 26 byte log entry
0709 13:02:14.252870 (+ 87351us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:14.278859 (+ 25989us) log.cc:844] Serialized 14 byte log entry
0709 13:02:14.279102 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.279187 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:14.339573 (+ 60386us) spinlock_profiling.cc:243] Waited 7.17 ms on lock 0x7b18000eadc0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:02:14.432080 (+ 92507us) raft_consensus.cc:1685] finished
0709 13:02:14.432178 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.434365 (+  2187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":399779,"prepare.queue_time_us":26989,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":387164,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":2963,"spinlock_wait_cycles":16485632,"thread_start_us":86899,"threads_started":1,"wal-append.queue_time_us":154893}
I20260709 13:02:14.503260 10082 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.515820  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.517093 10157 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.517621 10157 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:14.518091 10157 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.512239  9937 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:02:14.524803 10164 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:14.524952 10112 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.525480 10112 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:14.525893 10112 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.522404  8719 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:14.526759 10164 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:14.522918 10082 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.512176  9991 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.527240 10164 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.527302 10082 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.527738 10164 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.529171  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.531157 10150 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.531203 10159 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:14.531800 10159 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:14.532044  9385 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:14.532859 10159 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.535661  8645 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:14.536942 10115 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.532127  9999 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.540518 10152 raft_consensus.cc:3055] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:14.540962 10152 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:14.541190 10154 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.545849  8825 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:14.512483  8675 raft_consensus.cc:606] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Received request to transfer leadership
I20260709 13:02:14.529076  9991 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.548760  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.539870  8719 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:14.549714 10102 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:14.550793  8794 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.550707 10102 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:14.542368 10152 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.552026 10135 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.552768 10152 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.552377 10013 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.553730 10135 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:14.554208 10135 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:14.554661 10029 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:14.555087 10029 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.555474 10029 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.555538 10013 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:14.557320  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:14.520584 10094 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.558919  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:14.559402  8829 raft_consensus.cc:2368] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3.
I20260709 13:02:14.559784 10010 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.559790 10135 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.562206 10094 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:14.563498 10187 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:02:14.563597  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:14.564904 10082 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.561877 10135 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:14.565929  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.468776  8575 rpcz_store.cc:276] 0709 13:02:10.595915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.596134 (+   219us) service_pool.cc:224] Handling call
0709 13:02:14.468047 (+3871913us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:12.248189 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:12.248773 (+   584us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:12.248810 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:02:12.248828 (+    18us) tablet.cc:662] Decoding operations
0709 13:02:12.249502 (+   674us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:12.249585 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:12.249611 (+    26us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:02:12.250131 (+   520us) tablet.cc:701] Row locks acquired
0709 13:02:12.250144 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:02:12.250285 (+   141us) write_op.cc:270] Start()
0709 13:02:12.250401 (+   116us) write_op.cc:276] Timestamp: P: 1783602132250242 usec, L: 0
0709 13:02:12.250417 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:02:12.347383 (+ 96966us) log.cc:844] Serialized 2136 byte log entry
0709 13:02:13.066488 (+719105us) op_driver.cc:464] REPLICATION: finished
0709 13:02:13.704707 (+638219us) write_op.cc:301] APPLY: starting
0709 13:02:13.704916 (+   209us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:13.705219 (+   303us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:13.726677 (+ 21458us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:13.765945 (+ 39268us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:13.766370 (+   425us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:02:13.766450 (+    80us) write_op.cc:312] APPLY: finished
0709 13:02:13.768616 (+  2166us) log.cc:844] Serialized 47 byte log entry
0709 13:02:13.768945 (+   329us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:13.769233 (+   288us) write_op.cc:454] Released schema lock
0709 13:02:13.769313 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:13.788311 (+ 18998us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:13.788345 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12880,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":552050,"apply.run_cpu_time_us":5622,"apply.run_wall_time_us":111896,"num_ops":5,"prepare.queue_time_us":510349,"prepare.run_cpu_time_us":58742,"prepare.run_wall_time_us":320508,"raft.queue_time_us":223575,"raft.run_cpu_time_us":9372,"raft.run_wall_time_us":228809,"replication_time_us":792282,"spinlock_wait_cycles":18377600,"thread_start_us":986675,"threads_started":4,"wal-append.queue_time_us":406166}]]}
I20260709 13:02:14.552048  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.568260  9937 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:14.569412  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:02:14.572423  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 93347.6us
I20260709 13:02:14.573315  9355 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:14.571012  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:14.583989  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:14.589367  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:14.546233 10148 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.590289  8748 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:14.576664  9999 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:14.590863 10161 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.582587 10013 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.593029  9999 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.595973 10013 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.596436 10013 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.574133 10102 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:14.581144  8563 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 2147fd3c99a94f0c8e9c1a914bdb79c3 already in progress [suppressed 2 similar messages]
I20260709 13:02:14.606317  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.607924  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.609326  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.587674 10131 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:14.585953  9991 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:14.612099  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.596280  9999 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.615458  8676 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:14.616904 10102 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:14.617027  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:14.617736  8677 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.617661 10102 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.614949  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.620318 10102 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:14.621088 10102 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:14.621740 10102 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.622908  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:14.623129  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:14.611881 10074 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:14.614451 10051 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.626864  9937 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:14.627328  9937 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.627723  9937 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.632438 10161 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.636358 10115 raft_consensus.cc:993] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: : Instructing follower 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:14.637040 10115 raft_consensus.cc:1081] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Signalling peer 7aff04f53d4845a6a4fa491fd2a06145 to start an election
I20260709 13:02:14.614693  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.646804  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:14.648168  9387 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:14.644130  8719 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:14.583777 10164 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.601270 10159 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:14.656472  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.657511  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:14.669682  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68089us
I20260709 13:02:14.663244  8677 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
W20260709 13:02:14.671136  8563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 90165.5us
I20260709 13:02:14.672055  9387 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
 from {username='slave'} at 127.0.0.1:34746
I20260709 13:02:14.673147  9387 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:14.673674  9387 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.615356 10098 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.614919  9991 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:14.688922  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.691931 10098 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.694720 10013 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.697279 10148 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.655877 10154 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:14.698858 10161 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.699215 10094 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:14.700853 10013 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:14.702755 10157 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.705615 10157 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:14.708017 10102 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:14.694808  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.707679  9991 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.709503  9991 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.710526  9999 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.711735  8750 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:14.711830  8793 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:14.714293  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:14.666301  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 49645.2us
I20260709 13:02:14.723816 10013 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:14.716861 10150 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.719746  8754 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:14.734331  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54494.7us
I20260709 13:02:14.718405 10154 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:14.746021 10156 consensus_meta.cc:287] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.016s
I20260709 13:02:14.746640 10156 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.748848  8756 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:14.751207 10112 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.739537  9355 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:14.757267 10164 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.718878 10104 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.771559  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:14.777458 10195 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:02:14.778083  8830 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:14.750419 10156 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:14.781255  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.781858  8825 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:14.783681  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:14.777109  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 80492.6us
I20260709 13:02:14.785787  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.791059  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.790441 10082 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.795190  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:14.796146  8747 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:14.797686  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:14.797923  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:14.800993  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.801172 10148 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:14.803056 10148 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:14.803486 10148 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:14.804522 10148 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.806023 10148 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.806598  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.806833  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.808722  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.810922  9999 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.812211  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.813416  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:14.814289  9937 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.790756 10189 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.817430 10104 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.792389  9382 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:14.819082  8793 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:14.819818 10112 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:14.819244 10083 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:14.794016 10029 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.802148 10150 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.823191 10150 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.824982 10083 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:14.827265  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.827848 10154 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:14.828277 10154 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:02:14.829586  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 96768.6us
I20260709 13:02:14.831614  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:14.829295 10154 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.833004 10154 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.831882  9355 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:14.835747  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.836740 10083 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:14.836830  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.838829 10083 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:14.839213 10029 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.848680  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.849342 10188 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.849758 10188 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:14.850142 10188 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.853602  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:14.854791  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 78322.4us
I20260709 13:02:14.858070  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:14.839423 10083 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:14.841346  9937 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.868958  9387 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:14.878094  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 212095us
I20260709 13:02:14.877126  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.861495 10135 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:14.881982  9387 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.883092  8827 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:14.884168  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:14.884857  8752 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.868822 10092 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.886688 10074 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:14.887708 10104 raft_consensus.cc:1064] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
W20260709 13:02:14.896056  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 124634us
I20260709 13:02:14.899605  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.889935 10074 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.890015  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.908571  9937 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:14.911051  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.888929  9355 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:14.911878  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.914811 10189 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:14.905867 10074 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:14.915226  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.915184  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.920002  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.920807  8753 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 2: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:14.921607  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.919591 10029 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.923003  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:14.914422  9355 leader_election.cc:400] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3.
I20260709 13:02:14.923750  8747 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:14.919641  8754 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 2: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:14.949256  9999 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.949440  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:14.950506  8749 raft_consensus.cc:2368] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
I20260709 13:02:14.923187 10029 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.950699  9991 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.951457  9991 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.952827 10029 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:14.953751  9991 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.936709 10194 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:14.954878  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.954805 10194 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.957080 10194 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:14.957990  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:14.914259 10150 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.950064  9999 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:14.928462  9355 leader_election.cc:403] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:14.918095 10083 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:14.966389 10083 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:14.968528  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:14.966012  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.976235  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.976776  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.979452  8752 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:14.980803  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:14.980983  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:14.981171  8749 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:14.981717  8827 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:14.982884  8750 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:14.985888  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:14.988772 10189 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.000378 10188 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:15.001986  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:15.002751  8677 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.004395  8676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:53334
I20260709 13:02:15.005127  8676 raft_consensus.cc:606] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Received request to transfer leadership
I20260709 13:02:15.005182  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:14.989341  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.006826  8823 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:15.007284  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:14.995328  8755 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:15.008018  8678 raft_consensus.cc:3055] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:15.008337  8678 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:15.009065 10204 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:15.009730 10204 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:15.010547  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.011286  8679 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:15.009209  8678 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:15.012104 10204 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:15.012641  8678 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.013573 10150 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:15.013926  9999 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:15.014740  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 100606us
I20260709 13:02:15.016049  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:15.016831  8681 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.019119  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:15.019850  8675 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.022850  8822 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:15.015982  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:15.024184  8679 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:15.025156  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 168547us
W20260709 13:02:15.027626  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.028945  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:15.028815  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:15.031627  8826 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:15.027570  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.037833  8677 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:15.044750  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:15.043699  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:15.045812  8824 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.045861  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:15.046777  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:15.049612  8827 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:15.059463  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:15.033228 10150 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:15.058943 10029 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:15.076377 10029 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:15.076828 10029 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.078792 10197 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.067698 10150 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:15.080720 10150 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.070586  8579 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43482 (request call id 225) took 3317 ms. Trace:
I20260709 13:02:15.085681  9991 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:15.086416  9991 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.088625  9991 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:15.089190  9991 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.089855  9991 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:15.091337  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:15.092473 10195 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.092320  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:15.094511  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:15.095489  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.096165  8680 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:15.068172  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:15.096786 10204 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.097100  8672 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.090351  9991 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.099442  9991 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:15.081754  8579 rpcz_store.cc:276] 0709 13:02:11.753449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.753664 (+   215us) service_pool.cc:224] Handling call
0709 13:02:15.070513 (+3316849us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:14.459588 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:14.471904 (+ 12316us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:14.471963 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:02:14.471999 (+    36us) tablet.cc:662] Decoding operations
0709 13:02:14.472474 (+   475us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:14.472591 (+   117us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:14.472630 (+    39us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:02:14.472976 (+   346us) tablet.cc:701] Row locks acquired
0709 13:02:14.473037 (+    61us) write_op.cc:260] PREPARE: finished
0709 13:02:14.473220 (+   183us) write_op.cc:270] Start()
0709 13:02:14.473397 (+   177us) write_op.cc:276] Timestamp: P: 1783602134473151 usec, L: 0
0709 13:02:14.473419 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:02:14.474231 (+   812us) log.cc:844] Serialized 595 byte log entry
0709 13:02:14.738658 (+264427us) op_driver.cc:464] REPLICATION: finished
0709 13:02:14.919544 (+180886us) write_op.cc:301] APPLY: starting
0709 13:02:14.919676 (+   132us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:14.919793 (+   117us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:14.922972 (+  3179us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:14.923310 (+   338us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:14.923491 (+   181us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:02:14.923535 (+    44us) write_op.cc:312] APPLY: finished
0709 13:02:14.929150 (+  5615us) log.cc:844] Serialized 23 byte log entry
0709 13:02:14.933225 (+  4075us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:14.933331 (+   106us) write_op.cc:454] Released schema lock
0709 13:02:14.933386 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:14.933847 (+   461us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:14.933882 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":214940,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":166855,"apply.run_cpu_time_us":2678,"apply.run_wall_time_us":15903,"num_ops":1,"prepare.queue_time_us":133662,"prepare.run_cpu_time_us":3732,"prepare.run_wall_time_us":56305,"replication_time_us":264953,"thread_start_us":184070,"threads_started":2}]]}
I20260709 13:02:15.103562  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.104103  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:15.109819 10074 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.072214 10148 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:15.115932  8673 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 3: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:15.105731  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:15.116389  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47122us
I20260709 13:02:15.105556  8754 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:15.070447  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:15.117695  8645 leader_election.cc:400] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
I20260709 13:02:15.118103  8645 leader_election.cc:403] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:02:15.118026  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.121589  8828 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:15.121929  8678 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:15.116348 10148 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:15.115739 10189 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3.
I20260709 13:02:15.129555  8681 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:15.112423 10074 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:15.134684  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.131459 10010 consensus_queue.cc:1059] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:15.132573 10051 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:15.186972 10051 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:15.188153 10206 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:15.188006 10051 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.188761 10206 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.189635 10051 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.191601 10115 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.192008  8675 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:15.182456 10195 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
W20260709 13:02:15.165331  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 62424.3us
I20260709 13:02:15.171154  8747 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3. Ops: [2.4-2.4]
I20260709 13:02:15.190239  8576 catalog_manager.cc:5697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 2 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:15.198526 10148 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:15.186587  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:15.202481 10161 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:15.179738  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.198908 10092 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:02:15.203722 10188 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:15.204129 10092 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:15.204519 10092 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:15.205839  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:15.205719 10092 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.207186 10092 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.208307  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:15.209283  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.210772 10010 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:02:15.181509  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 63577.3us
I20260709 13:02:15.183008  8824 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:15.211194 10010 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:15.183393  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.203071 10161 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.212179 10010 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:15.207446  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:15.213495 10010 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.212666 10161 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:15.183632 10102 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
W20260709 13:02:15.215709  8627 auto_rebalancer.cc:893] Removing replace marker failed: Replica 0e8c59a1f0394eae8ce33b99f10efadc is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:15.215946 10089 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:02:15.216413  8627 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 2147fd3c99a94f0c8e9c1a914bdb79c3 already in progress
I20260709 13:02:15.207099  8719 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:15.218034  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:15.218156  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:15.220304  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 180324us
I20260709 13:02:15.181684  8679 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:15.221809  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.222532  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.224223  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:15.217442  8627 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 2147fd3c99a94f0c8e9c1a914bdb79c3 already in progress
I20260709 13:02:15.229737  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:15.230105  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.230593 10196 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.230994 10188 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:15.232008  9352 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:15.202385  8672 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:15.241807  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.244279  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.244702  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.248018  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:15.220920 10089 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:15.243947  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:15.222719 10061 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:15.255940  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.226393 10159 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.221258 10204 consensus_queue.cc:1059] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:02:15.254212 10013 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:15.258975  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.263026  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:15.259563 10013 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.264281 10061 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:15.251456  8821 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:15.258728  8719 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:15.264045 10161 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.273970  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:15.249715 10195 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.222767 10029 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:15.276860  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.279865  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.280526  8719 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:15.282711  8830 raft_consensus.cc:2368] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
I20260709 13:02:15.284505  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:15.251617 10089 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:15.287897  8826 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:15.294391  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.259696 10204 raft_consensus.cc:3055] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:15.257141 10196 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.258646 10206 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:15.301203  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:15.303633  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.300408  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.325996  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.326753  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.311993  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:15.327399  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 95485.2us
I20260709 13:02:15.314560 10089 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.256811 10199 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:15.292104 10197 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:15.265314 10061 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.345281  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.332618  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.333451  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.334167  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.334928  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.288554 10189 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:15.283385 10188 raft_consensus.cc:697] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:15.336882  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:15.338992  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:15.339670  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:15.356076  8751 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 2: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:15.257303 10096 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 13:02:15.345243  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.279596 10156 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:15.279213 10135 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:15.331416  8827 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 2: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:15.335858  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.359642  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.361416  8647 leader_election.cc:400] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
I20260709 13:02:15.364745  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:15.307504 10204 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:15.279352 10201 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:02:15.327872 10013 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:15.364959  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 139184us
I20260709 13:02:15.377058 10089 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.370453 10189 raft_consensus.cc:697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:15.384585 10194 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.370738 10135 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.395385  8673 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:15.409135  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.385754  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:15.410715  8722 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.410826  8829 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
W20260709 13:02:15.418314  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 47534.3us
I20260709 13:02:15.424554  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.427126  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.431257  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.432758  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:15.429580  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 63462us
I20260709 13:02:15.437846  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:15.438906  8681 raft_consensus.cc:2410] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:15.414523 10197 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:15.413471 10199 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:15.430467  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.437358  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.447222  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.449426 10029 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:15.450132 10161 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:15.457952  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47357.3us
I20260709 13:02:15.459757  8679 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:15.463318  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.458393  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:15.464802  8826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:58932
I20260709 13:02:15.465571  8826 raft_consensus.cc:606] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.465973  8826 raft_consensus.cc:614] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:15.467571  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.469619  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:15.454484  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.488858  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:15.470413  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.495044  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:15.496083  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.498957  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.458822  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.465024 10218 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.503456  8676 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:15.504078  8647 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.486757  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:15.507663  8754 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:15.497355 10224 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.508437 10199 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.511426 10051 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.416471 10204 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:15.429045 10188 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:15.512499 10013 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.517275  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:15.533864  8674 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:15.469247 10220 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:15.519399 10194 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.513164 10189 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:15.511745 10161 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.512167 10051 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.464907 10135 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:15.536029  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 78866.8us
I20260709 13:02:15.470144 10201 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:02:15.542786  8673 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:15.543864  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:15.551313  8748 raft_consensus.cc:2368] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4.
I20260709 13:02:15.555503  8823 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
W20260709 13:02:15.548511  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 93558.1us
I20260709 13:02:15.573372  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:15.542836  8822 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:15.560075 10051 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:15.552186 10204 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.576365 10051 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:02:15.558038  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 90518.7us
W20260709 13:02:15.562078  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 99658.8us
I20260709 13:02:15.584683  8722 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.589134 10226 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.591336  8675 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:15.592623  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.595234  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.595945  8676 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:15.598731  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:15.601315  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.606639 10228 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.545495 10197 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.607386  8675 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:15.608147  8675 raft_consensus.cc:606] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.610597  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:15.601792  8679 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:15.602319  8680 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:15.612378  8722 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.613121  8675 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:15.577775  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:15.622406  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.623414 10226 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.624835  8828 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:15.626044  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.626871 10226 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.628990  8681 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
W20260709 13:02:15.617426  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.577584 10051 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:15.615357 10228 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.619959 10230 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.620482  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.613759  8675 raft_consensus.cc:606] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.631436  8675 raft_consensus.cc:614] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:15.632594  8747 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:15.633433  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.633927 10230 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.635195  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.628813  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.636623 10094 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:15.614306  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.638741 10226 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.641018  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.641968 10094 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.638702 10092 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.646937  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:15.647748  8675 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:15.643918  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:15.653486  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.673827 10094 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.675084 10154 raft_consensus.cc:3055] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:15.675572 10154 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:15.676620 10154 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.677754 10154 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.653216 10092 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.633495 10104 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:15.659809  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:15.660414  8677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:15.717078  8677 raft_consensus.cc:606] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:15.751396  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 247398us
W20260709 13:02:15.751396  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 98850.4us
I20260709 13:02:15.752658  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.753846  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:15.754935  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:15.755545  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55605.6us
I20260709 13:02:15.756763  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.758327  8647 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.758841  8677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:15.759589  8677 raft_consensus.cc:606] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.760070  8794 leader_election.cc:400] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4.
W20260709 13:02:15.760143  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 131563us
I20260709 13:02:15.760466  8794 leader_election.cc:403] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:15.756609  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.760965  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:15.761832 10221 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.765352  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.759050 10206 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:15.791232  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.791131  8793 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.762293 10102 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:15.785856  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:15.848417  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 57440.6us
W20260709 13:02:15.831049  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 70154.7us
I20260709 13:02:15.759476 10230 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.758258 10094 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:15.887822  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 102076us
I20260709 13:02:15.852629  8722 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.784406 10083 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:02:15.772125 10135 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.820322 10223 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:15.819628 10089 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:15.909896  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 57408.5us
I20260709 13:02:15.820694 10061 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:15.916435 10061 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:15.916884 10061 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.853513 10102 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:15.891685 10229 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.919571 10051 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.914408  8722 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:15.908584  8575 catalog_manager.cc:5697] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 2 to 3, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:15.915509 10103 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:15.895380  9383 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:02:15.820627 10112 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.090s
I20260709 13:02:15.894163  9991 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.819392 10156 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:15.954564  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:15.961887 10209 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:02:15.974802  8681 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:15.959744  9355 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.982100  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.963377  8749 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:15.960053 10082 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:02:15.985538  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 90966.4us
I20260709 13:02:15.992090  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:15.999645  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.981029 10229 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:16.025415  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.988183 10228 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:16.032300  8722 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:15.994297  8675 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:16.000391 10082 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:16.035640 10082 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:16.038486 10237 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:16.039101 10089 raft_consensus.cc:3055] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:16.039507 10089 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:16.040407 10089 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:16.041688 10089 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:16.043220 10156 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:15.994375 10189 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:15.996052  9999 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:16.030668 10197 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:16.050407 10224 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:16.051677  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.027228 10112 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:16.075624 10102 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:16.075273  8575 catalog_manager.cc:5697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:16.080467  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 81732us
W20260709 13:02:16.086212  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.091805  8675 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff71213103664c0bab63c42773d0880b"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
W20260709 13:02:16.026638  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 74705.9us
W20260709 13:02:16.094882 10238 log.cc:927] Time spent T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:02:16.096623 10102 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:16.049180 10229 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.105309 10229 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:16.104938 10094 consensus_queue.cc:1059] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:16.106645  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:16.110919  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 78658.4us
W20260709 13:02:16.117441  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 68109.9us
I20260709 13:02:16.118326  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:16.120051  8722 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:16.126837  8751 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:16.156992 10225 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:16.157549  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:16.160163  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 53676.7us
I20260709 13:02:16.161029  8793 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:16.159071 10239 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:16.175832  8722 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:16.158175 10061 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:16.189608  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:16.192066  8749 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:16.163791 10228 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:16.168237 10092 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:16.170387 10229 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:16.207032  8753 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:16.197822 10225 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:16.172282 10010 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
W20260709 13:02:16.236505  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.221853  9381 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEARNER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:16.177402 10239 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.245725  8752 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:16.219923  8823 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:16.204722 10201 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:02:16.216169 10242 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:16.253765  8828 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:16.240528  9937 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.264438 10241 raft_consensus.cc:3055] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:16.256415 10208 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:02:16.270308  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:16.283824 10204 raft_consensus.cc:993] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: : Instructing follower 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:16.284454 10204 raft_consensus.cc:1081] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Signalling peer 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:16.287050  8675 raft_consensus.cc:606] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:16.287721  8675 raft_consensus.cc:614] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:16.239140 10229 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:16.296412 10241 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:16.329921 10241 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:16.330601 10195 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.305797 10224 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.331226 10195 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:16.331390 10224 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.334483 10195 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:16.319582 10154 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.641s	user 0.014s	sys 0.000s
I20260709 13:02:16.327912 10207 raft_consensus.cc:993] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: : Instructing follower 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:16.324749 10229 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:16.317649 10221 consensus_meta.cc:287] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.555s	user 0.006s	sys 0.007s
I20260709 13:02:16.364732 10156 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:16.358687  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:16.367760  8753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:39032
I20260709 13:02:16.367965  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:16.373191 10199 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:16.373922  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:02:16.373827 10199 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:16.358034 10010 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:16.301904  9937 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:16.376938  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:16.371039 10225 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:02:16.377036 10199 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:16.356051 10131 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:16.381251 10221 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:16.382157 10131 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.382779 10061 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:16.384527 10131 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:16.377035 10010 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.385910  9937 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:16.387231  8677 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:16.387593 10207 raft_consensus.cc:1081] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Signalling peer 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:16.388847 10010 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:16.392328 10010 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.393010 10010 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.394598  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:16.394971  8753 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:16.389818  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:16.395469  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:16.294286 10242 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:16.395628 10010 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:16.390141  8681 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:16.399400  8674 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:16.401302  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:16.390599 10204 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:16.403282  9937 raft_consensus.cc:3055] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:16.406010  9937 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:16.401650 10242 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:16.404561  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:16.405556  8751 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:39032
I20260709 13:02:16.408064  8751 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:16.408622  8751 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.402525  8747 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:16.411599  9353 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:16.408049  8755 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
W20260709 13:02:16.413057  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54481.9us
I20260709 13:02:16.395488  8753 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:16.348191 10224 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:16.442406  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:16.442883  8830 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:02:16.443187  8828 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:16.407112  9937 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:16.445267  8679 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:16.447463  8645 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:16.430769 10061 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:16.450008 10061 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.428824 10103 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:16.457919  8679 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:16.458729 10061 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:16.460860  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:16.461689  8823 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:16.463600 10194 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:16.456722  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:16.464766 10246 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:16.465353 10246 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:16.466161 10051 consensus_meta.cc:287] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.000s
I20260709 13:02:16.467182  9937 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:16.467298 10102 consensus_queue.cc:1059] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:02:16.468015  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 278557us
I20260709 13:02:16.468849  8722 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:16.445477 10188 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:16.472123 10188 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.464089 10194 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.472967 10194 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.473167  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:16.473922  8677 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:16.464816  8754 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:16.475342  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:16.472537 10188 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.477732 10199 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:16.478253 10246 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:16.479152 10103 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:16.479235 10199 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:16.479956 10103 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.484612 10225 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:16.492091 10239 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:16.492096  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:16.495537  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:16.493757  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:16.496927  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:16.497872  9381 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 2: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:16.498077 10199 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:16.498721 10199 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:16.499162 10225 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:16.499732 10225 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:16.500695 10225 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:16.501681 10010 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:16.502128 10218 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:16.499732 10199 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.532303  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:16.507341  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:16.521220  8751 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:16.522279  8753 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.565804  8747 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:16.566511  9387 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:16.571365  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:16.573613  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:16.575930  8824 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:16.581154  8824 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:16.583076  8827 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:16.449331  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:16.509310 10159 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.507685  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:16.506547 10103 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:16.587055  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:16.583034  8753 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:16.591454  8751 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:16.565528 10239 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:16.594475  8824 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:16.566860 10010 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:16.591082  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:16.621012 10104 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:16.603842  8675 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:16.676003  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:16.633369 10010 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.688237  9384 raft_consensus.cc:2393] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 2: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:16.689576 10010 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:16.628005  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:16.631990  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:16.695353 10216 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:16.696398  9385 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:16.595578  9991 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:02:16.595618 10159 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.700477 10159 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:16.633463  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:16.701285  8749 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:16.603236 10239 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.704149  8677 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:16.713598  8830 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
W20260709 13:02:16.725510  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.724026 10083 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.816s	user 0.005s	sys 0.009s
W20260709 13:02:16.741386 10249 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:02:16.745157  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:16.732184  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:16.756937  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:16.757472  8575 catalog_manager.cc:5697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 1 to 2, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 2 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:02:16.760462 10104 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:16.737002 10083 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4.
I20260709 13:02:16.762604 10239 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:16.762127 10223 raft_consensus.cc:2955] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:16.767422  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:16.778604  8680 raft_consensus.cc:3055] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:16.768679  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:16.779764  8678 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:16.780158  8678 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:16.771394  8674 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:16.761256  8576 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40096 (request call id 137) took 5205 ms. Trace:
W20260709 13:02:16.785197  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 273378us
I20260709 13:02:16.779098  8680 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:16.732740 10188 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.791297  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:16.792373  9385 raft_consensus.cc:2393] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:16.792289  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:16.793102  9381 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:16.781385  8576 rpcz_store.cc:276] 0709 13:02:11.555428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.555578 (+   150us) service_pool.cc:224] Handling call
0709 13:02:16.761134 (+5205556us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:15.339688 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:15.340319 (+   631us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:15.340372 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:02:15.340390 (+    18us) tablet.cc:662] Decoding operations
0709 13:02:15.340766 (+   376us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:15.340835 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:15.340860 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:02:15.341077 (+   217us) tablet.cc:701] Row locks acquired
0709 13:02:15.341090 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:02:15.341266 (+   176us) write_op.cc:270] Start()
0709 13:02:15.341402 (+   136us) write_op.cc:276] Timestamp: P: 1783602135341210 usec, L: 0
0709 13:02:15.341428 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:02:15.342172 (+   744us) log.cc:844] Serialized 533 byte log entry
0709 13:02:15.344297 (+  2125us) op_driver.cc:464] REPLICATION: finished
0709 13:02:15.345541 (+  1244us) write_op.cc:301] APPLY: starting
0709 13:02:15.345661 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:15.345859 (+   198us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:15.346008 (+   149us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:15.351973 (+  5965us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:15.352170 (+   197us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:02:15.352230 (+    60us) write_op.cc:312] APPLY: finished
0709 13:02:15.352467 (+   237us) log.cc:844] Serialized 23 byte log entry
0709 13:02:15.352660 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:15.352757 (+    97us) write_op.cc:454] Released schema lock
0709 13:02:15.352803 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:15.353211 (+   408us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:15.353244 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2127,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":924,"apply.run_cpu_time_us":2038,"apply.run_wall_time_us":7621,"num_ops":1,"prepare.queue_time_us":130261,"prepare.run_cpu_time_us":2875,"prepare.run_wall_time_us":2875,"replication_time_us":2623,"spinlock_wait_cycles":25344,"thread_start_us":67988,"threads_started":1}]]}
I20260709 13:02:16.772447 10104 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:16.800668  8678 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:16.801824  8678 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.787043  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:16.812160  8680 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:16.723655  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 134871us
I20260709 13:02:16.807894  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:16.819672  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:16.823189 10199 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:16.823586 10252 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:02:16.823755 10199 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.824185 10199 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:16.777169  9383 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:16.825775  8647 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:16.826520 10115 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:16.826937 10115 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.836264 10115 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:16.847728 10010 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:16.848891  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:16.849663  8830 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:16.851897  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:16.828874 10258 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:16.863245  9352 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:16.884527  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:16.830971 10224 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:02:16.831187 10226 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:02:16.854483 10262 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:16.892393  8678 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
W20260709 13:02:16.897116  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.896525 10262 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.901350 10194 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:16.887532  9383 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:16.903083 10262 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:16.903769 10262 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:16.905794 10115 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.862411 10237 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:02:16.888047 10258 raft_consensus.cc:2955] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:16.867527 10061 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:02:16.911466 10188 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:16.936654 10237 raft_consensus.cc:3055] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:16.937139 10237 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:16.937958  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:16.938030 10237 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:16.939633 10237 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:16.939937 10265 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:02:16.934484  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 63935us
I20260709 13:02:16.944464  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:16.945267  8678 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.956149 10194 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:16.974782  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:16.983242  8678 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:16.976466  8749 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.985594  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:02:16.986019  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 126875us
I20260709 13:02:16.982999  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:16.993176  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:16.993539  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:16.994278 10268 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:16.982038 10271 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.995853  8719 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:16.983790  9381 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:16.994979 10246 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:17.009941  8676 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:16.994813 10268 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:17.003155  9383 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:17.006031  8678 raft_consensus.cc:2955] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:17.023494  8722 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:02:17.047506  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54066.2us
I20260709 13:02:17.049973  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:17.000682  8822 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:17.015321 10246 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:17.016219 10089 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:17.022450 10061 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:17.067536  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 44183.6us
I20260709 13:02:17.016620 10268 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:17.002120 10271 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:17.045960 10269 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:17.014015 10199 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:17.061527  9385 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:17.067775 10103 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:17.078066 10188 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.078766 10115 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:17.087750  8829 raft_consensus.cc:2955] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:17.090201 10225 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:17.087762 10103 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:17.075181 10278 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:02:17.134100 10061 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:17.137388 10271 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:17.137977 10274 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:17.138724 10274 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:17.140314 10225 raft_consensus.cc:697] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:17.141512 10225 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:17.143573 10274 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:17.144810  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:17.155997 10241 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:17.160852  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.169412  8672 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:17.087095 10269 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:17.130389 10103 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:17.127269 10115 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:17.102721 10188 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:17.117168 10104 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:17.213210  8747 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:17.216001  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:17.217511  8722 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:17.218283  8676 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:17.218643  8824 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:17.219513  8830 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:17.185279 10281 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.190968 10270 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.192842 10074 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:17.230119 10255 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:17.237048 10269 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:17.237922 10257 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:17.241763 10188 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:17.242998 10275 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:02:17.243719 10281 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:17.248445 10199 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:17.111889 10246 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:17.251545  8679 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:17.260560  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:17.261364  8752 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:17.258630  8680 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:17.266363  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:17.263041  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:17.272548  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:17.275223  8677 raft_consensus.cc:2410] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:17.275324  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:17.277731  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:17.278427  8752 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:17.250320 10255 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:17.274086  8754 raft_consensus.cc:2955] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:17.283311 10255 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:17.289636  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:17.291334  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:17.292058  9381 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.292482 10083 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:17.292991 10083 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:17.293877  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:17.194476 10115 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:17.338868  8677 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:02:17.347379  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 84906.3us
I20260709 13:02:17.260667  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:17.224885 10225 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.085s
I20260709 13:02:17.355655  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:17.310468 10154 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:17.295482 10204 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.361529  8750 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:17.404760  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:17.336010  9991 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:17.409659  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 61449us
I20260709 13:02:17.274521 10074 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:17.412837  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 60127.4us
I20260709 13:02:17.362524  8822 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:17.423641  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:17.309067 10083 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.421698  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:17.427489  9387 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.439785  8828 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:17.423377 10277 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 13:02:17.359819  8756 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
W20260709 13:02:17.366274  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.459359  8752 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:17.419668  8823 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:17.427042 10154 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.309661 10061 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:17.250948 10270 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:17.475383 10280 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:02:17.477941 10254 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:17.356451 10199 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.109s
I20260709 13:02:17.410888 10204 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:17.486200  9387 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:17.419423  9991 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:17.437467 10284 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.091s
W20260709 13:02:17.504788 10258 consensus_meta.cc:287] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.538s	user 0.016s	sys 0.006s
I20260709 13:02:17.469811 10112 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.486924  9384 raft_consensus.cc:2955] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:17.558012  8674 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:17.498682 10154 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:17.560729  8828 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
W20260709 13:02:17.565037 10249 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:02:17.510222 10224 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 13:02:17.568393 10277 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.565222 10104 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:17.517827 10112 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:17.620793  8829 consensus_meta.cc:287] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.518s	user 0.005s	sys 0.007s
I20260709 13:02:17.585883  9385 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:17.589938 10195 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:17.646379 10292 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 13:02:17.640174 10277 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:17.657467 10239 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:17.658650 10288 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:02:17.661369 10281 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:17.664865 10074 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:17.670660 10270 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:17.673998 10268 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.593s	user 0.008s	sys 0.003s
I20260709 13:02:17.876017 10239 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:17.876933 10268 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:17.878860  8751 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:17.879281 10268 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:17.631551 10115 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:17.693215  8749 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.701s	user 0.008s	sys 0.002s
I20260709 13:02:17.880452  8749 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:17.870379 10195 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:17.885330 10195 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:17.699733  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:17.705380  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:17.708943 10241 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.552s	user 0.015s	sys 0.000s
I20260709 13:02:17.712054 10290 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.887859 10290 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:17.733230  8830 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:17.892050 10290 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:17.894007 10239 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:17.743414 10112 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:17.756611  9388 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:17.771811 10204 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:17.773074  9991 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:17.851089  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:17.898938  8678 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
W20260709 13:02:17.914860  9355 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.115s	sys 0.151s
W20260709 13:02:17.916472  9355 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.115s	sys 0.152s
W20260709 13:02:17.917725  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 689500us
I20260709 13:02:17.919480  9353 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:02:17.921271  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 172337us
W20260709 13:02:17.922487  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 465283us
I20260709 13:02:17.923514 10270 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:17.923985 10270 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.924358 10270 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:17.926872 10224 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:17.930212 10224 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:17.943346 10224 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:17.945614 10224 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:17.874295 10277 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:17.932545 10237 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:17.946453 10237 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:17.948812 10237 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:17.949580  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:17.950788 10277 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.951352 10277 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:17.951666  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:17.953694 10277 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:17.955421  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:17.954389  8753 raft_consensus.cc:2955] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:17.937129 10283 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.940291  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:17.959134  9388 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:17.942206  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:17.935062  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:17.949858  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:17.961874 10131 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:17.962390 10131 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:17.962692  9355 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:17.962929 10131 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:17.957934  8647 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:17.958030  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:17.958992 10283 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:17.965965  8825 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 2: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:17.966029 10234 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:17.966812 10234 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:17.940961 10278 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:17.967841 10278 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:17.968327 10278 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:17.967778 10234 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:17.969259 10278 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:17.972164 10295 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:17.976212  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:17.977226  9991 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:17.971176 10283 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:17.977849  9991 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:17.978031 10278 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:17.863847  9381 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.571s	user 0.013s	sys 0.000s
I20260709 13:02:17.936127 10270 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:17.986425  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.013s	sys 0.004s
W20260709 13:02:18.009380  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.013s	sys 0.004s
W20260709 13:02:18.009999  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 650010us
I20260709 13:02:17.977461  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:18.011415  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:18.024396  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.020s	sys 0.027s
W20260709 13:02:18.024772  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.020s	sys 0.027s
W20260709 13:02:18.025950  8680 consensus_meta.cc:287] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.762s	user 0.013s	sys 0.000s
W20260709 13:02:18.025146  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 570430us
W20260709 13:02:17.981153 10255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.004s	sys 0.008s
I20260709 13:02:18.050369  8830 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:18.054332  8826 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:02:18.023608  8822 consensus_meta.cc:287] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.571s	user 0.003s	sys 0.009s
I20260709 13:02:18.135396  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:18.024844 10103 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.816s	user 0.009s	sys 0.004s
W20260709 13:02:18.070868  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 63366.5us
W20260709 13:02:18.024174 10083 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.002s
I20260709 13:02:18.026476  8680 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
W20260709 13:02:18.133734  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 92392.5us
I20260709 13:02:18.138769  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:18.137640  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:18.069411 10255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.004s	sys 0.008s
I20260709 13:02:18.154933  8826 raft_consensus.cc:2410] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:18.009146  9381 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:18.028820 10218 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.156177  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:18.034173 10112 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:18.180102  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:18.178260  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:18.170982  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1161) took 1386 ms. Trace:
I20260709 13:02:18.191272  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:18.193552  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:18.195574  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:18.197134  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:17.979343  9991 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:18.200593  9388 raft_consensus.cc:2368] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:18.184798  8680 rpcz_store.cc:276] 0709 13:02:16.767009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.767140 (+   131us) service_pool.cc:224] Handling call
0709 13:02:17.250812 (+483672us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b58002420d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24550aa 00007f32e24d7447 00007f32e24effe7 00007f32e24f17de 00007f32e24fd27c 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c
0709 13:02:18.153354 (+902542us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":760030976}
I20260709 13:02:18.137856 10115 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:18.211642  8677 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:18.139245  8822 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
W20260709 13:02:18.230598  8796 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.028s	sys 0.061s
I20260709 13:02:18.136277 10239 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:18.229801  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:18.158798 10089 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:18.223100  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:18.170831  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:18.292024  8796 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.030s	sys 0.061s
W20260709 13:02:18.214769  8821 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.035s	sys 0.074s
I20260709 13:02:18.207404 10013 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.303680  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:18.201290 10216 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:18.312063 10239 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:18.158475 10103 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:18.224066 10082 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:18.337445  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:18.214933 10287 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.116s
I20260709 13:02:18.172896 10083 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:18.342200  8824 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.011s	sys 0.013s
W20260709 13:02:18.308825  8821 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.035s	sys 0.076s
I20260709 13:02:18.325299  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:18.211301 10029 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.120s
W20260709 13:02:18.353279  8827 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.000s	sys 0.023s
I20260709 13:02:18.200335 10218 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:18.359562  8829 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.359606  8680 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:18.317235 10013 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:18.230674 10115 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:18.208454 10112 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:18.316669  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1059) took 1076 ms. Trace:
I20260709 13:02:18.326923  8719 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:18.364950  8827 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.000s	sys 0.023s
W20260709 13:02:18.389740  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 62945.4us
I20260709 13:02:18.366242  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:18.348383 10246 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.010s	sys 0.022s
W20260709 13:02:18.390636 10246 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.010s	sys 0.023s
I20260709 13:02:18.371646  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:18.394150  9352 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:18.383363  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:18.395534  8747 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:02:18.360221  8824 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.011s	sys 0.013s
I20260709 13:02:18.389912 10268 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:18.331254  8825 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:18.308892 10089 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:18.367475 10061 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.024s	sys 0.027s
I20260709 13:02:18.341010 10194 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.419957  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:18.395862  8826 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:18.368836 10013 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.365875 10290 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.336372 10216 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:18.414219 10061 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.024s	sys 0.027s
I20260709 13:02:18.448948  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:18.399537 10302 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:18.452193 10131 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:18.432540  8795 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.046s	user 0.011s	sys 0.050s
I20260709 13:02:18.399920 10246 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:18.467088  8795 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.081s	user 0.011s	sys 0.050s
I20260709 13:02:18.431239 10188 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:18.386471  8822 rpcz_store.cc:276] 0709 13:02:17.217605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.221443 (+  3838us) service_pool.cc:224] Handling call
0709 13:02:18.293961 (+1072518us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.467175  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:18.466374  8829 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:18.420027 10277 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:18.397855 10241 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:02:18.349154 10082 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:18.468704 10302 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:18.475824 10303 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:02:18.465101 10154 consensus_meta.cc:287] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.893s	user 0.012s	sys 0.001s
I20260709 13:02:18.466742 10268 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:02:18.469148 10270 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:18.480330 10296 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:18.511008 10296 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:18.493532 10277 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:18.497773 10103 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:18.473001 10188 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.522697 10188 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.523270 10089 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:18.526026 10216 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:18.435534 10194 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:18.491175 10029 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:18.510146  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1184) took 1006 ms. Trace:
I20260709 13:02:18.538394  8674 rpcz_store.cc:276] 0709 13:02:17.503686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.669591 (+165905us) service_pool.cc:224] Handling call
0709 13:02:17.675407 (+  5816us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.676273 (+   866us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.676306 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.676839 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.173820 (+496981us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:18.173883 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.187897 (+ 14014us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.509791 (+321894us) raft_consensus.cc:1685] finished
0709 13:02:18.509827 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.510029 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":122,"spinlock_wait_cycles":51840,"thread_start_us":496580,"threads_started":1,"wal-append.queue_time_us":745786}
I20260709 13:02:18.516562  8747 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:18.460482 10290 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:18.507599 10218 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:18.501698 10302 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:18.538264 10029 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.504532 10154 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:18.605616  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 391354us
I20260709 13:02:18.573971  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1174) took 1564 ms. Trace:
I20260709 13:02:18.620821 10268 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:18.624164 10225 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:18.625674 10286 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:18.644328 10082 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:18.645219  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1064) took 1391 ms. Trace:
I20260709 13:02:18.645498 10115 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:18.646322 10112 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:18.651063  8824 rpcz_store.cc:276] 0709 13:02:17.253924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.254996 (+  1072us) service_pool.cc:224] Handling call
0709 13:02:17.684050 (+429054us) spinlock_profiling.cc:243] Waited 8.2 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:17.684291 (+   241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.684680 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.684711 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.685234 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.397188 (+711954us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:18.397231 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.397308 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.644875 (+247567us) raft_consensus.cc:1685] finished
0709 13:02:18.644897 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.645142 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35700,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"spinlock_wait_cycles":18852736,"thread_start_us":711549,"threads_started":1,"wal-append.queue_time_us":916838}
I20260709 13:02:18.604257 10029 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:18.682881  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1065) took 1428 ms. Trace:
I20260709 13:02:18.685730 10290 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:18.687480 10277 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:18.691004  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1274) took 1699 ms. Trace:
I20260709 13:02:18.695973  8827 rpcz_store.cc:276] 0709 13:02:17.254391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.254568 (+   177us) service_pool.cc:224] Handling call
0709 13:02:17.721222 (+466654us) spinlock_profiling.cc:243] Waited 7.91 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:17.721438 (+   216us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.721855 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.721885 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.722432 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.389709 (+667277us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:18.390053 (+   344us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.420445 (+ 30392us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.420549 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.682478 (+261929us) raft_consensus.cc:1685] finished
0709 13:02:18.682597 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.682797 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39625,"prepare.queue_time_us":701,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":32700,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":40417,"spinlock_wait_cycles":18201984,"thread_start_us":666824,"threads_started":1,"wal-append.queue_time_us":957594}
I20260709 13:02:18.602432 10194 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:18.693384 10239 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:18.640444 10083 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:18.652284 10300 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:18.723444  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 57399.6us
I20260709 13:02:18.724737  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:18.727095  8681 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:18.728394  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:18.752065  8830 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.002s	sys 0.042s
W20260709 13:02:18.755609  8830 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.002s	sys 0.042s
I20260709 13:02:18.767584  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:18.763983  8579 catalog_manager.cc:5697] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 2 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:18.777762  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:18.731856 10131 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:18.783106  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 483637us
W20260709 13:02:18.751782 10278 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.773s	user 0.009s	sys 0.005s
I20260709 13:02:18.734404 10082 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 13:02:18.818940  8827 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:18.820681 10082 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:18.821362 10082 raft_consensus.cc:697] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:18.739372 10287 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:18.765861 10237 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:18.849951  8575 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43478 (request call id 253) took 4253 ms. Trace:
W20260709 13:02:18.795640 10258 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.017s	sys 0.042s
I20260709 13:02:18.782259 10228 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:18.857982  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:18.854480  8646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.015s	sys 0.029s
W20260709 13:02:18.863692  8646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.016s	sys 0.029s
I20260709 13:02:18.851056  8575 rpcz_store.cc:276] 0709 13:02:14.596665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.599332 (+  2667us) service_pool.cc:224] Handling call
0709 13:02:18.849838 (+4250506us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:16.873293 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:16.873863 (+   570us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:16.873898 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:02:16.873914 (+    16us) tablet.cc:662] Decoding operations
0709 13:02:16.874490 (+   576us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:16.874575 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:16.874602 (+    27us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:02:16.932402 (+ 57800us) tablet.cc:701] Row locks acquired
0709 13:02:16.940889 (+  8487us) write_op.cc:260] PREPARE: finished
0709 13:02:16.948773 (+  7884us) write_op.cc:270] Start()
0709 13:02:16.958741 (+  9968us) write_op.cc:276] Timestamp: P: 1783602136945471 usec, L: 0
0709 13:02:16.960993 (+  2252us) op_driver.cc:348] REPLICATION: starting
0709 13:02:16.993704 (+ 32711us) log.cc:844] Serialized 1417 byte log entry
0709 13:02:17.972275 (+978571us) op_driver.cc:464] REPLICATION: finished
0709 13:02:18.435900 (+463625us) write_op.cc:301] APPLY: starting
0709 13:02:18.456919 (+ 21019us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:18.457095 (+   176us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:18.457243 (+   148us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:18.457829 (+   586us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:18.458023 (+   194us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:02:18.458083 (+    60us) write_op.cc:312] APPLY: finished
0709 13:02:18.458341 (+   258us) log.cc:844] Serialized 35 byte log entry
0709 13:02:18.458551 (+   210us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:18.458678 (+   127us) write_op.cc:454] Released schema lock
0709 13:02:18.458723 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:18.459140 (+   417us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:18.459175 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":962396,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":422042,"apply.run_cpu_time_us":2420,"apply.run_wall_time_us":40221,"num_ops":3,"prepare.queue_time_us":88992,"prepare.run_cpu_time_us":17191,"prepare.run_wall_time_us":215528,"raft.queue_time_us":856747,"raft.run_cpu_time_us":21129,"raft.run_wall_time_us":215868,"replication_time_us":1011039,"thread_start_us":847781,"threads_started":4,"wal-append.queue_time_us":111024}]]}
I20260709 13:02:18.736431 10300 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:18.865630  8647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.004s	sys 0.043s
W20260709 13:02:18.874502  8647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.004s	sys 0.044s
W20260709 13:02:18.874990  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 545634us
W20260709 13:02:18.876993  9355 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.005s	sys 0.014s
I20260709 13:02:18.878801  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:18.830241 10296 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:18.882443  9353 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:02:18.864528 10258 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.017s	sys 0.042s
I20260709 13:02:18.887498  8751 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
W20260709 13:02:18.879347  9355 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.006s	sys 0.014s
I20260709 13:02:18.824600 10286 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:18.893505  9388 raft_consensus.cc:2368] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3.
I20260709 13:02:18.895018  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:18.863155 10210 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:18.847426  8681 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:18.878333 10194 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:18.912696  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.020s	sys 0.030s
W20260709 13:02:18.913976  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.020s	sys 0.030s
I20260709 13:02:18.877519 10083 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:18.893595  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:18.895869 10207 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:18.852895 10287 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:18.822365 10082 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:18.897686 10074 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:18.929327  9352 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.010s	sys 0.014s
I20260709 13:02:18.903256 10327 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:18.957120  9352 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.010s	sys 0.018s
W20260709 13:02:18.961917  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.909241  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:18.964007  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 567290us
W20260709 13:02:18.988425  8794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.008s	sys 0.020s
W20260709 13:02:18.988938  8794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.008s	sys 0.020s
I20260709 13:02:18.928580 10154 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:18.924561 10255 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:19.001509  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:19.007614  8680 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
W20260709 13:02:18.969866 10301 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:02:18.951541  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:18.916373 10320 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:18.960584 10278 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 13:02:18.705194  8821 rpcz_store.cc:276] 0709 13:02:16.992533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.995200 (+  2667us) service_pool.cc:224] Handling call
0709 13:02:17.069771 (+ 74571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.090805 (+ 21034us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.095195 (+  4390us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.403152 (+307957us) log.cc:844] Serialized 556 byte log entry
0709 13:02:18.384611 (+981459us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:18.394398 (+  9787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.414670 (+ 20272us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.690708 (+276038us) raft_consensus.cc:1685] finished
0709 13:02:18.690746 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.690923 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":940054,"threads_started":1,"wal-append.queue_time_us":1168175}
I20260709 13:02:18.915161 10210 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:18.987747  8672 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.005s	sys 0.021s
I20260709 13:02:18.918027 10194 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.096112 10194 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.078706 10326 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:19.097084  8672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.005s	sys 0.022s
I20260709 13:02:19.097538 10326 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:19.008716  8795 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.013s	sys 0.017s
W20260709 13:02:19.099536  8795 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.013s	sys 0.017s
I20260709 13:02:18.916227 10286 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:18.952708 10207 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:18.924844 10083 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:19.018769  9991 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.010s	sys 0.018s
I20260709 13:02:18.996992 10305 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:18.971653 10074 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:19.009883 10159 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:19.010170 10234 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:19.002213 10327 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:19.011857 10195 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:19.132462 10195 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:19.079399  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 55332.8us
I20260709 13:02:19.103695 10323 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.135124 10323 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.135706  9352 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:02:19.136550 10188 consensus_meta.cc:287] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.613s	user 0.004s	sys 0.007s
I20260709 13:02:19.137013 10188 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:19.006971  9384 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.398s	user 0.011s	sys 0.005s
I20260709 13:02:19.011775 10255 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.022984 10154 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:19.092517 10237 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:19.092931 10210 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:19.026304 10277 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:19.026219 10304 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:19.181172  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:19.182415  8678 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:19.104910  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:19.103147 10089 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:02:19.108275  8753 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.003s	sys 0.026s
I20260709 13:02:18.630321  8673 rpcz_store.cc:276] 0709 13:02:17.004960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.009181 (+  4221us) service_pool.cc:224] Handling call
0709 13:02:17.058663 (+ 49482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.083446 (+ 24783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.085311 (+  1865us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.180109 (+ 94798us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.590781 (+410672us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:17.986342 (+395561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.029976 (+ 43634us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.498365 (+468389us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b54002934a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e246edbe 00007f32e2479930 00007f32e24798d8 00007f32e2479858 00007f32e2479813 00007f32e2478a9c 00007f32e8364626 00007f32e835edd8 00007f32e24b57cb 00007f32e24b82c0
0709 13:02:18.499595 (+  1230us) raft_consensus.cc:1685] finished
0709 13:02:18.502810 (+  3215us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.569248 (+ 66438us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3108,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":7435,"spinlock_wait_cycles":112208256,"thread_start_us":368327,"threads_started":1,"wal-append.queue_time_us":785014}
I20260709 13:02:19.108484  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:19.187355  8753 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.003s	sys 0.027s
I20260709 13:02:19.122606  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:19.123677 10083 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:19.125461  8796 leader_election.cc:400] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3.
I20260709 13:02:19.190261  8796 leader_election.cc:403] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:02:19.138283 10323 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:19.127889  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:19.173187  8722 leader_election.cc:400] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:19.184115  8826 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:19.068723 10320 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.195035  8722 leader_election.cc:403] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 13:02:19.195267  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 69961us
W20260709 13:02:19.139568 10317 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:02:19.071400 10290 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.180928 10154 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:19.201270 10290 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.185153 10237 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.203680 10290 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:19.205260 10237 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.101540 10229 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:19.114398  9991 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.011s	sys 0.023s
I20260709 13:02:19.188938  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 134) took 1236 ms. Trace:
W20260709 13:02:19.167277 10013 consensus_meta.cc:287] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.710s	user 0.010s	sys 0.003s
I20260709 13:02:19.126277  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:19.140698 10329 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.247963 10329 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:19.248591 10329 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.252707  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1186) took 1269 ms. Trace:
I20260709 13:02:19.252940 10195 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:19.253407  8672 rpcz_store.cc:276] 0709 13:02:17.983550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.029193 (+ 45643us) service_pool.cc:224] Handling call
0709 13:02:18.175230 (+146037us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:18.316153 (+140923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.327763 (+ 11610us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:18.431169 (+103406us) log.cc:844] Serialized 52 byte log entry
0709 13:02:19.097688 (+666519us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:19.097739 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.097817 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.252408 (+154591us) raft_consensus.cc:1685] finished
0709 13:02:19.252438 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.252623 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10439,"raft.queue_time_us":23494,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":118,"thread_start_us":620729,"threads_started":1,"wal-append.queue_time_us":702734}
I20260709 13:02:19.198817 10326 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.198691  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:19.256436  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:19.257258  8826 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:19.189741 10304 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.258057  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:19.200088 10225 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.259280 10188 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.260718 10304 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:19.261582 10255 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.189522 10277 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.200438 10074 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:19.263959  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:19.265657  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:19.266418  8826 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:19.187788 10300 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.237620  8722 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:19.244884  8753 rpcz_store.cc:276] 0709 13:02:17.952931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.953575 (+   644us) service_pool.cc:224] Handling call
0709 13:02:17.953913 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.954241 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:18.067492 (+113251us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.672200 (+604708us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b18002c4fa0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c557 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:02:19.188169 (+515969us) log.cc:844] Serialized 14 byte log entry
0709 13:02:19.188446 (+   277us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:19.188527 (+    81us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:19.188595 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.188697 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.188859 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61792,"prepare.queue_time_us":532560,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":43,"raft.queue_time_us":2018,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":22694,"spinlock_wait_cycles":13546112,"thread_start_us":1082322,"threads_started":2,"wal-append.queue_time_us":805698}
I20260709 13:02:19.246711 10013 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:19.269690 10154 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:19.138703 10159 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.272450  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:19.273047 10159 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.274048  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:02:19.149698 10307 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.377s	user 0.009s	sys 0.003s
I20260709 13:02:19.269862 10013 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:19.252667 10246 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:19.141651 10234 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.140403 10207 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.217288 10323 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3.
I20260709 13:02:19.274959 10255 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:19.275223  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1072) took 1225 ms. Trace:
I20260709 13:02:19.281541 10268 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:19.264030 10277 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:19.286662 10278 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:19.192433 10089 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:19.265974 10320 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.285009 10237 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:19.130868 10305 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.297767 10287 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.296484 10330 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.312781  8823 rpcz_store.cc:276] 0709 13:02:18.049949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.070562 (+ 20613us) service_pool.cc:224] Handling call
0709 13:02:18.511089 (+440527us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:18.511403 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.511903 (+   500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.511948 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:18.512594 (+   646us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.925344 (+412750us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:18.953872 (+ 28528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.964759 (+ 10887us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.183658 (+218899us) raft_consensus.cc:1685] finished
0709 13:02:19.183693 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.266598 (+ 82905us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":51506,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":14980,"spinlock_wait_cycles":27527424,"thread_start_us":400778,"threads_started":1,"wal-append.queue_time_us":586411}
I20260709 13:02:19.438735 10330 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:19.439639 10029 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.734s	user 0.009s	sys 0.005s
I20260709 13:02:19.440203 10029 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.284034 10229 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.411429  9380 raft_consensus.cc:2393] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:19.431011 10234 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.431469 10089 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:19.431977 10309 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:02:19.416131 10268 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.436332 10225 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:19.308888 10327 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:19.455389 10225 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:19.455969 10225 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.436784 10305 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:19.439652  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:19.441684  8751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1342) took 1235 ms (client timeout 1589 ms). Trace:
W20260709 13:02:19.460549  8751 rpcz_store.cc:269] 0709 13:02:18.190810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.190967 (+   157us) service_pool.cc:224] Handling call
0709 13:02:19.425179 (+1234212us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b540038b8d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:19.425359 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1420459904}
I20260709 13:02:19.290517 10246 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:19.447343 10302 consensus_meta.cc:287] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.860s	user 0.007s	sys 0.007s
W20260709 13:02:19.469357  8827 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.019s	sys 0.024s
I20260709 13:02:19.423955 10278 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.461696 10229 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:19.472438  8827 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.020s	sys 0.024s
I20260709 13:02:19.476796  9355 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:19.465258 10225 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.478948  9355 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:19.452503 10199 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:02:19.446491 10237 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:19.472594  8823 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:19.453075 10286 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:19.484368  8721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.030s	sys 0.039s
W20260709 13:02:19.488440  8721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.031s	sys 0.039s
W20260709 13:02:19.488710  8719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.012s	sys 0.024s
W20260709 13:02:19.485193  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.015s	sys 0.023s
W20260709 13:02:19.489260  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.016s	sys 0.023s
I20260709 13:02:19.497155 10343 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:02:19.470548 10283 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.026s	sys 0.052s
I20260709 13:02:19.520646  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:19.499266 10283 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.026s	sys 0.052s
I20260709 13:02:19.478528 10061 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:19.526027  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:19.529932  8681 raft_consensus.cc:2410] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:19.498114 10343 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:19.534374  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:19.536223  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:19.465950 10302 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:19.541469  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:19.537132  8680 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:19.540549  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:19.543610  8647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.011s	sys 0.007s
W20260709 13:02:19.543985  8647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.011s	sys 0.007s
I20260709 13:02:19.543541  8674 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:19.545203  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:19.546989  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:19.549847  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:19.551677  8681 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:19.533716 10305 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:19.538988 10336 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:19.563953  8645 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:19.536777 10349 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:19.549593 10159 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:19.553516 10229 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:02:19.554409 10319 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.019s	sys 0.038s
I20260709 13:02:19.588989  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:19.558038  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:19.536911 10061 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:19.564904 10336 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.600464  8680 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:19.575781 10347 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:19.592214 10319 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.020s	sys 0.038s
I20260709 13:02:19.579727 10195 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:19.591073 10234 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:19.569222 10305 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.614141  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:19.619531  8645 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:19.600373 10029 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.601460 10323 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:19.601486 10336 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.619103  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:19.609526  9382 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.003s	sys 0.036s
W20260709 13:02:19.620555 10228 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.007s	sys 0.028s
I20260709 13:02:19.592579 10349 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.632843  8645 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:19.621081 10074 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:19.600623 10229 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.591396 10159 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:19.621512 10268 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.689280  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:19.691534  8756 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:19.692435  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:19.631822 10228 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.007s	sys 0.029s
W20260709 13:02:19.634605  9382 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.003s	sys 0.037s
I20260709 13:02:19.617789 10195 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:19.684675 10206 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.629235 10355 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:19.639060 10234 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:19.725023  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.726601  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:19.702982  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:19.735381 10013 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:19.736078 10013 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.736368  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.736934  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.737579  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.738240 10013 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:19.695415 10278 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.734668  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.739499  8752 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:19.682811 10089 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:19.736281  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:19.632176 10246 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.682900 10322 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:19.728569  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:19.716857  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.687728 10326 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:19.735452 10343 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:19.746905  8747 raft_consensus.cc:2368] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4.
I20260709 13:02:19.623750 10347 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:19.742635  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:19.747864  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.629725 10323 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.751487 10322 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:19.752167 10344 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 13:02:19.752673  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.753314 10323 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:19.753518 10326 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:19.755158  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:19.753327  8748 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:19.750419  8747 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:02:19.754591 10326 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:19.721103 10345 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:02:19.707508  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 295) took 2282 ms. Trace:
I20260709 13:02:19.762842  8756 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 4: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:19.772743  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:19.741796 10206 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.718992  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.725852 10360 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:19.722559 10349 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.720660 10229 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:19.743538 10204 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:19.757414 10159 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:19.787202  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.743212 10131 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:19.715230 10237 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:19.718134 10268 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:19.707793 10199 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.788424 10206 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.748042 10089 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:19.805243  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.800637  8756 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:19.826046  8747 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:02:19.793279 10061 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:19.802769  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:19.838240  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1083) took 1003 ms. Trace:
I20260709 13:02:19.839541  8827 rpcz_store.cc:276] 0709 13:02:18.834704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.834922 (+   218us) service_pool.cc:224] Handling call
0709 13:02:18.835430 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.858909 (+ 23479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.858957 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:18.869628 (+ 10671us) log.cc:844] Serialized 26 byte log entry
0709 13:02:19.475204 (+605576us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:19.475254 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.475331 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.838029 (+362698us) raft_consensus.cc:1685] finished
0709 13:02:19.838055 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.838183 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22746,"raft.queue_time_us":24197,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":115,"thread_start_us":568175,"threads_started":1,"wal-append.queue_time_us":699979}
I20260709 13:02:19.801795 10204 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.843961  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.795538 10360 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.832692 10229 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:19.852891  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.838207 10305 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:19.881644  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.881637  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.883138  8751 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:19.843493  8825 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:19.841246 10199 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.890959  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.893177  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.895690  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.900766  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:19.903959 10363 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:19.843675 10268 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.908195 10199 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.908350 10268 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.911468 10115 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:19.776363  9384 rpcz_store.cc:276] 0709 13:02:17.415593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.417944 (+  2351us) service_pool.cc:224] Handling call
0709 13:02:17.443386 (+ 25442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.472436 (+ 29050us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:19.181192 (+1708756us) log.cc:844] Serialized 14 byte log entry
0709 13:02:19.624638 (+443446us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:19.628997 (+  4359us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:19.633032 (+  4035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.685130 (+ 52098us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.698188 (+ 13058us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1670,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":4938,"thread_start_us":421057,"threads_started":1,"wal-append.queue_time_us":503779}
I20260709 13:02:19.911998 10115 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:19.912437 10115 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.912674  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.913527  8754 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 2: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:19.914044  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.915993  8750 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:19.916052  8751 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:19.921898  8748 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:19.924031  8751 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:19.863801 10229 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.925792  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:19.929502 10229 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:19.929873 10354 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:02:19.923492 10268 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:19.931668 10195 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:19.908195 10204 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.934857 10336 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:19.936210 10329 consensus_meta.cc:287] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.687s	user 0.015s	sys 0.000s
I20260709 13:02:19.939246 10326 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.915443 10347 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:19.943534  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.948839  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.946066 10268 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.944900 10255 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:19.954241  9383 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:02:19.954480 10329 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:19.938588  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 82055.6us
I20260709 13:02:19.956471 10326 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.962344 10347 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:02:19.972258 10305 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:19.976560 10305 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.978539  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:19.980855  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:19.981865  8675 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:19.971036 10239 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:19.982443  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:19.982982 10239 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:19.983470  8672 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:02:19.964768 10318 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.953s	user 0.000s	sys 0.009s
I20260709 13:02:19.964874  9383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 453) took 1084 ms. Trace:
I20260709 13:02:19.960063 10195 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:02:19.966282 10255 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:19.968304 10300 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:19.957481  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:19.992096  9383 rpcz_store.cc:276] 0709 13:02:18.880810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.897430 (+ 16620us) service_pool.cc:224] Handling call
0709 13:02:19.953730 (+1056300us) spinlock_profiling.cc:243] Waited 9.22 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:19.953951 (+   221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:19.964481 (+ 10530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.964769 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21205376}
I20260709 13:02:19.972641  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:19.996090 10255 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:20.002717 10103 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.003290  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:19.996683 10321 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.990s	user 0.019s	sys 0.000s
I20260709 13:02:20.004582  8793 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:20.005434 10329 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:19.985203  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.007597  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.008311  8679 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:20.008860 10239 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:20.009231  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.009490 10239 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:19.985364  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.011514  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.012650 10330 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:19.986347  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.006268  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1071) took 1956 ms. Trace:
I20260709 13:02:19.983441  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.015786  8675 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:20.015911  9381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 449) took 1159 ms. Trace:
I20260709 13:02:20.014599  8828 rpcz_store.cc:276] 0709 13:02:18.049597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.052608 (+  3011us) service_pool.cc:224] Handling call
0709 13:02:18.053019 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.137772 (+ 84753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.137810 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:18.175909 (+ 38099us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.773222 (+597313us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:18.814929 (+ 41707us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.829375 (+ 14446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.829502 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.714433 (+884931us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b18001e7660. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c557 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:02:19.983990 (+269557us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:20.006018 (+ 22028us) raft_consensus.cc:1685] finished
0709 13:02:20.006043 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.006197 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98639,"prepare.queue_time_us":49619,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":112080,"raft.run_cpu_time_us":440,"raft.run_wall_time_us":47358,"spinlock_wait_cycles":50872064,"thread_start_us":479708,"threads_started":1,"wal-append.queue_time_us":686067}
I20260709 13:02:19.989109  9352 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:19.989109  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:19.973877 10229 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:20.035339  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 46331.3us
I20260709 13:02:20.035679 10229 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:20.037644 10342 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.489s	user 0.000s	sys 0.003s
I20260709 13:02:20.037806 10327 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:20.038043 10229 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:19.973673  9355 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:20.038334 10327 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.038818 10327 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:20.038926  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 65352.5us
I20260709 13:02:20.040263 10188 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:20.040673 10188 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.041033 10188 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:20.041509  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:20.044924 10234 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.045625 10234 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.006173 10323 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:20.016472  9381 rpcz_store.cc:276] 0709 13:02:18.856976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.860984 (+  4008us) service_pool.cc:224] Handling call
0709 13:02:20.015638 (+1154654us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b540061b554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:20.015806 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1618571520}
I20260709 13:02:20.017114  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:20.000820  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1074) took 1950 ms. Trace:
I20260709 13:02:20.049324  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:20.048489  8830 rpcz_store.cc:276] 0709 13:02:18.050772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.051685 (+   913us) service_pool.cc:224] Handling call
0709 13:02:18.052136 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.055995 (+  3859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.056045 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:18.134060 (+ 78015us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.759517 (+625457us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:18.759585 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.759694 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.964120 (+1204426us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:20.000485 (+ 36365us) raft_consensus.cc:1685] finished
0709 13:02:20.000522 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.000754 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13579,"raft.queue_time_us":18284,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":19278,"spinlock_wait_cycles":7702784,"thread_start_us":553509,"threads_started":1,"wal-append.queue_time_us":748484}
I20260709 13:02:19.998314 10300 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.050714  8793 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:20.051744 10103 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.052063 10300 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:19.999630  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.052531 10103 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:19.996417 10305 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:19.997327  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.070256  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:20.010505  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:20.064088  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 63717.4us
I20260709 13:02:20.071842 10360 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:20.027997 10336 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:20.074770  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:19.960676 10206 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.089946  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.090301  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1084) took 1251 ms. Trace:
I20260709 13:02:20.091539  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:19.990285 10326 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.070816  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.085626  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.087656 10229 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:20.090958  8829 rpcz_store.cc:276] 0709 13:02:18.839374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.839570 (+   196us) service_pool.cc:224] Handling call
0709 13:02:18.840077 (+   507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.859126 (+ 19049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.859176 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:18.859761 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 13:02:19.141092 (+281331us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:19.141139 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.141210 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:20.037043 (+895833us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:20.054312 (+ 17269us) raft_consensus.cc:1685] finished
0709 13:02:20.054336 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.090194 (+ 35858us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18491,"raft.queue_time_us":23305,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":4751,"spinlock_wait_cycles":7212928,"thread_start_us":280986,"threads_started":1,"wal-append.queue_time_us":661857}
I20260709 13:02:20.109297  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.109357  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:20.095528  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:20.102898  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 63267.2us
I20260709 13:02:20.104336  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.066838 10103 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.073005 10323 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:20.136682  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.128974  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.124926 10103 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.093786 10239 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:20.106940 10013 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.111366 10229 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:20.155328  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:20.159718  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.160965  8679 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:20.159713  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.164865  8824 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:20.167619  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:20.168701  8750 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.176777  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.136114 10204 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.102962 10115 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.134407 10323 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:20.181768  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.187723  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:20.162912 10229 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.191751  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.193208  8824 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:20.190562 10323 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:20.167207 10013 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.199338 10013 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:20.184098 10204 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.201545  8678 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:20.167109 10239 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:20.203702  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.207858  8750 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:20.187323 10376 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.210354 10204 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:20.212692  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.178925 10324 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:20.218530  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 109261us
I20260709 13:02:20.217944  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.189348  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.225492  8793 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:20.226697 10305 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:20.227195 10305 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.227037  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:20.227892 10305 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.228344 10013 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.229496  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:20.231948 10013 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:20.232378 10013 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.232782 10013 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.232993  8793 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:20.243427  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.201965 10206 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:20.246125  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.251398  8793 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:20.243818 10376 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:20.217832 10103 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.272599  8826 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:20.275816 10372 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.276829 10379 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:20.277303 10379 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.277761 10379 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:20.278200 10371 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:20.278483 10188 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.276538 10372 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.281011  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.296566  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.297152 10268 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:20.260526  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.299603  8829 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:20.264005 10234 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:20.242993 10324 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.297683 10188 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:20.300550  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.329361 10347 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.332407 10103 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:20.287161  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.280125 10327 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:20.343381  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 68970.2us
I20260709 13:02:20.352438  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:20.370510  8826 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:20.329933 10013 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.376199  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.381356  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:20.382637  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:20.382256  8751 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:20.377177  8825 raft_consensus.cc:2368] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:20.335922 10305 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.377676  8673 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:02:20.397838  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.399307  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:20.399961  8673 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:20.400290 10324 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:20.401284 10379 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:20.403303 10375 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:02:20.404923  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.405215  8828 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:20.406672  8823 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:20.407615  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.407994 10305 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:20.368521 10347 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.414927  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:20.416721 10367 log.cc:927] Time spent T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Append to log took a long time: real 0.527s	user 0.003s	sys 0.000s
I20260709 13:02:20.416888 10379 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:20.421949 10347 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:20.423027 10324 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.323892 10239 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.423688 10324 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.425976 10324 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:20.423252 10378 log.cc:927] Time spent T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:02:20.430368 10326 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.373618 10325 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.434593 10013 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:20.438910  8751 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:20.446440  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.447894  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.382252 10380 consensus_queue.cc:1059] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
W20260709 13:02:20.450446  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 81411.9us
I20260709 13:02:20.452824  9353 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:20.453006  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.454542  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.455993 10387 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:20.457058 10089 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:20.457188  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.458006  9382 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:20.459420 10322 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:20.460493 10380 raft_consensus.cc:3055] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:20.460904 10380 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:20.461221  9382 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:20.452188 10325 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.463603 10327 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:20.464991 10372 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:20.465626  8749 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:20.465858  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.418697 10115 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:20.466835  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1293) took 1442 ms. Trace:
I20260709 13:02:20.463641  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.469121  8681 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:20.465529  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.470232  8796 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:20.461833 10380 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.473155 10302 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:20.469508  8822 rpcz_store.cc:276] 0709 13:02:19.025074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.025264 (+   190us) service_pool.cc:224] Handling call
0709 13:02:19.078658 (+ 53394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.262639 (+183981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:19.262681 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:19.262740 (+    59us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:19.262782 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.466542 (+1203760us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54002189d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:20.466602 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.466765 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422483712}
I20260709 13:02:20.480598  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.481727  9383 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:20.483204  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.478410 10089 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.491444 10302 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:20.492304 10302 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.467841 10323 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.519620  8826 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:20.492525  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.535461  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:20.533232  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 80509.6us
I20260709 13:02:20.538286  9353 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:20.533708  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.541005  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:20.541987  8680 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.472041 10013 consensus_queue.cc:1059] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:02:20.549904  8747 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:20.556007  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.551080  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.551220  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.551795  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.552378  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.553187  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.558163  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:20.553527  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.554092 10234 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.566064  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.566504  8680 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:20.559152  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.562282  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:20.566833  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.564432  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.567608  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:20.566496  8830 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:20.556455  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.572477  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:20.573516  8750 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.533684 10391 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.494849  9381 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:20.596838  9384 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:20.527765  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.495827 10372 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.565850  8645 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:20.554306  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.581914  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.582566 10380 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.605738  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:20.585495 10379 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:20.586480 10391 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.586668  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.588060 10229 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.592692  8673 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:20.594188  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:20.547711 10327 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:02:20.628032  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.596333  9352 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:20.540112 10320 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.549127 10387 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:20.576148  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:20.600615  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:20.643519  8754 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:20.600975  9355 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:20.601725  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:20.606735  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.644845  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.645170  9388 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:20.562929 10206 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:20.623164  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:20.646350  8681 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:20.624575  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 66513.8us
I20260709 13:02:20.595181 10302 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:20.618988 10194 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:20.619974 10246 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 13:02:20.657155  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 60916.6us
I20260709 13:02:20.639209  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.620549 10379 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.612048  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:20.605141 10324 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:20.577761 10234 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.639670  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.641363  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:20.660966  8751 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:20.642546  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:20.652954  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.642524  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:02:20.656997  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 56107.4us
I20260709 13:02:20.611233 10372 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.621527 10195 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 13:02:20.771329 10207 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:20.803727  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 190131us
I20260709 13:02:20.665311  9381 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:20.727573  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.793480  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:20.681488  9353 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:20.677332  8750 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:20.725025  8751 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:20.842250  9352 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:20.784941  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.634393  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:20.708606  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 156556us
I20260709 13:02:20.839699  8678 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:20.845649  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:20.658855 10327 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:20.840875  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 159556us
I20260709 13:02:20.848577  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:20.848758  8681 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:20.849825  8826 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:20.660003 10194 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:20.660177 10322 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:20.848577  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.853224  8829 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:20.699651 10295 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:20.660482 10379 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.854596 10380 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.699112 10286 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:20.714383 10329 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.791309 10229 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:20.819859 10234 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:20.713914  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.764225 10397 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:20.792116 10391 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:20.818356 10324 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:20.873579 10397 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.863796 10295 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.874958 10324 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.875828  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:20.876068 10397 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:20.876775 10295 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:20.853848  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.660297 10388 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.879077  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.762820 10323 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:20.657469 10320 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.855449 10305 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:20.858201  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.845180 10131 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:20.776386 10395 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:20.850275  8793 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:20.862536  8794 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:20.734872 10386 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.128s
I20260709 13:02:20.847174 10218 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.659405 10206 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.865842  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:20.888597  9352 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:20.865128 10322 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:20.880210  8821 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:20.890540 10390 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:20.891093 10390 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:20.891485 10390 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:20.892521 10206 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:20.865314 10380 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.894894 10194 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.896399 10380 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:20.897977 10218 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.898788 10326 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:20.898561 10218 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.900476  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.901013 10320 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:20.901466  8680 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:20.902493 10372 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:20.903138  8679 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:20.903522  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.901126 10379 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.905072  8752 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:20.906090 10218 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:20.907411  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.907724  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.908177  9384 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:20.908335  8829 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:20.910300  9355 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:20.871939  9388 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:20.871304 10286 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:20.886438  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:20.868825  9353 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:20.838973 10207 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.883040  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.885131  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:20.886660  8794 leader_election.cc:400] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4.
I20260709 13:02:20.893762 10395 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.877367  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:20.882766  8681 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:20.927897  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.881179  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:20.929764  8678 raft_consensus.cc:2393] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 4: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:20.913192  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:20.870744 10329 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:20.913393  8796 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:20.940657  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.924237  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:20.942373  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:20.943295  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:20.929415  8681 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:20.888666 10397 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:20.945170  8829 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:20.868386  8826 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:20.929438  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:20.877697 10188 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:02:20.936331  8680 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:20.913324  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 45726.4us
I20260709 13:02:20.888391 10229 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:20.947453  8828 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:20.927922 10246 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:20.930781 10089 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:20.910858 10295 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.995144  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.931037 10369 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:20.935626  8673 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:20.997180  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:20.999172  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:21.000378  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:20.884249 10388 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.007450  8794 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:21.013809  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:21.016835 10391 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.024775  8673 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:21.024305 10322 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.025676 10207 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:21.026552  8673 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:21.002529 10229 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:20.987126 10380 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:21.035828  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:21.038035 10286 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:21.040752  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:20.958106 10305 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:21.043903 10305 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:21.044524 10305 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.031956 10329 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:20.951699 10397 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:20.993211 10376 consensus_queue.cc:1059] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:02:20.987833 10401 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:20.996160 10246 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:21.057188  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 43510.5us
I20260709 13:02:21.058573  8794 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:20.959156  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.031952 10229 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:20.940191 10268 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.113s
I20260709 13:02:21.049646 10396 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:21.051915 10089 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:21.050513 10397 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:21.049335 10369 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:21.080387  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:21.120803 10380 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:21.121374 10380 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:21.121855 10380 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.123845 10369 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.128602 10388 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:21.130777  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:21.130851  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.135922 10395 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:21.095250 10013 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:21.137377 10013 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:21.137787 10013 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:21.141404 10379 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:21.142935 10218 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:21.143373 10218 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:21.144217 10218 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.151443  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:21.065586 10246 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:21.153980  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:21.154722  9353 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:21.156337  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 76122us
I20260709 13:02:21.157063  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:21.158373  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:21.170159  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1109) took 1083 ms. Trace:
I20260709 13:02:21.170670  8830 rpcz_store.cc:276] 0709 13:02:20.089627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.217618 (+127991us) service_pool.cc:224] Handling call
0709 13:02:21.169881 (+952263us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b540016bf54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:21.170072 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":784598912}
I20260709 13:02:21.171942  8794 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:21.176581  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:21.177775  8674 raft_consensus.cc:2368] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:21.077967 10323 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:21.182147  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 243364us
I20260709 13:02:21.189913  8794 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:21.191681  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1100) took 1107 ms. Trace:
I20260709 13:02:21.192164  8827 rpcz_store.cc:276] 0709 13:02:20.083793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.179081 (+ 95288us) service_pool.cc:224] Handling call
0709 13:02:21.187144 (+1008063us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b5400127354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:21.191600 (+  4456us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":842565888}
I20260709 13:02:21.194455  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:21.196178  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:21.132740 10325 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:21.205092  8794 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:21.209812  8794 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:21.096210 10396 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:21.088064  8825 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:21.168066 10218 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.172173 10381 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:21.179190 10229 raft_consensus.cc:2764] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:02:21.221310  8794 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:21.172344 10336 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.193536 10376 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:21.250415  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:21.201406 10395 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.259707  8647 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:21.266479  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:21.227490 10396 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:21.208041 10323 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:21.231048 10103 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.227169 10403 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.215289  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:21.225669 10401 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.225046  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:21.247471 10204 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:21.200919 10379 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.082s
I20260709 13:02:21.246328 10229 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.274751 10286 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:21.288093  8676 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:21.275177 10194 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.276860  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:21.274480 10326 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:21.299805  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 107872us
I20260709 13:02:21.252095 10206 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:02:21.297894  8672 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:21.301246 10286 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:21.331290  8679 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:21.305428 10410 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:02:21.340153  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 98246.1us
I20260709 13:02:21.338555  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1208) took 1058 ms. Trace:
I20260709 13:02:21.341249 10410 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.342844 10379 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:21.343595  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:21.341447  8672 rpcz_store.cc:276] 0709 13:02:20.274760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.277269 (+  2509us) service_pool.cc:224] Handling call
0709 13:02:21.200499 (+923230us) spinlock_profiling.cc:243] Waited 9.91 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:21.333013 (+132514us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22791936}
I20260709 13:02:21.301556 10204 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.355686  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:21.299175 10089 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:21.322901 10403 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:21.360556  8647 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:21.365918  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 145210us
I20260709 13:02:21.347196 10379 consensus_queue.cc:1243] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:02:21.372275 10204 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:21.317384 10390 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.378239  8794 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:21.382718  8647 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.535s (SENT) [suppressed 9 similar messages]
I20260709 13:02:21.401085  8794 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:21.361778  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:21.384052 10194 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.362253 10403 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:21.411754  9388 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:21.398437  9991 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:21.374482 10380 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.406991 10103 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.356467 10323 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:21.431658  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 49014.8us
I20260709 13:02:21.434258 10013 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:21.383873 10286 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.440639 10286 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:21.411619 10429 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:02:21.441999 10429 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:21.442338 10429 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:21.411610 10229 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.438233 10305 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.459717  8647 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:21.471933  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 70997.1us
I20260709 13:02:21.472957  8794 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:21.454452 10013 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:21.477540  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:21.480376 10410 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:21.481436  8673 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:02:21.481900 10239 raft_consensus.cc:3055] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:21.497184 10239 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:21.498302 10013 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:21.499881 10425 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:02:21.501530 10390 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:21.498338 10239 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.511008 10239 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.480468 10302 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:21.512002  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.475580 10430 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:21.512853 10430 raft_consensus.cc:697] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:21.513856 10430 consensus_queue.cc:237] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:21.484609  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:21.509891 10013 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.481841 10429 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:21.516314  9352 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:21.527812  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.531983  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.532923  8828 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:21.535454  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.537664  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.540534  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.543047  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.547472  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.530050  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1226) took 1108 ms. Trace:
I20260709 13:02:21.548550  8677 rpcz_store.cc:276] 0709 13:02:20.421951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.639357 (+217406us) service_pool.cc:224] Handling call
0709 13:02:21.529758 (+890401us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b54002413d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:21.529963 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":755562368}
I20260709 13:02:21.489926  8673 pending_rounds.cc:85] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 4
I20260709 13:02:21.550287  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:21.585940  9352 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:21.587957 10322 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.579119 10246 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.588573  8753 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.579064 10429 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:21.590584 10246 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:21.593391 10411 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.446s	user 0.002s	sys 0.001s
I20260709 13:02:21.592128  9384 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:21.595876  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:21.596671  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:21.601616 10305 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:21.603241 10380 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:21.599773  8828 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:21.604792  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.583127  8673 pending_rounds.cc:107] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:02:21.596839  9381 raft_consensus.cc:2393] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:21.576514 10369 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:21.485991 10397 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:21.627930  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:21.463337 10403 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:21.633919  8830 raft_consensus.cc:2393] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 4: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:21.637743  8830 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:21.623072  8673 raft_consensus.cc:2889] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:21.640013  8829 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:21.642526  8827 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:21.627930 10369 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:21.724000  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51694.2us
I20260709 13:02:21.730677  8828 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:21.735342  9355 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:21.743549  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.745318  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.745386  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.746279  8830 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:21.645774 10369 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:21.722148 10355 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.012s	sys 0.020s
W20260709 13:02:21.761593  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 140691us
I20260709 13:02:21.761749  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.749106 10218 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:21.763876 10355 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.012s	sys 0.020s
I20260709 13:02:21.783567  8829 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:21.789604 10397 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:21.767603 10388 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:02:21.771833 10218 raft_consensus.cc:697] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:21.793045 10218 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:21.796283  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.796000 10386 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:21.804644 10396 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:21.807914 10396 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:21.809892 10390 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:21.817903  8822 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:21.819999  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:21.820639  8828 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.842748  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.844677  8824 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:21.846814  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:21.850978  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.851843  8825 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:21.854104  9991 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:21.854735  9991 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:21.849261  8828 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:21.867142  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:21.867547  9382 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:21.868623  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:21.868063  9388 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:21.870321  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.873764  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:21.877058  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.867228  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.881747  8821 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:21.882490  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 175) took 1402 ms. Trace:
I20260709 13:02:21.882982  8678 rpcz_store.cc:276] 0709 13:02:20.479675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.015657 (+535982us) service_pool.cc:224] Handling call
0709 13:02:21.326733 (+311076us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:21.326983 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.327024 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:21.327084 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:21.327132 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.327204 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:21.882326 (+555122us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b54001f9d50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:21.882429 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":714506496}
I20260709 13:02:21.889331  8821 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:21.893033  8828 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:02:21.896344  9388 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:21.872754 10355 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:21.898061  8796 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:21.921271  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:21.901117 10369 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.907598 10365 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:21.923172 10365 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:21.924219 10239 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:21.924450 10369 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:21.924850 10239 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:21.925400 10365 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:21.925921 10365 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.907898  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:21.926993 10407 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.793s	user 0.005s	sys 0.000s
I20260709 13:02:21.927276 10401 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:21.927675  8825 raft_consensus.cc:2410] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:21.927917 10401 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.908149  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.929809  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.912056  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:21.930523  8826 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
W20260709 13:02:21.933346  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.917971  8827 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:21.934080  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.919863  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:21.935298  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:21.899467  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 593537us
I20260709 13:02:21.919114  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:21.904247  8828 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:21.936786  8754 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:21.936689  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:21.933934  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:21.930147 10401 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:21.937858  9388 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 2: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:21.939417  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:21.939666  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:21.939831  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:21.940133  8751 raft_consensus.cc:2368] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3.
I20260709 13:02:21.930017  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.942121  8722 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:21.943130  8754 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:21.937919  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.944779  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:21.944972  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:21.948873  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:21.946197 10268 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.947180  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:21.900727  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:21.951987  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:21.925870 10239 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.953267  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:21.896386 10372 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:02:21.952184  8719 leader_election.cc:400] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:21.955367 10228 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:02:21.965137  8719 leader_election.cc:403] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:21.965907 10228 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:21.966333 10228 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:21.966970  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:21.954365  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:21.951366  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:21.953501 10347 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:21.906015 10355 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.946493 10194 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.951066 10428 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:02:21.961591 10390 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:21.926514 10365 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.961263 10386 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:02:21.982323 10390 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:21.993044 10355 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:21.957228  8722 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:21.944356 10397 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.948246 10422 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:21.972630 10414 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:21.955744 10320 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:21.974444 10228 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:21.976085  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:21.955929 10225 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:02:21.977448 10286 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:22.005196  8748 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:22.005841  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.979276  9353 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:21.980684  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:21.985008  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:21.985263  8794 leader_election.cc:400] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3.
I20260709 13:02:22.011111  8794 leader_election.cc:403] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:02:22.012094  8754 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:21.987445 10365 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:22.003818 10428 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:21.997869  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.002074 10414 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:22.012744  8794 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:21.985845 10206 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:02:22.013888 10228 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:22.024326  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 74870.8us
I20260709 13:02:22.015089 10390 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:21.995587  8722 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:22.015365  9991 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:22.013904  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:22.015090  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:22.015990 10305 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3.
I20260709 13:02:22.020896 10414 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.016215  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:22.025161  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 58291.1us
I20260709 13:02:22.062812  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:22.063757 10428 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:22.064218 10428 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:22.064589 10428 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:22.027580  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:22.066887  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:22.067456  9991 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:22.068243 10225 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:22.068275  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:22.077379 10428 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:22.078157 10422 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:22.078641 10422 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:22.078999 10422 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:22.079834 10428 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:22.083942  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:22.086580 10412 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:22.085354  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:22.088198 10412 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:22.088582 10412 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:22.088945 10412 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:22.099542  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:22.103374  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:22.106951  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:22.030133  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:22.020537  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 379) took 1106 ms. Trace:
I20260709 13:02:22.005041 10397 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:22.033147  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:22.017994 10386 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:22.036034 10451 consensus_queue.cc:1059] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:22.028179 10452 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:22.025918 10436 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:22.120922  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 90869.9us
I20260709 13:02:22.148916  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:22.123096  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:22.150979 10390 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:22.151237 10410 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:22.151614  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:22.151785  9353 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:22.152902 10390 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:22.153322  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:22.154973  8821 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:22.155194 10397 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:22.155845  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:22.156719 10229 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:22.157138 10229 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:22.157315  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:22.158130  9383 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:22.131404  8748 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:22.147246  8678 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:22.145723  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:22.163551  8796 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:22.148378  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:22.157603 10229 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:22.147430  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 111391us
I20260709 13:02:22.165218  8722 leader_election.cc:400] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:22.170295  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:22.171293  8647 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:22.173019  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:22.154395 10451 raft_consensus.cc:3055] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:22.173797  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:22.174299  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:22.175956  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:22.176312  8750 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:22.181743  8722 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:22.176429  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:22.132053 10254 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:22.183390  8793 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:22.178365  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:22.219448  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:22.221258  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:22.221647  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:22.159543 10325 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:22.218319  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:22.227877  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:22.243023  8674 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:22.247071  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:22.189365  9388 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:22.184258 10450 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:22.183889  8679 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:22.216396 10302 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:22.245468  9385 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:22.216055 10225 raft_consensus.cc:2764] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:02:22.167110 10397 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:22.217047 10410 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:22.137537 10379 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:22.183084  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:22.179229 10451 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:02:22.256619  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 72935us
I20260709 13:02:22.226009 10323 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:22.225756 10239 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:22.226186 10400 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:22.226208 10268 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:22.262735 10302 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:22.131199  9384 rpcz_store.cc:276] 0709 13:02:20.914561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.930272 (+ 15711us) service_pool.cc:224] Handling call
0709 13:02:20.989551 (+ 59279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:21.594349 (+604798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.020239 (+425890us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b540025ce54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:22.020471 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1378449664}
I20260709 13:02:22.250813  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:22.329303  8826 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:22.172072 10395 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:22.177547 10286 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:02:22.283222  8647 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:22.248756 10206 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:22.245523  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:22.294984  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:22.252630 10365 raft_consensus.cc:2764] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:02:22.256702 10401 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:22.173514 10420 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.109s
I20260709 13:02:22.250523 10254 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:22.270900 10397 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.280987 10410 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:22.317853  9383 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:22.351148  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68041.8us
I20260709 13:02:22.373924 10414 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:22.352300  8830 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:22.321110  8672 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:22.329603 10302 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.281634 10422 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:22.322176 10268 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:22.358213 10347 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:22.369267 10254 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.371165 10397 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:22.374121 10410 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:22.351912 10395 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:22.378084  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:22.357753  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 103313us
I20260709 13:02:22.384578  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:22.405894  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:22.406478  8752 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:22.407141  8829 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:58932
I20260709 13:02:22.407927  8829 raft_consensus.cc:606] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:22.409811  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:22.391654 10268 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.390853 10347 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:22.421577  8829 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:22.423785  8829 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:02:22.424494  8829 pending_rounds.cc:85] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 2
I20260709 13:02:22.424808  8829 pending_rounds.cc:107] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:02:22.425112  8829 raft_consensus.cc:2889] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:22.429695 10455 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:02:22.418740  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:22.434196 10453 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:02:22.434902 10206 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:22.438638 10422 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:22.439539 10347 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:22.441078 10461 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:22.443457 10456 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:02:22.448855  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 113566us
I20260709 13:02:22.456547  8747 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:22.412770 10204 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:22.492406  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 236351us
W20260709 13:02:22.494822  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 85131.3us
I20260709 13:02:22.496115  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:22.369578 10451 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:22.502051  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 92888.7us
I20260709 13:02:22.494731 10204 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:22.523774  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:22.530017  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:22.503589  9991 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:22.518576 10460 raft_consensus.cc:3055] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:22.514796 10239 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.521798 10450 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.525360  9380 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:22.556104 10239 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:22.557629 10403 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:22.557917 10476 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:22.530390 10204 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:22.524549 10429 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:02:22.513700 10461 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:02:22.521493 10400 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:02:22.535125  9991 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:22.571583  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:22.549371 10460 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:22.552731 10451 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.620304 10460 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:22.572118 10463 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:22.623337 10463 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:22.628345 10481 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:22.628991 10481 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:22.629472 10481 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.640168 10329 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.640928 10329 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:22.642773 10466 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:22.642825 10414 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:22.645546 10464 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
W20260709 13:02:22.659118  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 169723us
W20260709 13:02:22.667078  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.669631 10320 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:22.670142 10320 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:22.618659  9991 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.616204 10401 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:22.671309 10228 consensus_meta.cc:287] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.637s	user 0.016s	sys 0.000s
W20260709 13:02:22.692417  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 120967us
I20260709 13:02:22.648011  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1300) took 1132 ms. Trace:
I20260709 13:02:22.706911  8822 rpcz_store.cc:276] 0709 13:02:21.515369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.547083 (+ 31714us) service_pool.cc:224] Handling call
0709 13:02:22.613294 (+1066211us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b5400267fd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:22.638698 (+ 25404us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":616814336}
I20260709 13:02:22.569875 10450 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:22.704841 10329 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:22.697911  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:22.675122 10322 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.611692 10469 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:02:22.721119  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:22.732522  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:22.685258 10470 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
W20260709 13:02:22.723950 10471 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:02:22.719461  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:22.694239 10286 consensus_queue.cc:1059] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:02:22.722587 10195 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:22.723333 10395 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:22.727839 10322 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:22.699862 10474 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:02:22.753649 10322 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:22.736127 10482 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:02:22.755414 10450 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:22.713236 10421 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:02:22.758603  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:22.759510 10484 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:22.760043  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:22.761475 10484 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:22.761729 10268 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:22.718350 10229 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.538s	user 0.013s	sys 0.001s
I20260709 13:02:22.698210 10401 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:22.762262 10484 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.763557  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:22.768575  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.765923  8751 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:22.775243 10239 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:22.771603 10254 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:22.767417 10412 consensus_meta.cc:287] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.678s	user 0.014s	sys 0.000s
I20260709 13:02:22.767931  8823 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:02:22.799546 10412 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:22.807971  8823 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:22.808338 10457 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.535s	user 0.004s	sys 0.000s
I20260709 13:02:22.808600 10412 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:22.809947  8823 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:22.797322 10268 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:22.792940  8748 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:22.797559  8747 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:22.799216 10204 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:22.811610 10474 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:22.812057 10474 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:22.811760  9991 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:22.813020 10204 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:22.812980 10474 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:22.814186  9991 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:22.790951 10485 consensus_queue.cc:1059] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:02:22.805917 10254 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:22.815244 10254 raft_consensus.cc:3055] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:22.815434  8754 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:22.815670 10254 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:22.816816 10347 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:22.819950 10379 raft_consensus.cc:993] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: : Instructing follower 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:22.820813 10228 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:22.821228 10228 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:22.814477 10474 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:22.816684 10254 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:22.818357  8681 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:22.824267  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:22.834028  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:22.836360  8828 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:22.837721  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:22.841912  8828 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:02:22.753701  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:22.756861 10410 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:22.831163 10254 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:22.822109 10228 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:22.867890 10410 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:22.869333 10347 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:22.873986  8722 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:22.916419  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:23.143589  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.144565  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.802172 10229 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:22.859701  9991 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.863785  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:23.150821 10428 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:23.163151  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.199361  8750 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:23.200476  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1417) took 1058 ms. Trace:
I20260709 13:02:23.200929  8750 rpcz_store.cc:276] 0709 13:02:22.142050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.320463 (+178413us) service_pool.cc:224] Handling call
0709 13:02:23.198442 (+877979us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b5800083198. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:23.198806 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.200054 (+  1248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.200393 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3747200}
W20260709 13:02:23.220084  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 48773.5us
W20260709 13:02:23.233214  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 469006us
W20260709 13:02:23.272336  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.945174 10410 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:23.306115  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:22.871192 10286 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.318s
I20260709 13:02:23.143781 10369 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.339043 10369 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:23.341028 10462 raft_consensus.cc:1081] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Signalling peer 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:23.341389 10369 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:23.327559  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:23.331056 10323 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:23.342842  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:23.342983 10323 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:23.344102  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:23.345409  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:23.345947 10323 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:23.334256  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:23.335160 10372 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:02:23.335512 10451 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.716s	user 0.014s	sys 0.000s
I20260709 13:02:23.318706  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:23.318819 10428 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:23.331238 10430 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:23.317083 10228 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:23.324914 10486 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:23.348897 10302 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.957s	user 0.013s	sys 0.000s
I20260709 13:02:23.143431 10421 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:23.354420  8750 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:23.289390 10487 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.317435 10379 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:23.390556  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:23.391952  9389 raft_consensus.cc:2393] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:23.369294  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1419) took 1226 ms. Trace:
I20260709 13:02:23.296938  9991 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:23.399181  8750 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:23.380649 10451 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:23.422288  9389 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:23.410357 10397 consensus_meta.cc:287] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.004s	user 0.011s	sys 0.003s
I20260709 13:02:23.405736  9382 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 2: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:23.403765 10401 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:23.440497 10494 consensus_queue.cc:1059] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:02:23.442237  9991 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:23.443622 10469 raft_consensus.cc:3055] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:23.399211 10379 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:23.562069 10490 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:02:23.382089 10430 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:23.562310  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:23.563813  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:23.564330 10430 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:23.409956  8749 rpcz_store.cc:276] 0709 13:02:22.174302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.419798 (+245496us) service_pool.cc:224] Handling call
0709 13:02:22.496151 (+ 76353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.520222 (+ 24071us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:22.546093 (+ 25871us) log.cc:844] Serialized 14 byte log entry
0709 13:02:23.288756 (+742663us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.308913 (+ 20157us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:23.326395 (+ 17482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.354395 (+ 28000us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.366492 (+ 12097us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51287,"prepare.queue_time_us":5971,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":4362,"raft.queue_time_us":59458,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":86,"thread_start_us":534485,"threads_started":2,"wal-append.queue_time_us":241660}
I20260709 13:02:23.565932  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1412) took 1425 ms. Trace:
I20260709 13:02:23.566957  8749 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:23.447834  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1407) took 1343 ms. Trace:
I20260709 13:02:23.566500  8756 rpcz_store.cc:276] 0709 13:02:22.140268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.142803 (+  2535us) service_pool.cc:224] Handling call
0709 13:02:22.143374 (+   571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.247675 (+104301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.247711 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:22.247755 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:22.247786 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.565667 (+1317881us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400139554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:23.565716 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.565864 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239343872}
I20260709 13:02:23.491549  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1404) took 1393 ms. Trace:
W20260709 13:02:23.398047  8825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1336) took 1490 ms (client timeout 1955 ms). Trace:
I20260709 13:02:23.367274 10372 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:23.400636 10483 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.388693 10421 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:23.387006 10302 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:23.576301 10483 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:23.579896 10347 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:23.580533 10347 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:23.581094 10302 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:23.581518 10483 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:23.415387 10486 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:23.582861  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:23.583351  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:23.583925 10486 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:23.584283  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:23.584565 10450 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:23.585619 10379 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:23.585785 10450 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:23.550976 10388 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:23.434461 10451 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:23.437484 10397 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:23.535529 10481 consensus_meta.cc:287] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.898s	user 0.012s	sys 0.005s
I20260709 13:02:23.598843 10451 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:23.559628 10469 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:02:23.420882 10403 consensus_meta.cc:287] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.847s	user 0.000s	sys 0.013s
I20260709 13:02:23.569636  8681 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:23.570291  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:23.573119  8751 rpcz_store.cc:276] 0709 13:02:22.098529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.098715 (+   186us) service_pool.cc:224] Handling call
0709 13:02:22.765654 (+666939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.803204 (+ 37550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.451670 (+648466us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b54002fe9d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:23.491446 (+ 39776us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1532762368}
I20260709 13:02:23.551113 10473 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.572359  8752 rpcz_store.cc:276] 0709 13:02:22.104324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.104447 (+   123us) service_pool.cc:224] Handling call
0709 13:02:22.104782 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.407058 (+302276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.447425 (+1040367us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b540031cc54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:23.447740 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693270272}
W20260709 13:02:23.554270  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1331) took 1659 ms (client timeout 1694 ms). Trace:
I20260709 13:02:23.403048 10487 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:23.558816 10229 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:23.593431 10204 raft_consensus.cc:3055] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:02:23.574456  8825 rpcz_store.cc:269] 0709 13:02:21.908081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.928517 (+ 20436us) service_pool.cc:224] Handling call
0709 13:02:23.330823 (+1402306us) spinlock_profiling.cc:243] Waited 6.16 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:23.392377 (+ 61554us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14167424}
I20260709 13:02:23.596153 10397 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:23.596904 10372 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:23.586321 10450 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.598232 10320 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:23.599272 10481 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:23.896924  8647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.009s	sys 0.032s
W20260709 13:02:23.897660  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.898077  8647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.009s	sys 0.032s
I20260709 13:02:23.593909  8677 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:23.585700  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:23.899768  8673 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:23.901919  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:23.588366  8675 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:23.903079  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:23.903694  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:23.904297  8673 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:23.904539  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:23.904804  8647 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:23.905809  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:23.907048  8673 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:23.906921  8677 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:23.908075  8647 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:23.909045  8673 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:23.918488  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 204) took 1755 ms. Trace:
I20260709 13:02:23.919241  8678 rpcz_store.cc:276] 0709 13:02:22.162564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.163179 (+   615us) service_pool.cc:224] Handling call
0709 13:02:22.163526 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.493515 (+1329989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:23.493566 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.493630 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:23.493677 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.917891 (+424214us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54001f9354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:23.917961 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.918240 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2963621760}
I20260709 13:02:23.587191  8679 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:23.600926  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:23.586784  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:23.936470  8575 catalog_manager.cc:5697] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 2 to 4, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:23.949404  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1422) took 1424 ms. Trace:
I20260709 13:02:23.956312  9352 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:23.950111  8747 rpcz_store.cc:276] 0709 13:02:22.525115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.525943 (+   828us) service_pool.cc:224] Handling call
0709 13:02:22.526218 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.811090 (+284872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.948630 (+1137540us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b54001581d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:23.949124 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623550720}
I20260709 13:02:23.944047  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 168) took 2008 ms. Trace:
I20260709 13:02:23.697293 10496 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:23.648749 10495 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:02:23.682924 10379 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:23.599931 10194 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.368s
I20260709 13:02:23.839715 10460 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:23.708545 10484 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.872s	user 0.014s	sys 0.000s
I20260709 13:02:23.725642 10347 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:23.656831 10421 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:23.830729 10461 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:23.843902 10329 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:02:23.940486  8794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.034s	sys 0.023s
I20260709 13:02:23.965476  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:23.968847 10379 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:23.973558 10502 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.600040 10388 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:24.009119  8830 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1123) took 1749 ms (client timeout 1960 ms). Trace:
W20260709 13:02:24.011855  8794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.034s	sys 0.033s
W20260709 13:02:24.010707  8830 rpcz_store.cc:269] 0709 13:02:22.259223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.957726 (+1698503us) service_pool.cc:224] Handling call
0709 13:02:24.008984 (+ 51258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:23.974332 10461 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:02:23.939117  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 352289us
I20260709 13:02:23.962239  8753 rpcz_store.cc:276] 0709 13:02:21.935397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.935908 (+   511us) service_pool.cc:224] Handling call
0709 13:02:21.936234 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.989057 (+ 52823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.989093 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.989823 (+   730us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.031334 (+ 41511us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:22.283344 (+252010us) log.cc:844] Serialized 14 byte log entry
0709 13:02:22.375496 (+ 92152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.657670 (+1282174us) spinlock_profiling.cc:243] Waited 52.4 ms on lock 0x7b54002ac454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:23.657755 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.657828 (+    73us) raft_consensus.cc:1685] finished
0709 13:02:23.943084 (+285256us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.943665 (+   581us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1000,"prepare.queue_time_us":251232,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":164327,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":12948,"spinlock_wait_cycles":121088512,"thread_start_us":257559,"threads_started":3,"wal-append.queue_time_us":145463,"wal-append.run_cpu_time_us":9503,"wal-append.run_wall_time_us":1181590}
W20260709 13:02:23.812489 10503 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:02:23.972853 10322 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:24.009896 10481 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.025135 10476 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:24.025383 10204 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:24.026060 10225 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:24.043159 10473 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.043375 10225 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.044744 10347 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:24.045110  8674 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:24.046404 10322 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.039927  8579 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43462 (request call id 264) took 7664 ms. Trace:
I20260709 13:02:24.052533 10397 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:24.052870 10498 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:24.053015 10239 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:24.053628 10410 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:24.053879 10498 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.054549 10239 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.054771 10410 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.053532 10305 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:24.057753 10329 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.060400  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1425) took 1249 ms. Trace:
I20260709 13:02:24.060842  8748 rpcz_store.cc:276] 0709 13:02:22.768754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.817946 (+ 49192us) service_pool.cc:224] Handling call
0709 13:02:22.818391 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:22.818779 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.818810 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.848790 (+ 29980us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.353095 (+504305us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:23.353148 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.353238 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.992121 (+638883us) raft_consensus.cc:1685] finished
0709 13:02:23.995269 (+  3148us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.018213 (+ 22944us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":742,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":135,"spinlock_wait_cycles":74368,"thread_start_us":486021,"threads_started":1,"wal-append.queue_time_us":630470}
I20260709 13:02:23.733245 10482 raft_consensus.cc:3055] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:23.735868 10470 raft_consensus.cc:3055] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:23.966523 10496 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.026190 10477 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:23.845840 10428 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.027205 10485 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:24.028255 10486 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:24.029206 10497 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:24.030884 10463 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.031554 10302 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:23.809576 10474 consensus_meta.cc:287] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.986s	user 0.008s	sys 0.008s
I20260709 13:02:24.053514 10397 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:23.967634 10286 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:23.970798 10484 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:24.034551 10450 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:23.942768  8821 rpcz_store.cc:269] 0709 13:02:21.895023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.895686 (+   663us) service_pool.cc:224] Handling call
0709 13:02:21.900529 (+  4843us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:23.554005 (+1653476us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54003dead4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:23.554192 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":450615808}
I20260709 13:02:24.034965 10420 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:24.033067 10467 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:24.041304  8575 catalog_manager.cc:5697] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 3, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 3 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:24.036777 10206 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:24.085600  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:24.166054 10206 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.167119 10195 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:24.168787 10195 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:24.169195 10195 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:24.169299 10206 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.035499 10469 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.038997 10461 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.170835 10469 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:24.171309 10461 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.086731  8575 catalog_manager.cc:5697] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:24.083758 10420 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.082286  8755 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:24.083810 10502 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.178321 10225 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.075840 10103 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:24.074291 10302 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.179965 10103 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.064167 10482 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:24.181627 10302 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.069715  8678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:24.049883  8579 rpcz_store.cc:276] 0709 13:02:16.375577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.375770 (+   193us) service_pool.cc:224] Handling call
0709 13:02:24.039592 (+7663822us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:19.612956 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:19.752940 (+139984us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:19.752985 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:02:19.753009 (+    24us) tablet.cc:662] Decoding operations
0709 13:02:19.753423 (+   414us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:19.753490 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:19.753534 (+    44us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:02:19.753748 (+   214us) tablet.cc:701] Row locks acquired
0709 13:02:19.753761 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:02:19.753912 (+   151us) write_op.cc:270] Start()
0709 13:02:19.754045 (+   133us) write_op.cc:276] Timestamp: P: 1783602139753870 usec, L: 0
0709 13:02:19.754076 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:02:19.754854 (+   778us) log.cc:844] Serialized 533 byte log entry
0709 13:02:20.415870 (+661016us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:20.625956 (+210086us) op_driver.cc:464] REPLICATION: finished
0709 13:02:20.816174 (+190218us) write_op.cc:301] APPLY: starting
0709 13:02:20.816281 (+   107us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:20.816411 (+   130us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:20.816436 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:20.816793 (+   357us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:20.817039 (+   246us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:02:20.817093 (+    54us) write_op.cc:312] APPLY: finished
0709 13:02:20.817339 (+   246us) log.cc:844] Serialized 23 byte log entry
0709 13:02:20.820774 (+  3435us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:20.820898 (+   124us) write_op.cc:454] Released schema lock
0709 13:02:20.820949 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:21.795012 (+974063us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b18002668c0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:02:21.795142 (+   130us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:21.795178 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":386889,"reported_tablets":10,"spinlock_wait_cycles":11136,"child_traces":[["op",{"apply.queue_time_us":189899,"apply.run_cpu_time_us":2086,"apply.run_wall_time_us":18770,"num_ops":1,"prepare.queue_time_us":782939,"prepare.run_cpu_time_us":42103,"prepare.run_wall_time_us":241025,"raft.queue_time_us":207640,"raft.run_cpu_time_us":19385,"raft.run_wall_time_us":209354,"replication_time_us":871699,"spinlock_wait_cycles":9705472,"thread_start_us":932287,"threads_started":4,"wal-append.queue_time_us":91122,"wal-append.run_cpu_time_us":63136,"wal-append.run_wall_time_us":1966889}]]}
I20260709 13:02:24.070535 10477 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.183064  8678 raft_consensus.cc:606] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:24.063961 10372 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 13:02:24.155761 10305 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.075975 10369 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.142190 10498 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.202452  8575 catalog_manager.cc:5697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 2 to 4, term changed from 2 to 3, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:24.205453 10477 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.064579 10470 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:24.076125 10396 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:24.082734 10504 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:24.091396  8672 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:24.073630 10451 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:24.160210  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:24.160456  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.084599 10467 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.159313  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:24.072130  8681 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:24.161273  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:24.071394 10485 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.163512  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:24.075441 10476 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.143668 10347 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:24.165160  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:24.220252  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.164479 10484 consensus_queue.cc:1059] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:24.167271  9382 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
W20260709 13:02:24.175346  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.080333  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:24.042840  8680 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:24.081022 10320 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.073861 10463 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.177196 10195 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.074002 10468 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.155071 10239 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:24.062590  8647 connection.cc:256] RPC call timeout handler was delayed by -1.0070466689999762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1694s for a response
I20260709 13:02:24.071659  8675 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
W20260709 13:02:24.195394  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.178500  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:24.196457 10502 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.161798  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:24.198928 10397 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.164386 10194 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:02:24.162580  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:24.273099  9389 raft_consensus.cc:2393] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 2: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:24.069998 10204 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.274798  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:24.275241 10204 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:24.276026  9388 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 2: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:24.207302 10470 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.278918 10498 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:24.200421 10305 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.200479 10496 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.279779 10470 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.279687 10498 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.281495 10225 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.200999 10322 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.201592 10369 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.282338 10225 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.283713 10329 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.284600 10369 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.077015  9381 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:24.217798  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.226766  8826 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:24.227459 10388 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.229686 10484 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:24.230882  8830 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:24.231278 10103 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.231972  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.300206  8824 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:24.301519  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:24.302606  9385 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.231755 10396 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.303212  8674 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:24.304919 10396 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.232919 10451 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.211660 10504 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:24.233644  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.209218 10410 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.182723 10482 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.284798 10225 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.299866 10379 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:24.316677  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:24.312476  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 356624us
I20260709 13:02:24.285768 10239 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:24.279789 10305 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:24.324185  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 89515.8us
I20260709 13:02:24.306420 10347 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.297433 10484 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:02:24.332587  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 61954.4us
I20260709 13:02:24.340348  8681 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:24.319236 10504 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.345896  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.315414 10397 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:24.332587 10195 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.334443 10482 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:24.348282  8681 raft_consensus.cc:606] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:24.329384 10396 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:24.363363  9388 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:24.357012 10420 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.377686  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.380775  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:24.374109 10469 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:24.382836 10469 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.383831  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:24.384598  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.383845  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:24.385890  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.387004 10396 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.387861 10487 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.389940 10396 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.329770 10468 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.376832 10194 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.383138  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:24.390695 10329 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:24.394375  8681 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:24.359158 10429 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.403950  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:24.365295 10239 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.383167  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:24.407459 10239 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.414464  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:24.414520  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:24.415196  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:24.417456  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.418737 10510 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:24.386461 10469 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.364988 10305 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:24.421376  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.417249  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.424926  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.427851  8755 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
W20260709 13:02:24.430932  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.403095 10497 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:24.419490 10510 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.391479 10397 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.410730 10225 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:24.438364  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:24.440395  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.440856 10305 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.454885  8756 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:24.441816  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.421051 10476 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.443851  8753 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:24.443912 10397 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.445461  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:24.446770  8754 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:24.447063  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:24.447281 10410 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:24.471401 10468 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.472427  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.448071  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:24.473975  8825 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:24.436254 10194 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.454069  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:24.454999 10510 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.426234 10463 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.430932 10286 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:24.457950  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.470122  8821 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:24.457019  8755 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:24.471189  8830 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:24.493090 10451 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.474174  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.480717  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.494341 10485 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.495646  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:24.495998  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.497587  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.480105  9385 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
W20260709 13:02:24.487776  8647 leader_election.cc:336] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.955s (SENT)
I20260709 13:02:24.498772  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.503276  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:24.503959  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 45955.7us
I20260709 13:02:24.505573 10305 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:24.506521 10305 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.508162  8756 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:24.480752 10225 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.411845 10429 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.509960 10305 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.478469 10194 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.502916  8647 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:24.512023  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 54729.4us
I20260709 13:02:24.512696  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.512909 10485 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:24.515347  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.514231  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.516135 10484 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.519598  8747 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:02:24.478717  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:24.524335  8677 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:24.522039  8747 pending_rounds.cc:85] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 4
I20260709 13:02:24.529467  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.531093  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:24.531745 10502 raft_consensus.cc:3055] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:24.532248 10502 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:24.533414 10502 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.498754 10397 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:24.536254 10397 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.537994  8794 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:24.539834  8678 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.542912  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.538479  8796 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:24.525784  8747 pending_rounds.cc:107] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:02:24.548261 10239 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:24.548914  9385 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:24.548302  8747 raft_consensus.cc:2889] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:24.555178  8681 raft_consensus.cc:606] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:24.530376  8749 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:24.548966 10239 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.558228  9381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 389) took 2341 ms. Trace:
I20260709 13:02:24.558759  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.559270 10507 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:24.555709  8681 raft_consensus.cc:614] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:24.570178  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.565948  9352 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:24.567019  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.566104  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:24.573264  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:24.574093  9380 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:02:24.546224 10397 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.577987  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.579475  8750 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:24.547869 10483 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.574555 10239 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:24.585558  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 63949.8us
I20260709 13:02:24.585153  8796 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:24.591738  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.590943  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:24.595451  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.596978  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.597888  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:24.582417 10379 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:24.598659  8821 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:24.598733 10379 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:24.598949 10436 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:24.599354 10379 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:24.597110  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:24.575011 10507 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:24.598116 10430 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:24.580134  8673 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:24.598168  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:24.582556 10225 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.609189  8830 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:24.612673  8752 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:24.599762 10436 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.611296  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.593551 10396 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:02:24.619517 10436 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.607126 10510 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:24.585673 10504 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:24.629550  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.630023  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:24.632040  8674 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:02:24.635010  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 50822.1us
I20260709 13:02:24.636173  8747 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:02:24.643769  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.624301 10484 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:24.644356  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.627564 10510 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:24.621147  8755 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:24.570250  9381 rpcz_store.cc:276] 0709 13:02:22.216626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.225396 (+  8770us) service_pool.cc:224] Handling call
0709 13:02:23.727647 (+1502251us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:23.728039 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.306794 (+578755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.557919 (+251125us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b54001a4854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:24.558089 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":816995968}
I20260709 13:02:24.647814  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:24.652779  8824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1325) took 2806 ms (client timeout 1793 ms). Trace:
W20260709 13:02:24.653263  8824 rpcz_store.cc:269] 0709 13:02:21.846079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.846398 (+   319us) service_pool.cc:224] Handling call
0709 13:02:24.652424 (+2806026us) spinlock_profiling.cc:243] Waited 2.19 s on lock 0x7b540032f854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:24.652660 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5039436544}
I20260709 13:02:24.654582  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1255) took 1076 ms. Trace:
I20260709 13:02:24.641206 10514 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:24.657245  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 119398us
I20260709 13:02:24.658227  9387 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 3. Ops: []
I20260709 13:02:24.657281 10514 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.658932 10514 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.656023 10510 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.660408 10229 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:02:24.657398 10325 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:24.655021  8680 rpcz_store.cc:276] 0709 13:02:23.578316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.587500 (+  9184us) service_pool.cc:224] Handling call
0709 13:02:23.587756 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.248096 (+660340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.654229 (+406133us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b5400046d54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:24.654510 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":892069120}
I20260709 13:02:24.642330 10239 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:24.663968  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.639989  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.649391 10436 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:24.663347  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:24.679173  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 126260us
I20260709 13:02:24.639467 10430 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.683832  8747 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:24.685482  8749 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:24.662883  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:24.687497  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:24.700946  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 131509us
I20260709 13:02:24.679337  8674 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:24.706900  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.707688  8680 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:24.709585 10470 consensus_queue.cc:1059] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:24.711202 10470 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:24.711592 10470 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:24.712617 10470 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.713622 10467 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.714040 10470 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.641650 10507 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.715157 10507 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:24.695024 10436 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:24.716536 10436 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.693045 10239 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:02:24.730333  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.704797 10513 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:24.645598 10483 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.742307  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.744061  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.744082 10103 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:24.744870  8749 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:24.751317  9355 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:24.763607 10450 consensus_meta.cc:287] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.682s	user 0.010s	sys 0.010s
I20260709 13:02:24.764357 10450 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.766191  8679 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:24.768852  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:24.774786  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.784741 10502 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.788429 10379 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:24.789206 10498 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.784874  8676 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
W20260709 13:02:24.792878  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 172674us
I20260709 13:02:24.794027  9353 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:24.775099  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:24.757499  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:24.776641 10483 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.776892  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:24.777825 10522 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:02:24.778802 10450 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.736086 10513 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.781275 10239 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:24.800465  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 49303.5us
I20260709 13:02:24.813604  9352 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:02:24.818985  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 98959.2us
I20260709 13:02:24.801402  8678 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:24.809751 10513 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.822352  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:24.817577 10218 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:24.832279  9353 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:24.833173  8793 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:24.810498  8749 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:24.823194  8719 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:24.837503  8674 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:24.825975 10329 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:24.829810 10470 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:24.846431  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:24.853641 10430 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.858824 10498 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:24.862120  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 49801.4us
I20260709 13:02:24.831009 10518 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.840322 10504 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.849030 10218 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:24.849135 10329 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.844837 10379 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:24.872730  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:24.876262  8821 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:24.884936 10322 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:24.888723  8719 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:24.889570 10429 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:24.889798 10470 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:24.893144 10470 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:24.887617  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:24.895071  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.863240 10467 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:02:24.901504  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 210861us
I20260709 13:02:24.905055  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:24.879906 10329 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.894749 10410 raft_consensus.cc:3055] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:24.863320 10498 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:02:24.864221 10396 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:02:24.916945  8826 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:24.925457  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:24.926569  8827 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:24.927196  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:24.928049  8822 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.910178 10410 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:24.942411 10410 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:24.944060 10225 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.945205 10410 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:24.945026 10225 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:24.946973  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.949530  8719 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:24.951053 10467 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:24.953778 10286 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:24.933991  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:24.960667  8681 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:24.961776  8681 raft_consensus.cc:606] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:24.962087  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.962296  8681 raft_consensus.cc:614] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:24.963258  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:24.963496  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.972453  8681 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff71213103664c0bab63c42773d0880b"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:24.974964  8681 raft_consensus.cc:606] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:24.976393  8681 raft_consensus.cc:614] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:24.979800  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:02:24.987035  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:24.910321 10388 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:02:24.992807  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:24.993783  8821 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:24.998349  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:24.999944  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 58039.5us
I20260709 13:02:24.921693 10473 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:24.977972 10510 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:25.026144  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 121261us
I20260709 13:02:25.027305  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:25.007948 10225 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:25.044323  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.037277 10498 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:25.066305  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 157204us
I20260709 13:02:25.068605  9353 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:25.072950  9353 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:25.055050 10229 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:25.077822  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 82875.3us
I20260709 13:02:25.079305  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:25.115707  8827 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:25.116358  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:25.080960 10388 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:25.119789  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.128810  9381 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:25.128865 10510 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:25.120347  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:25.130224  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:25.135783  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 52818.7us
I20260709 13:02:25.135812  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:25.140334  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:25.120098 10504 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:25.145047  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:25.144446  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:25.148533  8748 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:25.120318 10533 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:25.163776  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.154289  8750 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:25.153357  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1808) took 2373 ms. Trace:
I20260709 13:02:25.140834 10239 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:25.180760  8754 rpcz_store.cc:276] 0709 13:02:22.780180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.819978 (+ 39798us) service_pool.cc:224] Handling call
0709 13:02:22.820257 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.470635 (+1650378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.147709 (+677074us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b540038b3d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:25.153268 (+  5559us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3739292544}
I20260709 13:02:25.184558  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.188860  8722 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:25.149967 10379 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:25.181558 10514 consensus_meta.cc:287] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.003s
I20260709 13:02:25.150715  8752 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:39024
I20260709 13:02:25.194689  8752 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:25.195223  8752 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:25.195465  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.161166 10518 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:25.147032 10504 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:25.151016 10410 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:25.166069  8822 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:25.205538  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.153407  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:25.202314  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 57342.8us
I20260709 13:02:25.206879 10532 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:25.187019 10430 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:02:25.206930 10544 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:25.215785  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:25.215826  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:25.219319  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:25.193962 10379 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.222014  8824 raft_consensus.cc:2376] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2. Re-sending same reply.
I20260709 13:02:25.204972 10504 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:25.228329  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 76272.4us
W20260709 13:02:25.231514  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 223563us
I20260709 13:02:25.220765 10532 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:25.251942  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:25.256889  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:25.241536  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.259298  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.211458 10329 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:25.260941  9355 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:25.261021  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:25.261870  8829 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:25.262732 10329 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:25.263487 10329 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:25.263662  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.265003  8749 raft_consensus.cc:2410] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:25.241869  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:25.245260  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:25.270762  9387 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:25.247015  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.266086  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.249707 10103 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:25.272893 10103 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:25.284096 10507 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:25.237578  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.251451  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.242463  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.296239  8827 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:25.253579 10513 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:25.202828 10514 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:25.255306 10532 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:25.272187  8753 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 2: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:25.247826  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.272073  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:25.266279  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.313202  8674 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:25.225668 10379 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:25.310703  8747 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:25.316275  8752 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:25.315867  8748 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:25.274119 10103 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.315169  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:25.321738  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:25.325564  8827 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:25.326413 10544 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:25.330960  8752 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:25.266389  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.329834 10286 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:02:25.337025  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.338886  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.321154  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.338974  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.340039  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.339697  8756 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 2: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:25.341333  8830 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:25.346165  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.317361 10410 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:02:25.351819  8647 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:25.333097 10429 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.356408  8826 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:25.356613  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:25.335170 10513 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:25.362380  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.342059 10533 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.365267  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.365571  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.366766  8748 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:25.367235  8747 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:25.360726  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.371956  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.376699  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.372902  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.368732 10429 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.381095  8821 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:25.382153 10533 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.382854  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:25.383646  8822 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:25.369125 10498 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.384931 10498 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.386170  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.387480  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.387925  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.388266  8827 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 2: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:25.388718  8755 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 2: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:25.389196  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.391496  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:25.393455  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.394845  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.396251  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.397538  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.359000 10484 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:25.399789 10484 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:25.400331 10484 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:25.401378 10484 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.403484  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.365972 10305 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.404384 10225 raft_consensus.cc:3055] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:25.404899  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.405202 10544 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:25.404974 10225 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:25.405694 10544 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:25.373948  8722 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:25.390370  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.415378 10286 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:25.416042 10286 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.416460 10286 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:25.421152 10523 log.cc:927] Time spent T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.813s	user 0.000s	sys 0.008s
I20260709 13:02:25.374230 10473 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:02:25.404431 10305 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:25.411404  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:25.431200  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1746) took 1285 ms. Trace:
I20260709 13:02:25.431682  8672 rpcz_store.cc:276] 0709 13:02:24.145669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.232228 (+ 86559us) service_pool.cc:224] Handling call
0709 13:02:24.232737 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.233458 (+   721us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:24.320678 (+ 87220us) log.cc:844] Serialized 14 byte log entry
0709 13:02:24.441387 (+120709us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.442456 (+  1069us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.442741 (+   285us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:24.442807 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.442917 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.406360 (+963443us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:25.430828 (+ 24468us) raft_consensus.cc:1685] finished
0709 13:02:25.430941 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.431128 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65920,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":10089,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":278,"spinlock_wait_cycles":14280448,"thread_start_us":147974,"threads_started":2,"wal-append.queue_time_us":205553}
I20260709 13:02:25.416051 10544 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.436372 10544 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:25.436628 10533 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:25.443811 10429 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:25.410598 10225 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.445441 10225 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:25.447016 10239 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:25.447865 10239 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.448838 10498 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:25.453501  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.454085  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.454491  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.454948  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.457332  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:25.459008 10507 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:25.459683 10507 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:25.462036  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 84021us
I20260709 13:02:25.462438 10467 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:25.462913 10467 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:25.463881 10467 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.464434  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:25.464639 10507 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:25.465287 10467 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:25.466840  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:25.468528  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:25.472518  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:25.473868  8679 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:25.476786  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:25.483296  8674 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:25.484192  8829 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:25.492857  8679 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:25.492690  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:25.498479  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:25.502032  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.503203  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.498701  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:25.504534  8678 raft_consensus.cc:2393] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:02:25.505724  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 153993us
I20260709 13:02:25.507225  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:25.511565  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 100535us
I20260709 13:02:25.512358  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:25.509289  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.521198  8829 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:25.521351  8680 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:25.529886  8756 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:02:25.530491  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.537839  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:25.548082  9352 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:25.464470  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:25.547262 10532 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.562951 10229 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:25.564743  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:25.494230  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:25.566298  8794 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:25.572192  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:25.570921 10531 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:25.567756 10502 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:25.574301  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:25.583746 10531 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:25.584125  8680 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:02:25.584261 10556 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:25.584779 10532 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:25.493885 10410 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 13:02:25.586058  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:25.503217 10498 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:02:25.576382  8829 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:25.546725 10484 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:25.565788  8672 raft_consensus.cc:1813] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Handling vote request from an unknown peer e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:25.551847 10533 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:02:25.593703 10430 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:25.584803 10556 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:25.596367 10473 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:25.629642 10430 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:25.594415 10563 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:25.632100 10430 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:25.632364 10563 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.615145 10557 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:02:25.631806  8672 raft_consensus.cc:2410] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:02:25.599179 10502 raft_consensus.cc:697] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:02:25.620857 10555 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.057s	user 0.002s	sys 0.000s
I20260709 13:02:25.631884 10473 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:25.625880 10239 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:25.605728 10484 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.630472 10429 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:25.634454 10550 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.585090 10531 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.634974 10556 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:25.636611 10410 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:25.645567  8822 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:25.637800 10507 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:25.662498  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 96288.7us
I20260709 13:02:25.664077  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:25.661031  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:02:25.665971  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.644280 10473 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:25.677093 10379 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.679044  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:25.691257  8824 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:02:25.692596  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 180337us
I20260709 13:02:25.695060 10502 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:25.695757  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:25.697324 10397 raft_consensus.cc:2764] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:02:25.698204 10397 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:25.695606 10483 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:25.661330 10410 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:25.693789  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:25.690237 10531 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:25.722002  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:25.584918 10470 raft_consensus.cc:2955] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:25.724978  8680 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:25.720494 10305 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:25.726815  9352 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:25.679481 10507 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:25.662412 10550 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.708058 10483 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:25.733412 10541 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:25.761781  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:25.769876  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 74203.9us
I20260709 13:02:25.773887  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:25.779311 10484 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:25.780272  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.780962 10507 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:25.781596 10559 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:02:25.787840  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:25.788529 10550 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:25.788969 10103 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:25.789402 10504 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:25.789932 10379 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:25.790603 10379 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:25.744831 10410 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.791738 10504 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:25.789373  8794 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:25.792629 10410 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:25.793715  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:25.794816 10239 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:25.795008  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:25.795508 10239 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:25.796337  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:25.797636  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:25.803581  8794 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:25.739895  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:25.741515 10561 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.769034  8674 raft_consensus.cc:2955] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:25.744144 10560 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.836273 10560 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.741817 10565 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:25.838852 10565 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:25.839336 10565 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:25.843765 10507 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.844751 10507 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:25.846969 10514 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:25.761850 10573 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.848024 10573 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:25.850237  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 113220us
I20260709 13:02:25.851068  8645 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:25.852026  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:25.853446  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:25.849511 10571 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:25.799714 10305 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:25.855858  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:25.822968  9387 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:25.813544 10396 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:25.823593 10397 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:25.822410  9381 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:25.815873 10429 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:25.801051 10473 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:25.864655 10561 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.855664 10507 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:25.866153 10507 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.866724 10473 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:25.866686 10507 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:25.817199  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:25.815094 10550 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:25.868860 10550 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:25.811789 10541 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:25.844795 10286 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:25.870085 10396 raft_consensus.cc:2764] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:02:25.870589 10541 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:25.871143  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54045.7us
I20260709 13:02:25.872599 10286 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:25.872694  8794 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:25.818593 10532 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:25.875627 10531 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:25.875995  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:25.835484 10566 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:02:25.814863 10225 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:25.873801 10483 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:25.865455  8575 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40096 (request call id 138) took 5724 ms. Trace:
I20260709 13:02:25.869385 10507 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:25.888796 10507 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:25.889520 10507 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:25.889968 10507 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:25.878293  8794 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:25.893631  8794 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:25.894791 10225 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:25.898875 10514 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:25.898993 10225 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:25.869324 10550 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:25.899458 10514 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:25.900204 10571 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:25.901934 10571 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:25.903031 10483 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:25.905783 10561 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:25.873392  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:25.906911  8828 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:25.910274  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.910852  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.898443  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:25.959771  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 61436.5us
W20260709 13:02:25.960937  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 48632.9us
I20260709 13:02:25.906445 10561 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:25.965759  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:25.850455 10573 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:25.876920 10531 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:25.901790 10514 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:25.969815  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 302930us
I20260709 13:02:25.887137  8575 rpcz_store.cc:276] 0709 13:02:20.140898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.141198 (+   300us) service_pool.cc:224] Handling call
0709 13:02:25.865388 (+5724190us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:24.498452 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:24.579735 (+ 81283us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:24.579795 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:02:24.579818 (+    23us) tablet.cc:662] Decoding operations
0709 13:02:24.647238 (+ 67420us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:24.669336 (+ 22098us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:24.669401 (+    65us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:02:24.708740 (+ 39339us) tablet.cc:701] Row locks acquired
0709 13:02:24.708796 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:02:24.708984 (+   188us) write_op.cc:270] Start()
0709 13:02:24.709171 (+   187us) write_op.cc:276] Timestamp: P: 1783602144708917 usec, L: 0
0709 13:02:24.709215 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:02:24.710087 (+   872us) log.cc:844] Serialized 1798 byte log entry
0709 13:02:25.131962 (+421875us) spinlock_profiling.cc:243] Waited 4.09 ms on lock 0x7b54002d8320. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e246edbe 00007f32e2479930 00007f32e24798d8 00007f32e2479858 00007f32e2479813 00007f32e2478a9c 00007f32e8364626 00007f32e835edd8 00007f32e24b57cb 00007f32e24b82c0
0709 13:02:25.327647 (+195685us) op_driver.cc:464] REPLICATION: finished
0709 13:02:25.524807 (+197160us) write_op.cc:301] APPLY: starting
0709 13:02:25.524988 (+   181us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:25.525269 (+   281us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:25.525321 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:25.526450 (+  1129us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:25.527001 (+   551us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:02:25.527142 (+   141us) write_op.cc:312] APPLY: finished
0709 13:02:25.527711 (+   569us) log.cc:844] Serialized 41 byte log entry
0709 13:02:25.564161 (+ 36450us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:25.573923 (+  9762us) write_op.cc:454] Released schema lock
0709 13:02:25.593247 (+ 19324us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:25.637791 (+ 44544us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:25.640985 (+  3194us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":102093,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":196738,"apply.run_cpu_time_us":4440,"apply.run_wall_time_us":102042,"num_ops":4,"prepare.queue_time_us":138848,"prepare.run_cpu_time_us":23939,"prepare.run_wall_time_us":354847,"raft.queue_time_us":194836,"raft.run_cpu_time_us":50704,"raft.run_wall_time_us":177877,"replication_time_us":618257,"spinlock_wait_cycles":9408512,"thread_start_us":351364,"threads_started":4,"wal-append.queue_time_us":54262}]]}
I20260709 13:02:25.911306 10286 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.958709 10532 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:25.965111 10372 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:26.008034  9353 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:02:26.005741  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 244072us
I20260709 13:02:26.008224  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:26.019405  8750 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.020006  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:26.011965  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:26.021170  8752 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:25.968520 10396 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:25.976533 10561 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:25.915679 10560 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:26.027225  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:26.022519  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:26.024823  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 169020us
I20260709 13:02:26.032727  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:26.039836  8754 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:26.039883  8753 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:25.995817 10467 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:26.030815  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:26.045050  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:26.046043 10567 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:26.046110  8793 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:26.046693 10567 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:26.047134 10567 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:26.047160 10329 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:26.047724 10329 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:26.048159 10329 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.052256  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:25.984941 10379 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:02:26.014228 10573 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.062776  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:26.066298  8793 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:26.066936  8829 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.026957 10445 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.033476 10561 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.031605 10510 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:26.071092  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51294.1us
W20260709 13:02:26.069228  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:26.074110  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.041371 10372 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:25.983846 10430 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:26.078261  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 356386us
I20260709 13:02:26.082237  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:26.079353  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.088294  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:26.092444  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:26.095584  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:26.096385  8677 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:26.043908 10560 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:26.093523  8681 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:26.049780  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:26.052192  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:26.099838  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:26.100560  8681 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:26.102279  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:26.104413  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:26.105198  8677 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:26.105892  8828 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:26.106741  8826 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:26.106520  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:26.054075  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:26.065130 10533 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:26.125090  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.266600  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:26.112313  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 44407.3us
I20260709 13:02:26.097450 10584 raft_consensus.cc:993] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: : Instructing follower 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:26.254319  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1360) took 1181 ms. Trace:
I20260709 13:02:26.279937  8828 rpcz_store.cc:276] 0709 13:02:24.926163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.976600 (+ 50437us) service_pool.cc:224] Handling call
0709 13:02:25.431818 (+455218us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:25.873233 (+441415us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4e9c 00007f32df3d4a5b 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:26.107825 (+234592us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51828224}
I20260709 13:02:26.261497  8829 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:26.267719  8676 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:26.102927  8681 raft_consensus.cc:2376] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4. Re-sending same reply.
I20260709 13:02:26.290679  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:26.068563 10576 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.078663 10410 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:26.098304 10510 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:02:26.292999  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 204819us
I20260709 13:02:26.293182  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:26.055848 10467 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.290477  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:26.305469  8793 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:26.051129 10286 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:26.069805 10397 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.215s
I20260709 13:02:26.248237 10577 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:26.319243  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 261457us
I20260709 13:02:26.323606  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:26.071370 10573 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:26.327836  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:26.328788 10305 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:26.331885  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:26.331180  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 284459us
W20260709 13:02:26.427503  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 95708.9us
I20260709 13:02:26.278151 10560 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.430123  8750 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:26.390021 10576 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:26.391687 10470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:39018 (request call id 74) took 7088 ms. Trace:
I20260709 13:02:26.398200  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:26.399175 10575 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:26.403019  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:26.281512  8672 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:26.074946 10445 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:26.408191 10430 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:26.409761 10573 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:26.325717 10544 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:26.322634 10410 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:26.414330 10372 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:26.285542 10473 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:26.389189 10591 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:26.329926 10533 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:26.417816 10541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.000s	sys 0.016s
I20260709 13:02:26.390589 10550 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:26.442816  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:26.431492 10239 consensus_meta.cc:287] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.627s	user 0.008s	sys 0.003s
W20260709 13:02:26.463145  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 151611us
I20260709 13:02:26.501617  8722 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:26.464280  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 59767.7us
I20260709 13:02:26.434010 10229 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:26.427158 10305 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.466910 10573 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.456461 10397 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:26.482406  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:26.471714 10507 consensus_meta.cc:287] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.000s
I20260709 13:02:26.502036  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:26.458566  8755 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:26.468408 10574 raft_consensus.cc:1081] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Signalling peer 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:26.434162 10576 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.441751 10470 rpcz_store.cc:276] 0709 13:02:19.410863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.411785 (+   922us) service_pool.cc:224] Handling call
0709 13:02:19.723698 (+311913us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b2c000f0558. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e0df2a11 00007f32e0e01a0a 00007f32e0e01693 00007f32e0e01631 00007f32e2534902 00007f32e25346dd 00007f32e24fff3f 00007f32e24ff6b3 00007f32e24f79fa 00007f32e74bf4c9 00007f32e0a3bc54
0709 13:02:21.587820 (+1864122us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b580019d6d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24550aa 00007f32e24d7447 00007f32e24f470a 00007f32e24f576a 00007f32e24f491b 00007f32e250003c 00007f32e24ff6b3 00007f32e24f79fa 00007f32e74bf4c9 00007f32e0a3bc54
0709 13:02:21.620237 (+ 32417us) log.cc:844] Serialized 494 byte log entry
0709 13:02:26.391515 (+4771278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122642,"raft.queue_time_us":137830,"raft.run_cpu_time_us":1487,"raft.run_wall_time_us":1485,"spinlock_wait_cycles":2071836800,"thread_start_us":143051,"threads_started":1,"wal-append.queue_time_us":228755,"wal-append.run_cpu_time_us":2445,"wal-append.run_wall_time_us":1157183}
I20260709 13:02:26.520659 10239 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:26.505002 10229 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.533092 10229 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:26.509864  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:26.481750 10225 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.003s
W20260709 13:02:26.484300 10483 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.581s	user 0.012s	sys 0.000s
I20260709 13:02:26.539485 10225 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.541257  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:26.542390 10225 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:26.542894 10470 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.543401 10225 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:26.545766  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:26.496389 10541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.000s	sys 0.018s
I20260709 13:02:26.473982 10430 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:02:26.563197  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 297465us
I20260709 13:02:26.553723  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:26.544391  8750 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:26.556639  8748 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:26.484844 10533 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.572350  8722 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:26.564486  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:26.514516 10305 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.526714 10544 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.575754  8647 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:26.573732  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:26.598460 10581 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.598778  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:26.599617 10581 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:26.600410  8750 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:26.600772  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:26.573581 10591 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:26.588660 10533 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:26.589620 10594 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:02:26.526297 10396 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:26.594105  8754 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.551s	user 0.007s	sys 0.002s
W20260709 13:02:26.525363 10556 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.846s	user 0.003s	sys 0.009s
I20260709 13:02:26.464044 10575 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.602041  8679 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:26.597383  8722 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:26.550973 10576 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.479887 10473 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:26.609440  8754 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:26.570820  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:26.590593 10286 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:26.605870 10581 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:26.611811 10587 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.552269 10507 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.552150 10573 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.558362 10397 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:26.612313 10470 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.495779 10510 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:26.576735 10430 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.637835  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:26.637126  8753 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:26.640673 10470 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.641546 10560 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:26.643081 10470 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:26.643977 10507 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:26.644740 10286 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.645267 10286 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.647697 10286 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:26.632562 10567 consensus_meta.cc:287] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.585s	user 0.011s	sys 0.002s
I20260709 13:02:26.648634 10567 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.614028 10396 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.653143  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:26.653683 10396 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:26.654047  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:26.654718  8677 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:26.653708  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:26.657155  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:26.657344 10581 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:26.660341 10286 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:26.659437 10581 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.658300 10567 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:26.662899 10286 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.663594 10286 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.667328  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:26.633767  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:26.669203  8647 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:26.669835  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:26.670594  8821 raft_consensus.cc:3055] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:26.672879  8821 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:26.673105 10550 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:26.673878  8821 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:26.674335  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:26.675545  8821 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:26.677839 10585 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.672484  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:26.671594  8673 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 4: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:26.680477 10397 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:26.680694  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 1770) took 1212 ms. Trace:
I20260709 13:02:26.681113  8673 rpcz_store.cc:276] 0709 13:02:25.467977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.493683 (+ 25706us) service_pool.cc:224] Handling call
0709 13:02:26.563769 (+1070086us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:26.680599 (+116830us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31072000}
I20260709 13:02:26.681489 10598 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:26.683534  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.671175  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:26.614850 10556 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:26.661744 10571 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.759s	user 0.010s	sys 0.000s
I20260709 13:02:26.672115  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:26.623603 10225 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.674392 10550 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:26.631321 10589 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:26.717994  8679 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:02:26.701248  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:26.701555 10597 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:02:26.703075  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:26.694424 10584 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.704561 10556 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:26.705322 10575 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:26.706002 10563 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:26.706209 10573 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:26.707206 10532 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.685s	user 0.011s	sys 0.002s
I20260709 13:02:26.690989 10585 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:26.771895  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:26.700865 10229 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.671998 10473 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:26.703069 10430 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:26.708822  8675 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:26.794907 10229 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:26.711067  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:26.696472 10571 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:26.799006  8677 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:26.750736  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:26.751663  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:26.809012  8828 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
W20260709 13:02:26.809829  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 47655.4us
I20260709 13:02:26.810493  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:26.752395  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:25.907943 10429 consensus_queue.cc:1175] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7aff04f53d4845a6a4fa491fd2a06145" responder_term: 4 status { last_received { term: 3 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:02:26.757937  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 86878us
I20260709 13:02:26.700301 10410 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:26.860249  8681 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:26.764099  8564 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 51388ea026cc4d098949a4a0327a35fc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:02:26.863127  8564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 118790us
I20260709 13:02:26.863602  8825 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.863709  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:26.849931  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:26.857604 10550 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.867403 10550 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:26.768028  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:26.671406 10445 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:26.854928  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:26.744737 10589 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.773393  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:26.874231 10589 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:26.768428  8673 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:26.768841 10329 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.720s	user 0.011s	sys 0.002s
I20260709 13:02:26.771000 10286 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.784865  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:26.787760 10305 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.791128  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:26.878961 10445 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:26.791145 10592 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.383s	user 0.001s	sys 0.004s
I20260709 13:02:26.803445  8675 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:26.810971  8722 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:26.872013  9389 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:26.883985  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 73084.2us
I20260709 13:02:26.885663  8722 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:26.856745  8719 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:26.850759  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:26.853361  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:26.877565  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 71956.4us
W20260709 13:02:26.805754 10565 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.966s	user 0.008s	sys 0.006s
I20260709 13:02:26.891081  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:26.771895  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:26.892107  8827 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:26.771687 10532 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:26.806486 10372 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:26.769380 10560 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.857829 10575 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.771858 10585 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:26.782464 10544 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:26.892377  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:26.800388 10533 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:26.989065  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:27.073073  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:26.894593 10473 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:27.047638 10571 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:27.086990  9381 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 2: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:26.906224  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1279) took 1415 ms. Trace:
I20260709 13:02:26.804239  8821 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:27.047425 10591 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:26.811757  9386 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
W20260709 13:02:27.033279  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:26.987574  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 75903.6us
I20260709 13:02:27.123373  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:26.995162  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:27.126022  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:27.127377  8754 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 2: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:27.053170 10550 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:27.093624  8679 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.884702 10329 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.155791  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:27.159333  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:27.084802 10397 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.160004  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:26.890017 10410 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:27.159693  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:26.895061 10563 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:26.918176 10561 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.806s	user 0.009s	sys 0.003s
I20260709 13:02:26.895358 10470 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.996157 10467 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:26.894924  8822 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:27.161586  8753 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
W20260709 13:02:27.179979  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.900719 10598 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.225533  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:27.139626  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:27.157982  8678 rpcz_store.cc:276] 0709 13:02:25.491168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.532750 (+ 41582us) service_pool.cc:224] Handling call
0709 13:02:25.743960 (+211210us) spinlock_profiling.cc:243] Waited 3.98 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:25.761952 (+ 17992us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.016505 (+254553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.016542 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:26.033931 (+ 17389us) log.cc:844] Serialized 26 byte log entry
0709 13:02:26.290067 (+256136us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:26.290126 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.292495 (+  2369us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.782174 (+489679us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:26.905882 (+123708us) raft_consensus.cc:1685] finished
0709 13:02:26.905918 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.906131 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72278,"raft.queue_time_us":78317,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":134,"spinlock_wait_cycles":14996096,"thread_start_us":376679,"threads_started":2,"wal-append.queue_time_us":352949}
I20260709 13:02:26.992157  8825 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:27.170533  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:27.258498  9388 raft_consensus.cc:2393] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:27.094784 10585 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:26.895529 10569 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:27.237331  8824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1369) took 2119 ms (client timeout 2121 ms). Trace:
W20260709 13:02:27.318459  8824 rpcz_store.cc:269] 0709 13:02:25.118143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.262988 (+144845us) service_pool.cc:224] Handling call
0709 13:02:27.236965 (+1973977us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b5400127354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:27.237239 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":961636608}
I20260709 13:02:26.903558 10286 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:27.052847 10510 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.094265 10473 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:27.137796 10591 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.000209 10305 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:27.158161 10550 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:27.262513  8830 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:27.262682  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:27.317140  8828 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:27.110329 10372 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.110594 10560 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.899520 10225 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.500059  8750 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
W20260709 13:02:27.240815 10576 consensus_meta.cc:287] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.612s	user 0.011s	sys 0.002s
I20260709 13:02:27.332593  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:27.069391 10445 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.209s
I20260709 13:02:27.505625 10445 raft_consensus.cc:3055] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:27.506132 10445 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:27.229074 10397 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:27.335943  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 178856us
I20260709 13:02:27.508080  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:27.230603 10606 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.342703  9389 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:27.509255 10397 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:27.343796  8679 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:27.510226 10570 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.349177  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:27.518345  8794 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:27.365839 10613 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 13:02:27.156093 10544 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:27.376484  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1151) took 2012 ms. Trace:
I20260709 13:02:27.376581 10473 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:27.497017  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 604701us
I20260709 13:02:27.349570 10591 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:27.373558  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:27.357650  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:27.494164 10582 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.512472 10556 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:27.513213 10606 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.133863 10575 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.248638 10561 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.256083 10532 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.263573 10598 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.514212 10570 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.258802 10470 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:26.896973 10565 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.524555  8826 rpcz_store.cc:276] 0709 13:02:25.364485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.407811 (+ 43326us) service_pool.cc:224] Handling call
0709 13:02:25.907071 (+499260us) spinlock_profiling.cc:243] Waited 7.85 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:26.106508 (+199437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.107124 (+   616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.376105 (+1268981us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54003abbd0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:27.376357 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":476116608}
I20260709 13:02:27.353849 10498 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.360179 10329 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:27.353973 10603 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:27.529572  8756 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:27.329248 10581 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:27.493011 10396 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:27.535753  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:27.335783 10510 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.528489 10576 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.656298 10396 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:27.652549  8719 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.632s (SENT) [suppressed 15 similar messages]
I20260709 13:02:27.657807 10576 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:27.368105 10533 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:27.521989 10599 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:27.540194 10286 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:27.542301  8576 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43482 (request call id 226) took 10317 ms. Trace:
I20260709 13:02:27.627521 10591 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.543603 10467 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:27.329003 10569 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.521078 10225 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:27.234828 10563 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:27.586148  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 309474us
I20260709 13:02:27.567641 10532 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.553421 10410 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.647967  8754 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:27.553805 10556 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:27.643383 10498 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.623243 10445 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.352874  8673 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:27.159937 10571 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.658968  8829 raft_consensus.cc:2368] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5.
I20260709 13:02:27.565126 10598 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.650068  8751 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
W20260709 13:02:27.663695  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.718650  8753 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:27.532366 10544 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:27.678840  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 340832us
I20260709 13:02:27.686228  8576 rpcz_store.cc:276] 0709 13:02:17.224508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.224797 (+   289us) service_pool.cc:224] Handling call
0709 13:02:27.542057 (+10317260us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":11}
I20260709 13:02:27.543061  8823 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 2: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:27.686666  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 204) took 1675 ms. Trace:
I20260709 13:02:27.700256 10617 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.723114  8748 rpcz_store.cc:276] 0709 13:02:26.011123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.011332 (+   209us) service_pool.cc:224] Handling call
0709 13:02:26.011877 (+   545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.571653 (+559776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.686328 (+1114675us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54002ac954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:27.686574 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1252197248}
I20260709 13:02:27.671056 10396 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:27.544534 10582 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.540511 10550 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.703104 10225 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.715680  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:27.716096 10484 raft_consensus.cc:993] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: : Instructing follower 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:27.730688 10498 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:27.731703  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:27.732369  8748 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:02:27.733274  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:27.733415 10571 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:27.733829  8748 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.734176 10571 raft_consensus.cc:1081] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Signalling peer 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:02:27.720450  8676 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:27.710291 10467 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.737469 10467 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:27.737780  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:27.671901 10606 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:27.648838 10603 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.740096 10565 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:27.740252  8675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1286) took 1744 ms (client timeout 1632 ms). Trace:
I20260709 13:02:27.740593 10603 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:27.726878  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 226) took 1665 ms (client timeout 1846 ms). Trace:
W20260709 13:02:27.740852  8675 rpcz_store.cc:269] 0709 13:02:25.995430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.996020 (+   590us) service_pool.cc:224] Handling call
0709 13:02:27.739991 (+1743971us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b5400298954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:27.740158 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1565526912}
I20260709 13:02:27.720081  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:27.741632 10575 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:27.742200  9387 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.692595 10286 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.728361 10544 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:27.728757  9389 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:27.744758 10286 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:27.744969 10578 raft_consensus.cc:3055] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:27.745527 10578 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:27.727299 10560 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:27.746728 10532 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:27.729239 10563 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:27.729382 10550 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.748626  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:27.730803 10372 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:27.725189  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1292) took 1037 ms. Trace:
I20260709 13:02:27.749191 10556 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.749550  8752 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:27.749696  8673 rpcz_store.cc:276] 0709 13:02:26.688115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.692083 (+  3968us) service_pool.cc:224] Handling call
0709 13:02:27.723233 (+1031150us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:27.741487  8823 rpcz_store.cc:269] 0709 13:02:26.061826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.066232 (+  4406us) service_pool.cc:224] Handling call
0709 13:02:27.373273 (+1307041us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:27.726807 (+353534us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26807168}
I20260709 13:02:27.701678 10591 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.728868 10582 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:27.754276  8754 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:27.755263 10591 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:27.755937 10617 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.756464 10569 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:27.757153  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:27.757805  8754 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:27.758303 10617 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:27.763741  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1282) took 2104 ms. Trace:
I20260709 13:02:27.762712  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1463) took 1506 ms. Trace:
I20260709 13:02:27.759477 10470 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:27.709244 10618 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:27.712105 10410 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.761575 10561 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:27.823803 10561 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:27.824481 10561 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.777045  8674 rpcz_store.cc:276] 0709 13:02:25.659160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.659392 (+   232us) service_pool.cc:224] Handling call
0709 13:02:25.659958 (+   566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.766795 (+106837us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:25.924999 (+158204us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.333610 (+408611us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.333671 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:26.333717 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.763426 (+1429709us) spinlock_profiling.cc:243] Waited 84.4 ms on lock 0x7b54002571d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:27.763496 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.763650 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48792,"raft.queue_time_us":169372,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":13524,"spinlock_wait_cycles":194220928,"thread_start_us":356618,"threads_started":1,"wal-append.queue_time_us":464905,"wal-append.run_cpu_time_us":1318,"wal-append.run_wall_time_us":1001334}
I20260709 13:02:27.778455  8755 rpcz_store.cc:276] 0709 13:02:26.255974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.603938 (+347964us) service_pool.cc:224] Handling call
0709 13:02:26.618701 (+ 14763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.774341 (+155640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.774381 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.774436 (+    55us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:26.774477 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.762360 (+987883us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b540038b8d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:27.762434 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.762635 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":357359360}
W20260709 13:02:27.804273  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 296288us
I20260709 13:02:27.827754  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:27.827966  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:27.828580  8755 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:27.829335  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:27.829824  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:27.749055 10550 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:27.833655  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:27.846520  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:27.804495 10583 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.812422  9387 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:27.822091 10618 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.814927 10577 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:27.820071 10589 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.944s	user 0.009s	sys 0.005s
I20260709 13:02:27.854514  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:02:27.855154  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 152762us
I20260709 13:02:27.748652 10570 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:27.859194  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 173139us
I20260709 13:02:27.873831  8794 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:27.848783 10561 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:27.874441  8722 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:27.875263  8794 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:27.881350  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 71774.6us
W20260709 13:02:27.882480  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 229997us
I20260709 13:02:27.886392  8794 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:27.890141  8754 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:27.806593  8645 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:27.895567  8748 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:27.896489  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:27.896862 10570 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:27.897432 10570 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:27.898347 10570 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.859520 10571 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:27.900439  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 93938.9us
I20260709 13:02:27.904270  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:27.904940  9387 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:27.873114 10569 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:27.906105  8645 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:27.919203  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:27.868160 10581 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:27.922820  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:27.814771 10578 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.880576 10532 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:27.923707  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:27.900252 10396 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:27.927631  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1149) took 2573 ms. Trace:
I20260709 13:02:27.956220  8756 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 2: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:27.955513  8751 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:27.902233 10599 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:27.881690 10436 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:02:27.956292  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 50277.1us
I20260709 13:02:27.905762 10498 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:27.962459  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:27.960176  8830 rpcz_store.cc:276] 0709 13:02:25.354640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.406062 (+ 51422us) service_pool.cc:224] Handling call
0709 13:02:27.238163 (+1832101us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.484348 (+246185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.927357 (+443009us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b5400127350. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:27.927528 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4212627840}
I20260709 13:02:27.926152  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:27.964546  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 202) took 1954 ms. Trace:
I20260709 13:02:27.983376  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:27.983997  8751 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:27.984194  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:27.906790 10561 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:27.972348  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:27.861419 10583 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.985661  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:27.987555  8756 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:27.973291 10225 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.974387  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:27.993536  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:27.902362 10551 consensus_queue.cc:1059] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:02:27.975044  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:27.995081  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:27.975415 10430 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:27.977813  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:27.869015 10589 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.907045 10556 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.981292 10498 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.927925 10626 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:27.919689 10619 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:27.989265  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:27.921743 10622 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:27.873719 10577 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.990129  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:27.992295  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.001744  8756 raft_consensus.cc:2368] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4.
I20260709 13:02:27.996388  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.003324  8680 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:27.959506 10396 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.924746 10571 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.990130 10578 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:27.979072 10599 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:27.971019 10532 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:27.998603  9386 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:27.982709  8722 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:28.000054  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 170818us
I20260709 13:02:28.001672  8679 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:27.992740  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:27.986315 10436 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:27.958259 10569 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.025529 10577 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.026194 10550 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:28.026757 10569 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:28.002882 10498 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.960843 10627 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:27.994892  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:28.014997  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:28.004732 10622 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:28.036610  8756 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:28.017177  8675 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:28.017611  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:28.024621 10436 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:28.010030  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:27.980620 10581 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.031098 10583 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.030871  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:28.031726  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.010715 10619 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.002574 10626 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.061061  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1168) took 1299 ms. Trace:
I20260709 13:02:28.014238 10396 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:28.014994 10599 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.045614  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.066061  8674 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:28.066138  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.049127  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:28.049170  8679 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:28.041354 10618 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.058594  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 232) took 1164 ms. Trace:
I20260709 13:02:28.061622  8829 rpcz_store.cc:276] 0709 13:02:26.761756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.853008 (+ 91252us) service_pool.cc:224] Handling call
0709 13:02:28.060859 (+1207851us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b54003dd6d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:28.060998 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1772900224}
I20260709 13:02:28.035817 10627 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.061211  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:28.044318 10550 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:28.042665 10569 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:28.041762 10622 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.080070  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.080822  8675 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:28.089406  8751 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:28.064890 10583 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:28.067608 10633 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.063717 10626 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:28.099416  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 191503us
I20260709 13:02:28.074846 10617 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:27.985703  8749 rpcz_store.cc:276] 0709 13:02:26.010265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.014683 (+  4418us) service_pool.cc:224] Handling call
0709 13:02:27.926618 (+1911935us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b5800083198. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:27.957724 (+ 31106us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.960449 (+  2725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.964101 (+  3652us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:27.964183 (+    82us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:27.964245 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.964332 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.964481 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7405312}
I20260709 13:02:28.078413 10550 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:28.096441 10473 consensus_meta.cc:287] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.559s	user 0.006s	sys 0.008s
I20260709 13:02:28.110460  8756 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:28.079293 10627 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:28.097432 10583 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.074005 10561 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:02:28.096827  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 112740us
I20260709 13:02:28.070098  8822 rpcz_store.cc:276] 0709 13:02:26.894363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.339852 (+445489us) service_pool.cc:224] Handling call
0709 13:02:27.513174 (+173322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.758732 (+245558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.758790 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:27.758876 (+    86us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:27.758923 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.058215 (+299292us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b540032e454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:28.058279 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.058485 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468459264}
I20260709 13:02:28.092516 10569 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:28.128700  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 111960us
I20260709 13:02:28.135334  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.136317  8674 raft_consensus.cc:2410] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:28.114867 10571 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.115890 10550 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:28.138881  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:28.124744 10617 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:28.131641 10583 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:28.148377  8679 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:28.150382  8675 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:28.151664  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.161162  8674 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:28.161193  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:28.155640  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.166293  8645 leader_election.cc:400] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4.
I20260709 13:02:28.166780  8645 leader_election.cc:403] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:02:28.172300  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:28.173576  8830 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:28.170490  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:28.177182  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:28.177932  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:28.178105  8824 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:28.121431 10473 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.179153  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:28.183367  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:28.190850  8647 connection.cc:256] RPC call timeout handler was delayed by -2.0353032780000149s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
I20260709 13:02:28.191017  8675 raft_consensus.cc:2376] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3. Re-sending same reply.
I20260709 13:02:28.192495  8647 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:28.200874  8719 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:28.115520 10633 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.199036  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:28.202368  8719 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:28.206586  8719 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:28.207659  8719 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:28.208643  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:28.211071 10286 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:28.211576 10286 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.211952 10286 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:28.182039 10589 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.215026  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 207) took 1078 ms. Trace:
I20260709 13:02:28.215582  8749 rpcz_store.cc:276] 0709 13:02:27.136491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.126690 (+990199us) service_pool.cc:224] Handling call
0709 13:02:28.143441 (+ 16751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.178941 (+ 35500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.182567 (+  3626us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:28.184514 (+  1947us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:28.195543 (+ 11029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.203518 (+  7975us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.214941 (+ 11423us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:28.217780 10544 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:28.218461 10544 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.220139  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 253) took 1427 ms. Trace:
I20260709 13:02:28.220561  8676 rpcz_store.cc:276] 0709 13:02:26.793271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.794571 (+  1300us) service_pool.cc:224] Handling call
0709 13:02:27.720312 (+925741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.735834 (+ 15522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.219909 (+484075us) spinlock_profiling.cc:243] Waited 925 ms on lock 0x7b5400299350. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:28.220077 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2127892352}
I20260709 13:02:28.185667 10570 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:28.240494  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 101326us
I20260709 13:02:28.241835  8645 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:28.243571  8645 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:28.247105  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:28.200182  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:28.221619 10585 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.885s	user 0.008s	sys 0.007s
I20260709 13:02:28.196403  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:28.254489  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1858) took 1127 ms. Trace:
I20260709 13:02:28.227137 10560 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:28.204481 10630 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:28.219024 10563 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:28.226882 10603 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:28.233318 10570 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:28.252243 10571 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:28.255795  8756 rpcz_store.cc:276] 0709 13:02:27.127270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.121999 (+994729us) service_pool.cc:224] Handling call
0709 13:02:28.138282 (+ 16283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.138505 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:28.208757 (+ 70252us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.254093 (+ 45336us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:28.254161 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:28.254209 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.254286 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.254420 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2474,"raft.queue_time_us":131,"thread_start_us":90832,"threads_started":2}
I20260709 13:02:28.230295 10544 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.220930  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:28.261438  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:28.233111 10640 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:28.246476 10532 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.264699 10563 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.228989  9381 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:28.288484  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1853) took 1162 ms. Trace:
I20260709 13:02:28.204438 10621 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:02:28.257706 10585 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.277328  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:28.267441 10603 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.261101  8822 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:28.291913 10473 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.257611 10537 consensus_queue.cc:1059] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:02:28.297365  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1856) took 1170 ms. Trace:
I20260709 13:02:28.291797 10372 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:28.251469  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:28.303160  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:28.297704  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 1849) took 1174 ms. Trace:
I20260709 13:02:28.281312  8821 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:28.304687  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:28.305162  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 112844us
I20260709 13:02:28.371064  8748 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:28.371579  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:28.271790 10570 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:28.286062 10631 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:28.285542 10638 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4.
I20260709 13:02:28.284744 10637 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:28.261121  8679 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:28.260138 10560 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.276333 10630 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:28.380815  8647 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:28.308501  8755 rpcz_store.cc:276] 0709 13:02:27.125988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.863263 (+737275us) service_pool.cc:224] Handling call
0709 13:02:27.863825 (+   562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.866520 (+  2695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.866582 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:27.891319 (+ 24737us) log.cc:844] Serialized 26 byte log entry
0709 13:02:28.094653 (+203334us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:28.094709 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.094801 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.288186 (+193385us) raft_consensus.cc:1685] finished
0709 13:02:28.288218 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.288408 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1198,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":15897,"thread_start_us":171779,"threads_started":1,"wal-append.queue_time_us":233935}
I20260709 13:02:28.278014 10571 raft_consensus.cc:697] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:28.255908 10561 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.307242 10563 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.301546  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 1852) took 1175 ms. Trace:
I20260709 13:02:28.328497  8751 rpcz_store.cc:276] 0709 13:02:27.126764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.090699 (+963935us) service_pool.cc:224] Handling call
0709 13:02:28.103365 (+ 12666us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:28.108164 (+  4799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.108199 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:28.117823 (+  9624us) log.cc:844] Serialized 26 byte log entry
0709 13:02:28.184275 (+ 66452us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:28.184813 (+   538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.184921 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.297064 (+112143us) raft_consensus.cc:1685] finished
0709 13:02:28.297090 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.297285 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8197,"raft.queue_time_us":7663,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":7235,"thread_start_us":20270,"threads_started":1,"wal-append.queue_time_us":125601}
W20260709 13:02:28.355917  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.398100  8753 rpcz_store.cc:276] 0709 13:02:27.125738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.262765 (+137027us) service_pool.cc:224] Handling call
0709 13:02:28.301221 (+1038456us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b540008ff54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:28.301440 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":499815168}
I20260709 13:02:28.258255 10617 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.339618  8750 rpcz_store.cc:276] 0709 13:02:27.122928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.123062 (+   134us) service_pool.cc:224] Handling call
0709 13:02:28.297477 (+1174415us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b540038aed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:28.297606 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":863210752}
I20260709 13:02:28.400687  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:28.401669  8750 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:28.377434 10581 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:28.327459 10603 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:28.387681 10572 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:28.389555 10569 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:28.369917 10372 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.392665  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:28.439867  8756 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:28.395851  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.403767  8750 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:28.368651  8828 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.441627  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.384310  8647 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:28.403210  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 210) took 1118 ms. Trace:
I20260709 13:02:28.369223 10619 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.280907 10640 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.378314  8673 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:28.383857 10560 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.380721 10637 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.416000  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 206) took 1286 ms. Trace:
I20260709 13:02:28.419494  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:28.463533  8752 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:28.383932 10584 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:28.391075 10633 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.431231 10627 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.432477  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:28.432798  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.434492 10396 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:28.438201 10598 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.000s
I20260709 13:02:28.438467  8677 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:28.444674  8750 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:28.444312  8755 rpcz_store.cc:276] 0709 13:02:27.285310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.382449 (+1097139us) service_pool.cc:224] Handling call
0709 13:02:28.402694 (+ 20245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.402892 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.402919 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:28.402965 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:28.402996 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.403051 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.403151 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:28.442523 10533 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:28.446398 10571 consensus_queue.cc:237] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:28.447976 10645 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:02:28.499713  8675 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 3: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:28.448274 10410 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.448565 10510 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.451293 10229 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.452893  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:28.464483  8754 rpcz_store.cc:276] 0709 13:02:27.129516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.123021 (+993505us) service_pool.cc:224] Handling call
0709 13:02:28.129814 (+  6793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.130044 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:28.275588 (+145544us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.415360 (+139772us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:28.415491 (+   131us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:28.415567 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.415727 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.415920 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":142162,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":105,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":17846,"thread_start_us":273617,"threads_started":2}
W20260709 13:02:28.467226  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 83034.6us
I20260709 13:02:28.469791  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.520780  8755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:39032
I20260709 13:02:28.521492  8755 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:28.521492  8674 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 1.
I20260709 13:02:28.521932  8755 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:28.472580  8752 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:02:28.475893  8793 connection.cc:256] RPC call timeout handler was delayed by -1.281207481000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17170000000000002s for a response
I20260709 13:02:28.443544  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:28.468713 10578 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:28.492389 10598 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.458231 10569 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.502012  8756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:39032
I20260709 13:02:28.534991  8756 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:28.535439  8756 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:28.515013  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:28.516966  8830 raft_consensus.cc:2368] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
I20260709 13:02:28.491544 10498 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.479286 10619 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:28.523653  8793 connection.cc:256] RPC call timeout handler was delayed by -1.2617205980000108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2154s for a response
I20260709 13:02:28.468811 10637 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.537246  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:28.525806  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:28.414754 10536 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:02:28.530480  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:28.540742  8674 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.413381 10630 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.537973  8824 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
W20260709 13:02:28.537317  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.549093  8747 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:28.540325  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:28.551158 10578 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.551784 10578 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.553850 10578 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.530441 10229 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.555603  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.556274  8672 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:28.556396 10229 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:28.544040 10619 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.554231  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.560376  8722 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:28.565045  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:28.499032 10533 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.573647  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:28.576575  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.577670  8680 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:28.579262  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:28.580469  9387 raft_consensus.cc:2368] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
I20260709 13:02:28.584187  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.588102  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:28.586988  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.590965  8754 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:28.591746  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:28.592644  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.594210  8827 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:28.572609  9388 raft_consensus.cc:2955] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:28.543476  9380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 413) took 1651 ms. Trace:
I20260709 13:02:28.618053  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.607695  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:28.620441  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:28.610176 10410 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:28.611444  9384 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:02:28.631675  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:28.612100  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.592041 10229 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.568208 10532 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:28.613091 10473 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:28.540958 10569 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.622437  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:28.623464  8828 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:28.622437  8826 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:28.571300 10619 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:28.618076  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 44536.9us
I20260709 13:02:28.615739 10103 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:28.670166  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47829.5us
I20260709 13:02:28.670591  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:28.628942  8756 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.656736  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:28.658761  8793 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:28.632551  8680 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:28.653177 10229 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.662400 10569 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.662777 10532 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.664462 10473 raft_consensus.cc:3055] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:02:28.668043 10647 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:02:28.628268 10585 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.631079 10589 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:28.635640 10617 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:28.682152 10473 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:28.682716  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:28.683409 10396 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:28.683540 10569 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:28.681703 10532 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.615674  9380 rpcz_store.cc:276] 0709 13:02:26.888527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.890342 (+  1815us) service_pool.cc:224] Handling call
0709 13:02:27.052622 (+162280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.538948 (+486326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.540583 (+  1635us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:27.553442 (+ 12859us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:27.563554 (+ 10112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.513685 (+950131us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b5400018bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:28.523419 (+  9734us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.536085 (+ 12666us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1095797120}
I20260709 13:02:28.705641  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:28.706712 10630 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.725142  8680 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:28.726763  8674 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:28.729359  8756 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.730096  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.730836 10585 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:28.731284 10585 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:28.731428  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:28.731725 10585 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:28.732056  8674 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.736249  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:28.737255 10639 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:28.738126  8647 leader_election.cc:400] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
I20260709 13:02:28.738476  8647 leader_election.cc:403] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:02:28.739266 10639 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
I20260709 13:02:28.636106 10652 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:28.694942 10617 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:28.708597 10626 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.583s	user 0.022s	sys 0.023s
I20260709 13:02:28.684566 10410 raft_consensus.cc:2764] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:02:28.694077 10570 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.642023 10286 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.685665 10103 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.741746 10617 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:28.750967  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:28.752126 10627 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.751708 10544 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.707571 10619 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:28.659147  8672 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
W20260709 13:02:28.754604 10583 consensus_meta.cc:287] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.580s	user 0.016s	sys 0.003s
I20260709 13:02:28.681703 10229 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.738888  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:28.755927  8755 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:28.742604 10556 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.740s	user 0.012s	sys 0.002s
W20260709 13:02:28.758548  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68815.8us
I20260709 13:02:28.755641 10598 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.707361 10473 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.772909  8753 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:28.768183  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:28.769486  8749 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:28.769891 10533 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:28.725615 10560 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.772215 10103 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.742304 10589 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:02:28.772964  8722 leader_election.cc:400] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5.
I20260709 13:02:28.774000  8826 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.743124 10652 raft_consensus.cc:697] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:28.767549  8755 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.760179  8674 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:28.775758 10372 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:28.764081  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:28.780987 10570 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.784036  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:28.786171 10544 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.768586 10556 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:28.787910  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:28.794816  8674 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:28.800025  9386 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:28.756067 10626 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.783950 10560 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.803324  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:28.766870 10583 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.806077  9386 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.798295  9384 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:28.813926  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:28.814234 10225 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.814706  9384 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:28.804893 10473 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.819300  8675 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:02:28.817399 10550 consensus_meta.cc:287] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.658s	user 0.010s	sys 0.006s
I20260709 13:02:28.829110 10533 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:28.829932 10533 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.831427  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:28.832930 10619 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.833559 10619 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:28.825268 10599 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.746s	user 0.008s	sys 0.006s
I20260709 13:02:28.819258 10560 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.833032  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 208) took 1672 ms. Trace:
I20260709 13:02:28.847734 10532 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.857270 10532 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.859712 10532 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:28.848640  8825 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:28.848711 10638 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:28.836313 10550 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.854360  8749 rpcz_store.cc:276] 0709 13:02:27.161027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.216761 (+1055734us) service_pool.cc:224] Handling call
0709 13:02:28.217220 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.789481 (+572261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.832735 (+ 43254us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54002feed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:28.832955 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263067904}
I20260709 13:02:28.855007 10560 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.848030 10229 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:28.872219  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:28.872653 10229 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.873840  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:28.872283 10560 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.875597  8796 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:28.866804 10638 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:28.878563  8796 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:28.832968  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:28.876945 10229 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.881383 10591 raft_consensus.cc:2764] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:02:28.881659  8822 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:28.881822 10229 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:28.882711  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:28.882958  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:28.883594  8822 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.883736  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:28.884341 10229 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:28.885114 10229 raft_consensus.cc:697] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:28.885145 10531 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:28.861290  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.849023 10619 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:28.888923  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:28.890362  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:28.819334 10590 raft_consensus.cc:3055] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:28.862987  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:28.892697  8719 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:28.894184  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:28.895183  8756 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:28.895267  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:28.880785 10550 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.886334  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:28.909322  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:28.851209 10569 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:28.827776 10103 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:28.880520 10622 consensus_meta.cc:287] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.784s	user 0.000s	sys 0.022s
I20260709 13:02:28.885747 10531 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.898095  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:28.903168  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:28.927386  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:28.851394 10286 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.880713 10532 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:28.838243  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:28.897053 10590 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:28.935770  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:28.929736  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:28.909883 10560 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:28.953852 10229 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.909430 10533 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:28.950783  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:28.954741 10626 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:28.861653 10599 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.935811 10532 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.930653 10531 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:28.930675 10569 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:28.939930 10550 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:28.935282 10502 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:02:28.948696  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:28.930505 10103 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.913112 10225 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:28.959782 10560 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.906726 10396 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:02:28.968598  8752 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:28.932013 10603 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.972100  8750 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 4. Current term is 6. Ops: []
I20260709 13:02:28.977298  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:28.929734 10591 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:02:28.959992 10473 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:28.984522  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:28.928089 10653 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:28.985777  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 107291us
I20260709 13:02:28.987874  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:29.024624  8756 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:28.989832  8827 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:28.937263 10510 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:02:28.991381 10560 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:28.996716  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:28.997426  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 111765us
I20260709 13:02:28.972662 10532 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:28.878949 10638 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:29.021325 10653 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.007711  8828 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.973032  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:29.041457  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:29.042593  8681 raft_consensus.cc:2368] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:28.934942 10622 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:28.975621  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:28.887251 10652 consensus_queue.cc:237] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:29.020511  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:28.930925  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:28.986716 10533 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:28.988595 10225 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.033002  8674 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:28.986285  9384 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:28.958117 10621 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:02:29.048496  8826 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:29.049458  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1308) took 1051 ms. Trace:
I20260709 13:02:29.069061  8681 rpcz_store.cc:276] 0709 13:02:27.998317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.998451 (+   134us) service_pool.cc:224] Handling call
0709 13:02:29.041262 (+1042811us) spinlock_profiling.cc:243] Waited 916 us on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:29.049351 (+  8089us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2106752}
I20260709 13:02:29.069707  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.070899  8826 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:29.024973 10396 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:02:29.050587  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54071.4us
I20260709 13:02:29.071985  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:29.070926  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:29.072810  8681 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:29.052604  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 112482us
I20260709 13:02:28.999673 10585 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:29.064185  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.074884 10396 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:29.049978 10653 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:29.061491 10561 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:29.051826 10640 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:29.051436  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:29.106354  8722 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:29.105314  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.059203  9383 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:29.096104 10626 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:29.098990  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:29.080194  8719 leader_election.cc:400] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:29.118880  8719 leader_election.cc:403] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:02:29.102586  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:29.121461  8747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 209) took 1888 ms (client timeout 2211 ms). Trace:
I20260709 13:02:29.063750 10225 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:29.064103 10372 raft_consensus.cc:2764] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:02:29.107769  8826 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:29.115779 10672 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:29.060448 10663 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:29.088574 10396 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:29.118170  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:29.123128  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 49943.8us
W20260709 13:02:29.123374  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 88910.6us
W20260709 13:02:29.122026  8747 rpcz_store.cc:269] 0709 13:02:27.232975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.737413 (+504438us) service_pool.cc:224] Handling call
0709 13:02:29.121232 (+1383819us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b540038bdd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:29.121380 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1863574272}
I20260709 13:02:29.125257  9386 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:29.127005  8681 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:29.145547  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:29.100438  9381 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:29.135550 10651 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:29.073652  8749 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:29.115944 10599 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:29.130398 10570 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:29.099350 10584 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:02:29.104017 10561 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:29.091634 10639 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:02:29.108270 10627 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:02:29.061681 10590 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:29.222437  8828 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:29.196810  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1306) took 1203 ms. Trace:
I20260709 13:02:29.168354  8794 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:29.147608  8719 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:29.146384  8680 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:29.225839  8678 rpcz_store.cc:276] 0709 13:02:27.993296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.994823 (+  1527us) service_pool.cc:224] Handling call
0709 13:02:29.196558 (+1201735us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b5400175554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:29.196716 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2252881664}
I20260709 13:02:29.170878  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:29.227208  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 79766.5us
I20260709 13:02:29.172040  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:29.148236  8722 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:29.223815 10651 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:29.245812 10651 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.245851  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:29.230022  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 131454us
W20260709 13:02:29.243184  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 95079.2us
I20260709 13:02:29.231524  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:29.246770  8824 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:29.248472  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:29.227492 10561 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:29.156548  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.204437 10502 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:02:29.238054  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 69835.9us
I20260709 13:02:29.265131  8681 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 3: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:29.243696 10544 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:29.244515 10445 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.230916 10569 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:29.226859 10589 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:29.155761 10659 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 13:02:29.186822 10637 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.632s	user 0.014s	sys 0.000s
I20260709 13:02:29.252823  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:29.307605  8747 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.256979  9380 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:29.265007  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:29.244814 10663 raft_consensus.cc:2764] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:02:29.226642 10473 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:29.323037 10673 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:02:29.324788 10473 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:29.263713 10502 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:29.325881 10473 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:29.172171 10672 raft_consensus.cc:697] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:29.295374  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:29.335832  8824 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:29.231115 10103 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:29.242930 10570 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:29.275811  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:29.340668  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:29.342186  8681 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:29.344209  8747 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
W20260709 13:02:29.299710  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 279283us
I20260709 13:02:29.192740 10550 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:29.294672  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:29.303341  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:29.312537  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 47691.7us
W20260709 13:02:29.247952 10668 log.cc:927] Time spent T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.051s	user 0.001s	sys 0.001s
I20260709 13:02:29.195693 10617 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:29.248560  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:29.250851  8826 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:29.267691 10639 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:02:29.298142 10544 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:29.347406  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:29.347689  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:29.335505  9383 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:29.353967  8794 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:29.317013 10637 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:29.274101 10531 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:29.355600  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:29.362700  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.374934  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.360989  8681 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:29.373018  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:29.349701 10550 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:29.376788  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:29.379724  8747 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:29.357686  8749 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:29.379951  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:29.377730  8824 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:29.358829 10585 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:29.407219 10667 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:02:29.380935  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:29.412865  8826 raft_consensus.cc:2368] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5.
I20260709 13:02:29.414083  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:29.392661 10599 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:29.394460  8755 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:29.382109  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:29.415154 10544 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:29.416553 10591 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:29.417145 10591 raft_consensus.cc:697] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:29.418843 10672 consensus_queue.cc:237] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:29.399077 10570 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:29.358650 10502 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:29.402992  8679 raft_consensus.cc:2368] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4.
I20260709 13:02:29.403646  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:29.405447  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:29.406032 10622 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:29.360491  8672 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.410719  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.420351 10329 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.438819 10329 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:29.440045  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:29.443711 10531 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:29.445070  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:29.450870  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:29.437636  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:29.428984  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:29.380767 10676 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:29.361384 10470 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:02:29.431159 10502 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:29.380601 10103 consensus_queue.cc:1059] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.028s
W20260709 13:02:29.421386  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 45030.4us
I20260709 13:02:29.435889  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:29.437186  8824 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:29.455264  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:29.460992  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:29.381361 10630 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:29.378450 10582 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.378762 10663 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:29.418944 10591 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:29.542793 10675 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.092s	user 0.002s	sys 0.000s
I20260709 13:02:29.379336  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.550410  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1388) took 1390 ms. Trace:
I20260709 13:02:29.544255 10550 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:29.628137  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.659669  8676 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:29.660573  8681 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:29.661906  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 81450.9us
I20260709 13:02:29.660902  8754 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:29.664213  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:29.664357  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 235466us
I20260709 13:02:29.462844  8673 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:02:29.668833  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:29.696818  8675 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.695173  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:29.682014  8829 rpcz_store.cc:276] 0709 13:02:28.159886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.573243 (+413357us) service_pool.cc:224] Handling call
0709 13:02:28.573616 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.628234 (+ 54618us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:28.757323 (+129089us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.142959 (+385636us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.143020 (+    61us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:29.143093 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.549948 (+406855us) spinlock_profiling.cc:243] Waited 54.3 ms on lock 0x7b540016bf54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:29.550034 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.550292 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13941,"prepare.queue_time_us":164896,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":76898,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":188,"spinlock_wait_cycles":124823936,"thread_start_us":483478,"threads_started":3,"wal-append.queue_time_us":430401}
I20260709 13:02:29.662564 10655 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.656965 10676 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:29.704945  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.709936  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:29.711601  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.712996  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.715198  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:29.737803  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:29.739075  8829 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:29.740454  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.741400  8824 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
W20260709 13:02:29.744762  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1382) took 1587 ms (client timeout 1894 ms). Trace:
I20260709 13:02:29.690683 10483 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:29.745347  8823 rpcz_store.cc:269] 0709 13:02:28.157567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.161840 (+  4273us) service_pool.cc:224] Handling call
0709 13:02:29.363345 (+1201505us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:29.744677 (+381332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51790080}
I20260709 13:02:29.746943  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:29.747653  8823 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:29.751736  9385 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:29.650998 10532 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:29.708204 10563 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.291s	user 0.017s	sys 0.000s
I20260709 13:02:29.761576  8754 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:29.680012 10582 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:29.844511  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.788569  8681 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:29.678603 10630 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:29.811932  8675 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:29.703205 10599 raft_consensus.cc:3055] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:29.856889  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 214) took 2137 ms. Trace:
I20260709 13:02:29.869748  8753 rpcz_store.cc:276] 0709 13:02:27.719730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.419148 (+699418us) service_pool.cc:224] Handling call
0709 13:02:28.432629 (+ 13481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.773482 (+340853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.856420 (+1082938us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b54001572d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:29.856768 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782093568}
I20260709 13:02:29.861995  8829 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:29.872208  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1192) took 1019 ms. Trace:
I20260709 13:02:29.757092 10229 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.873229  8829 rpcz_store.cc:276] 0709 13:02:28.853092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.737418 (+884326us) service_pool.cc:224] Handling call
0709 13:02:29.872059 (+134641us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.873621  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:29.879206  8675 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:29.879400  8681 raft_consensus.cc:1813] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Handling vote request from an unknown peer e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:29.874284  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:29.880167  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1796) took 1439 ms. Trace:
I20260709 13:02:29.875545  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:29.880950  8754 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:29.880182  8681 raft_consensus.cc:2410] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20260709 13:02:29.803789 10225 consensus_meta.cc:287] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.660s	user 0.011s	sys 0.001s
I20260709 13:02:29.880568  8675 rpcz_store.cc:276] 0709 13:02:28.440693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.878601 (+1437908us) service_pool.cc:224] Handling call
0709 13:02:29.878988 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.879864 (+   876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.880097 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.884723  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:29.886067  8750 raft_consensus.cc:2393] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:29.897583  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:29.898026 10653 consensus_meta.cc:287] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.772s	user 0.013s	sys 0.000s
I20260709 13:02:29.898514 10653 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:29.900940 10582 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:29.902100 10669 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:29.845041 10570 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:29.904088 10570 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:29.906699  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:29.891691  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1797) took 1420 ms. Trace:
I20260709 13:02:29.908361  8681 rpcz_store.cc:276] 0709 13:02:28.471153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.890331 (+1419178us) service_pool.cc:224] Handling call
0709 13:02:29.890880 (+   549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.891173 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.891215 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.891304 (+    89us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:29.891360 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.891441 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.891598 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.758518 10550 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:29.898887  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:29.906699  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:29.914377  8753 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:29.907778  8754 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
W20260709 13:02:29.803360 10675 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
W20260709 13:02:29.889586  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 230350us
I20260709 13:02:29.919162  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:29.919096  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:29.912923 10570 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:29.912039 10676 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:29.882601 10560 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.841s	user 0.013s	sys 0.001s
W20260709 13:02:29.916675  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 244359us
I20260709 13:02:29.917676  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 269) took 1031 ms. Trace:
I20260709 13:02:29.935616  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:29.936058  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:29.936957  8753 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:29.876328 10599 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:29.844539 10563 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:29.923779 10669 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:29.944550  8751 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:29.900940 10653 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:29.927969  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.928223  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:29.882601  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.911166  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.932564  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1188) took 1137 ms. Trace:
I20260709 13:02:29.938107  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:29.949260  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 238) took 1110 ms. Trace:
I20260709 13:02:29.949600  8750 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.949729  8829 rpcz_store.cc:276] 0709 13:02:28.839000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.910792 (+1071792us) service_pool.cc:224] Handling call
0709 13:02:29.949185 (+ 38393us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:29.938345  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.938506  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:29.940528  8681 rpcz_store.cc:276] 0709 13:02:28.886732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.916704 (+1029972us) service_pool.cc:224] Handling call
0709 13:02:29.917092 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.917323 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.917354 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.917398 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:29.917435 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.917491 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.917611 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.932415  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:29.954072  8752 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 2: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:29.954327  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1176) took 1191 ms. Trace:
I20260709 13:02:29.951086  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:29.954499  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:29.955331  8829 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:29.955983  8722 leader_election.cc:400] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5.
I20260709 13:02:29.956032  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1189) took 1149 ms. Trace:
I20260709 13:02:29.949296  8828 rpcz_store.cc:276] 0709 13:02:28.795530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.862951 (+ 67421us) service_pool.cc:224] Handling call
0709 13:02:29.932378 (+1069427us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b540009d8d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:29.932496 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266297600}
W20260709 13:02:29.931917  8822 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.048s	user 0.016s	sys 0.006s
I20260709 13:02:29.980818  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:29.983217  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 270) took 1096 ms. Trace:
I20260709 13:02:29.983855  8681 rpcz_store.cc:276] 0709 13:02:28.887061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.982129 (+1095068us) service_pool.cc:224] Handling call
0709 13:02:29.982581 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.982845 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.982875 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.982921 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:29.982957 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.983015 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.983146 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.956480  8722 leader_election.cc:403] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:02:29.890192 10229 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:29.955018  8824 rpcz_store.cc:276] 0709 13:02:28.763070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.908295 (+1145225us) service_pool.cc:224] Handling call
0709 13:02:29.954221 (+ 45926us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.974767  8829 rpcz_store.cc:276] 0709 13:02:28.806646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.950562 (+1143916us) service_pool.cc:224] Handling call
0709 13:02:29.955959 (+  5397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.952903 10585 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:29.952230  8825 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:29.987152  8722 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:30.001348  8722 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:30.002556  8722 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:29.917470 10225 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:29.970676  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:29.979094 10484 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:02:29.980566 10544 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:30.004813  8752 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:29.995316  8681 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:02:29.979204 10671 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:30.008695  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 271) took 1121 ms. Trace:
I20260709 13:02:29.981834  8822 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:29.987156 10582 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5.
I20260709 13:02:29.979835  8821 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:29.739552 10655 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:30.024502  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1186) took 1252 ms. Trace:
I20260709 13:02:30.024812 10582 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:30.025105  8821 rpcz_store.cc:276] 0709 13:02:28.773532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.719520 (+945988us) service_pool.cc:224] Handling call
0709 13:02:30.024234 (+304714us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.027086 10655 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:30.007915 10591 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:30.027802 10591 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.028224 10591 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:30.029845 10630 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:30.031431 10680 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:02:30.032433  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1395) took 1864 ms. Trace:
W20260709 13:02:30.033380 10682 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:02:30.035884  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.012749  8829 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:30.064579  8830 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1392) took 1899 ms (client timeout 1535 ms). Trace:
I20260709 13:02:29.945756 10560 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:30.016642  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1174) took 1254 ms. Trace:
I20260709 13:02:30.017557  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:30.009203  8681 rpcz_store.cc:276] 0709 13:02:28.887388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.987608 (+1100220us) service_pool.cc:224] Handling call
0709 13:02:29.992821 (+  5213us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.007222 (+ 14401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.008624 (+  1402us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.019745  8828 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:30.021126  8754 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:29.988466 10639 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:02:29.997345 10653 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:30.077847  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 273) took 1189 ms. Trace:
I20260709 13:02:30.077859  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:02:30.078786  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 101997us
I20260709 13:02:30.078477  8681 rpcz_store.cc:276] 0709 13:02:28.887936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.076787 (+1188851us) service_pool.cc:224] Handling call
0709 13:02:30.077186 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.077416 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.077441 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.077487 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:30.077522 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.077581 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.077701 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.080062  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:30.026921 10498 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:30.032346 10229 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:30.019145 10589 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.713s	user 0.009s	sys 0.005s
I20260709 13:02:30.064666  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 234) took 1226 ms. Trace:
I20260709 13:02:30.007819 10473 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:30.065244  8830 rpcz_store.cc:269] 0709 13:02:28.164903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.205419 (+ 40516us) service_pool.cc:224] Handling call
0709 13:02:30.064322 (+1858903us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b540016a654. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:30.064485 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591367424}
I20260709 13:02:30.069665  8824 rpcz_store.cc:276] 0709 13:02:28.762393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.991958 (+1229565us) service_pool.cc:224] Handling call
0709 13:02:30.015844 (+ 23886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.016147 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.016187 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.016252 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:30.016305 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.016388 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.016546 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.071414  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:30.032979  8827 rpcz_store.cc:276] 0709 13:02:28.167733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.597437 (+429704us) service_pool.cc:224] Handling call
0709 13:02:28.597756 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.028594 (+430838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.032115 (+1003521us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b540009d3d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:30.032351 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":901042048}
I20260709 13:02:30.072264 10569 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:30.014500  8750 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:30.107182  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:30.083078  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:30.109248  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:02:30.063329  8749 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.634s	user 0.007s	sys 0.006s
I20260709 13:02:30.084424  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:30.063375 10651 consensus_meta.cc:287] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.817s	user 0.029s	sys 0.014s
I20260709 13:02:30.020220 10671 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:30.087765  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:02:30.077272 10681 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:02:30.025471 10582 raft_consensus.cc:697] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:30.094794  8753 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:02:30.119602  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 117089us
W20260709 13:02:30.092240  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.120496  8722 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:30.122480  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:30.099728  8829 rpcz_store.cc:276] 0709 13:02:28.837917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.992383 (+1154466us) service_pool.cc:224] Handling call
0709 13:02:30.012438 (+ 20055us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.064393 (+ 51955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.064605 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.069660  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1402) took 1174 ms. Trace:
I20260709 13:02:30.103981  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:30.101392 10585 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:30.080569  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 274) took 1192 ms. Trace:
I20260709 13:02:30.111755 10630 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:30.111217  8752 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:30.117638  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:30.133414  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 236) took 1295 ms. Trace:
I20260709 13:02:30.133829  8829 rpcz_store.cc:276] 0709 13:02:28.838451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.124110 (+1285659us) service_pool.cc:224] Handling call
0709 13:02:30.132837 (+  8727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.133048 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.133078 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.133123 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:30.133161 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.133230 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.133349 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.135239 10691 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:30.136063 10691 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:30.138599 10582 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:30.125017 10655 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:30.141197 10655 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:30.143129 10687 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:02:30.147588 10590 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:30.121307 10430 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:30.148461 10430 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.148869 10430 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:30.118467 10651 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:30.151091  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 524) took 1124 ms. Trace:
I20260709 13:02:30.151546  9384 rpcz_store.cc:276] 0709 13:02:29.026465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.108673 (+ 82208us) service_pool.cc:224] Handling call
0709 13:02:29.196150 (+ 87477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.196616 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.196642 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.753622 (+556980us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b2c00012c98. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e0df2a11 00007f32e0e02372 00007f32e2534a01 00007f32e24f943a 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2
0709 13:02:29.755242 (+  1620us) log.cc:844] Serialized 494 byte log entry
0709 13:02:29.935347 (+180105us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:29.942757 (+  7410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.942941 (+   184us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:30.150703 (+207762us) raft_consensus.cc:1685] finished
0709 13:02:30.150726 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.151007 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2692864,"thread_start_us":174769,"threads_started":1,"wal-append.queue_time_us":248097}
I20260709 13:02:30.128742 10229 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:30.156641  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:30.157419  8752 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:30.114806 10473 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:30.126894  8822 rpcz_store.cc:276] 0709 13:02:28.849089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.882585 (+ 33496us) service_pool.cc:224] Handling call
0709 13:02:30.023462 (+1140877us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.160670 10473 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:30.161890  8822 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:30.163070 10563 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:30.163801  9353 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:30.164944  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1196) took 1245 ms. Trace:
I20260709 13:02:30.165458  8825 rpcz_store.cc:276] 0709 13:02:28.919093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.937950 (+ 18857us) service_pool.cc:224] Handling call
0709 13:02:30.164691 (+1226741us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b540041d054. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:30.164863 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1970310016}
I20260709 13:02:30.085083 10653 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:30.128089  8719 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:30.171557  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 43542.1us
I20260709 13:02:30.073686  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:30.129927  8681 rpcz_store.cc:276] 0709 13:02:28.888162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.079904 (+1191742us) service_pool.cc:224] Handling call
0709 13:02:30.080190 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.080342 (+   152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.080357 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.080388 (+    31us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:30.080411 (+    23us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.080444 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.080522 (+    78us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.175961  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:30.176419  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 237) took 1337 ms. Trace:
I20260709 13:02:30.176793  8829 rpcz_store.cc:276] 0709 13:02:28.838747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.134547 (+1295800us) service_pool.cc:224] Handling call
0709 13:02:30.175847 (+ 41300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.176069 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.176097 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.176140 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:30.176186 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.176251 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.176365 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.177932  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:30.131189  8722 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:30.107336 10589 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:30.129413  8676 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:30.021842 10599 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:30.114161 10602 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:02:30.165140 10561 consensus_meta.cc:287] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.871s	user 0.006s	sys 0.006s
I20260709 13:02:30.084860 10544 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:30.168282  8752 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:30.117457 10445 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:30.124156  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:30.111235  8749 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
W20260709 13:02:30.206722  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.207685  8680 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:30.207857  8821 raft_consensus.cc:2393] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:30.207877 10550 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:30.209000 10603 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:30.209712 10603 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:30.209033 10550 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:30.210680 10691 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:30.211263 10691 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:30.212908  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.213451  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.214192  8796 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:30.214828 10550 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:30.215339 10550 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:30.215873  9381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 518) took 1618 ms. Trace:
I20260709 13:02:30.215790 10550 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:30.216274 10103 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:30.220767 10570 raft_consensus.cc:2764] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:02:30.171669  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:30.104545  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 235) took 1266 ms. Trace:
I20260709 13:02:30.222878  8827 rpcz_store.cc:276] 0709 13:02:28.838210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.103644 (+1265434us) service_pool.cc:224] Handling call
0709 13:02:30.104009 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.104209 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.104233 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.104277 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:30.104314 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.104368 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.104486 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:30.172899 10502 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.512s	user 0.002s	sys 0.010s
W20260709 13:02:30.197717  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 117770us
W20260709 13:02:30.204510  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 73397.3us
I20260709 13:02:30.200515  8830 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:30.225342  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:30.168675  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1401) took 1323 ms. Trace:
I20260709 13:02:30.202365  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:30.204823  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 80725.8us
I20260709 13:02:30.205899 10498 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:30.206370 10630 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:30.213371 10691 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:30.222398  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 50838.3us
I20260709 13:02:30.116942 10532 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:30.223605  8680 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:30.200455  8829 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:30.231976  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:30.179908  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1322) took 1276 ms. Trace:
I20260709 13:02:30.233515  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:30.267113  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1197) took 1347 ms. Trace:
I20260709 13:02:30.267083  9386 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:30.267764  8829 rpcz_store.cc:276] 0709 13:02:28.919460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.177446 (+1257986us) service_pool.cc:224] Handling call
0709 13:02:30.200232 (+ 22786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.266757 (+ 66525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.267050 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.268177  8676 rpcz_store.cc:276] 0709 13:02:28.903666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.435560 (+531894us) service_pool.cc:224] Handling call
0709 13:02:30.179810 (+744250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.224433 10599 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:30.241315  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:30.242345  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:30.273365  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:30.259879  9389 raft_consensus.cc:2955] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:30.248889  8681 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:02:30.202509  8672 consensus_meta.cc:287] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.764s	user 0.006s	sys 0.008s
I20260709 13:02:30.207998 10470 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:30.201562 10602 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:30.180860 10563 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:30.287657  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.263471 10502 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:30.240509  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:30.269524  8750 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:30.270653  8680 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:30.268978  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 43837.1us
I20260709 13:02:30.207710  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:30.271001 10286 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:30.203598  8823 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:30.276360  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:30.280751  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:30.275563  8794 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:30.271701 10656 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 1.536s	user 0.011s	sys 0.017s
I20260709 13:02:30.281749 10498 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:30.254684  8825 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:30.284989  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:30.202366 10561 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:30.302496  8680 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:02:30.295208 10602 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:30.214249 10229 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:30.302215  8752 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:30.271938  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:30.273492  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:30.252965 10630 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:02:30.341300  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.430465 10630 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:30.357411  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:30.430905  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:30.432397  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:30.432463  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:30.433766  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:30.217686 10669 raft_consensus.cc:2955] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:30.382118 10483 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:30.374568 10473 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:30.367085 10484 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:30.427268  9383 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:30.434574  8829 raft_consensus.cc:2393] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:30.338194 10445 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:30.444828  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:30.445736  8829 raft_consensus.cc:2393] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:30.447248  9387 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:30.447899  8675 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:30.383330 10533 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.466450  8680 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
W20260709 13:02:30.467914  8680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1324) took 1563 ms (client timeout 1947 ms). Trace:
W20260709 13:02:30.468364  8680 rpcz_store.cc:269] 0709 13:02:28.904270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.267103 (+362833us) service_pool.cc:224] Handling call
0709 13:02:30.207532 (+940429us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b540040efa0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2470dc0 00007f32e24fce18 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6
0709 13:02:30.467836 (+260304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736428288}
I20260709 13:02:30.216316  9381 rpcz_store.cc:276] 0709 13:02:28.597504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.612418 (+ 14914us) service_pool.cc:224] Handling call
0709 13:02:28.612768 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.230905 (+618137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.215564 (+984659us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b54006e1754. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:30.215794 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1043525376}
I20260709 13:02:30.364023  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:30.364821  8681 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
I20260709 13:02:30.475289  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:30.476610  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:30.280774 10693 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:30.365907  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:30.308262  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:30.481902  8676 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:30.483598  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:30.484552  8676 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:30.364302  8830 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:30.281353  8672 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
W20260709 13:02:30.365068  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 89569.5us
I20260709 13:02:30.492714  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:30.371181 10691 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.362965 10498 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.286314 10637 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:30.294801 10470 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:30.240074  8822 rpcz_store.cc:276] 0709 13:02:28.838920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.161255 (+1322335us) service_pool.cc:224] Handling call
0709 13:02:30.161657 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.162342 (+   685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.162584 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.265151 10591 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:30.320458 10563 raft_consensus.cc:697] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:30.284765 10619 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:30.319851 10671 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:30.412465  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.427048  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1403) took 1577 ms. Trace:
I20260709 13:02:30.347265  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:30.477636  8827 raft_consensus.cc:2368] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3.
W20260709 13:02:30.479261  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 229106us
W20260709 13:02:30.479993  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 369630us
W20260709 13:02:30.300089  8673 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.584s	user 0.008s	sys 0.005s
I20260709 13:02:30.444998 10445 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:30.538329  8673 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:30.539011  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 1813) took 1545 ms. Trace:
I20260709 13:02:30.539399  9355 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:30.495820  8794 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:30.429936 10430 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:30.497089  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 242661us
W20260709 13:02:30.499619  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.495349 10663 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:30.381840 10103 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:02:30.488577 10584 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:02:30.516321  8681 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:30.516716  8676 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
W20260709 13:02:30.514602  8672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 1809) took 1529 ms (client timeout 1744 ms). Trace:
I20260709 13:02:30.539353  8673 rpcz_store.cc:276] 0709 13:02:28.994119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.403269 (+409150us) service_pool.cc:224] Handling call
0709 13:02:30.538939 (+1135670us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.311066 10692 consensus_queue.cc:1059] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
W20260709 13:02:30.540769  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 45056.1us
I20260709 13:02:30.514595 10498 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:30.443197 10484 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:30.499074  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:30.531817  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.489662 10533 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:30.521927 10653 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:30.522732 10619 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.556593  8722 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:30.557966  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:30.549927 10699 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:30.558560  8794 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:30.380368 10655 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:30.606403  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 47931.4us
I20260709 13:02:30.607534  8794 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:30.517176 10691 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:30.610072  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.559787  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 275) took 1671 ms. Trace:
I20260709 13:02:30.610769  8676 rpcz_store.cc:276] 0709 13:02:28.888400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.558958 (+1670558us) service_pool.cc:224] Handling call
0709 13:02:30.559265 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.559452 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.559479 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.559522 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:30.559558 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.559609 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.559721 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.371917 10544 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:30.492326 10694 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:02:30.567619  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:30.550711 10671 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:30.568223  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 136958us
I20260709 13:02:30.576162  9389 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:02:30.556465  8672 rpcz_store.cc:269] 0709 13:02:28.992230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.178859 (+186629us) service_pool.cc:224] Handling call
0709 13:02:30.499540 (+1320681us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.647953  8678 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:30.652943  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1321) took 1749 ms. Trace:
I20260709 13:02:30.603684  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:30.558446  8830 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:30.615923  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 76603.5us
I20260709 13:02:30.639359 10653 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:30.619949 10704 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:30.600895 10619 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:30.624670 10695 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:30.556591 10103 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:30.564844 10502 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:02:30.630671 10697 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:02:30.637081  9387 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 521) took 1945 ms (client timeout 1976 ms). Trace:
I20260709 13:02:30.637894 10691 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:30.638569  8752 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:30.631384 10690 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.639189  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1802) took 2087 ms. Trace:
I20260709 13:02:30.520852  8825 rpcz_store.cc:276] 0709 13:02:28.849428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.166414 (+1316986us) service_pool.cc:224] Handling call
0709 13:02:30.178862 (+ 12448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.363545 (+184683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.426936 (+ 63391us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.642352 10430 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:30.564582  9382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 514) took 2075 ms. Trace:
I20260709 13:02:30.612914  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:30.701658  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 88846.7us
I20260709 13:02:30.601210  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:30.646492  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1320) took 1743 ms. Trace:
I20260709 13:02:30.569339 10473 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 13:02:30.618152 10705 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:30.667585  8678 rpcz_store.cc:276] 0709 13:02:28.903366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.630200 (+1726834us) service_pool.cc:224] Handling call
0709 13:02:30.647475 (+ 17275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.651246 (+  3771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.652840 (+  1594us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.741281  8678 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:30.569085  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:30.570367  8680 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:02:30.675132 10582 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.011s	sys 0.004s
I20260709 13:02:30.679646  8719 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:30.639616 10676 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:30.686142  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 73737.3us
I20260709 13:02:30.633919 10563 consensus_queue.cc:237] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:30.655553 10671 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:02:30.690730  9387 rpcz_store.cc:269] 0709 13:02:28.691962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.439760 (+747798us) service_pool.cc:224] Handling call
0709 13:02:30.636826 (+1197066us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001a3bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:30.636970 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2314704256}
I20260709 13:02:30.693619  8675 rpcz_store.cc:276] 0709 13:02:28.551818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.894429 (+1342611us) service_pool.cc:224] Handling call
0709 13:02:29.927323 (+ 32894us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.448290 (+520967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.638885 (+190595us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b54003f43d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:30.639110 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1196776832}
I20260709 13:02:30.689319 10470 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:30.780855 10103 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:30.781446 10470 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:30.781890 10470 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:30.782341 10470 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:30.705909  8676 rpcz_store.cc:276] 0709 13:02:28.903018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.627745 (+1724727us) service_pool.cc:224] Handling call
0709 13:02:30.645728 (+ 17983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.646001 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.646043 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.646103 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:30.646160 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.646247 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.646395 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.689919 10653 raft_consensus.cc:697] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:02:30.715647  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.680691 10694 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:02:30.626768 10699 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:30.656616  8829 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:30.749034  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 277) took 1676 ms. Trace:
I20260709 13:02:30.799364  8678 rpcz_store.cc:276] 0709 13:02:29.072680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.740738 (+1668058us) service_pool.cc:224] Handling call
0709 13:02:30.741080 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.748753 (+  7673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.748966 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:30.800767  8678 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:30.802109  8678 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:30.704005 10533 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:30.779392  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 102107us
I20260709 13:02:30.779938  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 278) took 1652 ms. Trace:
I20260709 13:02:30.782878 10484 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:30.687162 10561 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:30.705731 10692 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:30.760754 10582 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.012s	sys 0.004s
I20260709 13:02:30.776696 10693 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:02:30.824474 10702 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.171s
I20260709 13:02:30.804641  8675 rpcz_store.cc:276] 0709 13:02:29.127194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.775751 (+1648557us) service_pool.cc:224] Handling call
0709 13:02:30.779354 (+  3603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.779562 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.779590 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.779635 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:30.779670 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.779733 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.779868 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:30.808333  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.777392 10556 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:02:30.828204 10372 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:30.852610 10372 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:02:30.864910  8679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 1786) took 2619 ms (client timeout 2187 ms). Trace:
W20260709 13:02:30.865484  8679 rpcz_store.cc:269] 0709 13:02:28.245715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.484145 (+238430us) service_pool.cc:224] Handling call
0709 13:02:30.864609 (+2380464us) spinlock_profiling.cc:243] Waited 866 ms on lock 0x7b54002409d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:30.864785 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1992355072}
I20260709 13:02:30.866976 10653 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:30.871107 10698 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:02:30.871598  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:30.826144  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:30.804222  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 126367us
I20260709 13:02:30.695884 10695 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:30.708822 10690 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:30.838976  8680 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:30.823834  8678 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:30.853922 10372 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:30.969992  8824 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:30.996513  8830 raft_consensus.cc:3055] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:30.827461  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1325) took 1922 ms. Trace:
W20260709 13:02:30.968954  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 412410us
I20260709 13:02:31.060210  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1173) took 2298 ms. Trace:
I20260709 13:02:31.207841  8830 raft_consensus.cc:740] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:02:30.990572  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.227892  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:31.209961  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1399) took 2371 ms. Trace:
I20260709 13:02:30.984548  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:30.915642  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.943315  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:02:30.945456 10630 consensus_meta.cc:287] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.514s	user 0.003s	sys 0.010s
I20260709 13:02:31.012669  8681 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:02:30.985252  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.211992  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1336) took 1080 ms. Trace:
I20260709 13:02:31.064778  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:31.144140  8680 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:02:31.057183  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1795) took 2634 ms. Trace:
I20260709 13:02:31.063273  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 1889) took 1326 ms. Trace:
W20260709 13:02:30.999270 10712 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
W20260709 13:02:31.202138 10652 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.882s	user 0.011s	sys 0.031s
I20260709 13:02:30.946593 10651 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:31.307653  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.986603  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1897) took 1063 ms. Trace:
I20260709 13:02:31.229832  8828 rpcz_store.cc:276] 0709 13:02:28.762062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.981998 (+1219936us) service_pool.cc:224] Handling call
0709 13:02:30.019512 (+ 37514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.070022 (+ 50510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.059953 (+989931us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b54003de0d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:31.060120 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54388480}
I20260709 13:02:31.230805  8824 rpcz_store.cc:276] 0709 13:02:28.838368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.122966 (+1284598us) service_pool.cc:224] Handling call
0709 13:02:30.863135 (+740169us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:30.969736 (+106601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.209568 (+239832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.209878 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3872000}
I20260709 13:02:30.947919 10329 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:31.066876 10225 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:31.062556 10690 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:31.135351 10599 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.859s	user 0.010s	sys 0.004s
I20260709 13:02:31.375334  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:31.375980  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:31.376613  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:31.052608 10498 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:31.377454  8824 raft_consensus.cc:2376] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4. Re-sending same reply.
I20260709 13:02:31.242928  8830 consensus_queue.cc:260] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:31.058596  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:31.251883  8678 rpcz_store.cc:276] 0709 13:02:30.131510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.804333 (+672823us) service_pool.cc:224] Handling call
0709 13:02:30.823626 (+ 19293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.211632 (+388006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.211910 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:30.956002  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.885335 10692 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:30.998490  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:30.998317  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:31.253935  8754 rpcz_store.cc:276] 0709 13:02:29.736763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.920298 (+183535us) service_pool.cc:224] Handling call
0709 13:02:31.063043 (+1142745us) spinlock_profiling.cc:243] Waited 72.3 ms on lock 0x7b540034a154. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:31.063204 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166363904}
I20260709 13:02:31.241860  8674 rpcz_store.cc:276] 0709 13:02:28.423299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.836767 (+413468us) service_pool.cc:224] Handling call
0709 13:02:28.843625 (+  6858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.061367 (+217742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.056895 (+1995528us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54002567d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:31.057105 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434930048}
W20260709 13:02:31.005373  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.501415  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.333716  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.065776 10550 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:30.974468 10618 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:02:31.056418 10602 consensus_meta.cc:287] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.608s	user 0.011s	sys 0.001s
I20260709 13:02:30.924283 10671 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:31.254912 10630 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:31.376714  8747 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:31.250547  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1346) took 1011 ms. Trace:
I20260709 13:02:30.969726  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:31.006086 10581 raft_consensus.cc:3055] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:30.940994 10699 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:31.382344  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 104714us
I20260709 13:02:31.382972 10694 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.384047  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 276) took 2495 ms. Trace:
I20260709 13:02:31.385246 10701 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.874799  8679 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:31.251210 10652 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.935s	user 0.013s	sys 0.032s
W20260709 13:02:31.463680  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 74193.5us
I20260709 13:02:31.462345  8830 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:30.928516 10583 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:31.493487  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:31.638051  8828 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:31.460223  8821 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:31.497865 10651 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:30.986136 10619 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:31.248865  8748 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:31.502797  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:31.648046  8824 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:31.347537 10372 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:30.839643 10502 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:31.512075  8681 rpcz_store.cc:276] 0709 13:02:30.241171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.011922 (+770751us) service_pool.cc:224] Handling call
0709 13:02:31.012250 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.250215 (+237965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.250472 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.520030  9353 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:02:31.652797  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 136345us
I20260709 13:02:31.520081  8829 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:31.508608 10618 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:31.508677  8825 raft_consensus.cc:2368] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
I20260709 13:02:31.356709 10470 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:31.227000  8672 rpcz_store.cc:276] 0709 13:02:28.904578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.676491 (+1771913us) service_pool.cc:224] Handling call
0709 13:02:30.694728 (+ 18237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.709540 (+ 14812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.772075 (+ 62535us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.773215 (+  1140us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:30.792541 (+ 19326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.797103 (+  4562us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.824029 (+ 26926us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.547891 10695 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:31.548990 10692 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:31.509800 10591 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:31.508260  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:31.671146  8754 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:31.671859  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 1906) took 1016 ms. Trace:
I20260709 13:02:31.672290  8754 rpcz_store.cc:276] 0709 13:02:30.655497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.507839 (+852342us) service_pool.cc:224] Handling call
0709 13:02:31.671756 (+163917us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.673969  8754 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:31.555437  8647 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:31.675905  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 120569us
W20260709 13:02:31.575930  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.583791  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 280) took 1447 ms. Trace:
I20260709 13:02:31.539337  9380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 529) took 1310 ms. Trace:
I20260709 13:02:31.519225 10602 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:31.554637 10581 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:31.549019 10498 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:31.558024 10639 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.100s
I20260709 13:02:31.663383 10707 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:31.657570 10651 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:31.691398 10707 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.691839 10707 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:31.693295 10651 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:31.663717  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:31.664335  8823 raft_consensus.cc:2368] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3.
I20260709 13:02:31.664649 10618 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:31.665230  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:31.666954  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1901) took 1580 ms. Trace:
W20260709 13:02:31.677171  9355 leader_election.cc:336] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.846s (SENT)
I20260709 13:02:31.676959  8679 rpcz_store.cc:276] 0709 13:02:30.136142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.866404 (+730262us) service_pool.cc:224] Handling call
0709 13:02:30.874549 (+  8145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.583400 (+708851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.583704 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.700213  8756 rpcz_store.cc:276] 0709 13:02:30.086118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.665663 (+1579545us) service_pool.cc:224] Handling call
0709 13:02:31.666061 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.666310 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.666338 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:31.666390 (+    52us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:31.666442 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.666527 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.666873 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.700834  9355 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:31.677922  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1903) took 1556 ms. Trace:
W20260709 13:02:31.678967  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.682658 10583 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:31.680516 10652 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.683462 10470 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:30.701319  9382 rpcz_store.cc:276] 0709 13:02:28.488845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.495000 (+  6155us) service_pool.cc:224] Handling call
0709 13:02:28.599782 (+104782us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:28.667399 (+ 67617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.667444 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:28.668337 (+   893us) log.cc:844] Serialized 52 byte log entry
0709 13:02:28.771642 (+103305us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:28.775522 (+  3880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.791696 (+ 16174us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:30.165425 (+1373729us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b180031b460. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db
0709 13:02:30.197430 (+ 32005us) spinlock_profiling.cc:243] Waited 6.67 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:30.550524 (+353094us) raft_consensus.cc:1685] finished
0709 13:02:30.551634 (+  1110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.557589 (+  5955us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24482,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":21764,"spinlock_wait_cycles":18234368,"thread_start_us":87516,"threads_started":1,"wal-append.queue_time_us":66581}
I20260709 13:02:31.683876  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:31.688215  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:31.680949 10716 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:02:31.695119  8748 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:31.618539 10694 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:31.700519  8828 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:31.645603 10701 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:31.702277  8754 rpcz_store.cc:276] 0709 13:02:30.121222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.673234 (+1552012us) service_pool.cc:224] Handling call
0709 13:02:31.673709 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.674516 (+   807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.677837 (+  3321us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:31.702277  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.705432  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.708527  8824 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:31.720129  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:31.721158  8828 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:31.702554 10651 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.723445  8748 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:31.725204  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:31.726022  8748 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:31.727167  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:31.722381  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:31.707686 10702 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:31.732590  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:31.709754 10583 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:31.733002 10329 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:31.733462  8824 raft_consensus.cc:2393] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:31.709856 10618 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:31.733608 10329 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:31.733927 10583 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:31.703675 10599 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:31.730737 10692 raft_consensus.cc:3055] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:31.579177  8673 rpcz_store.cc:276] 0709 13:02:28.888687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.570957 (+1682270us) service_pool.cc:224] Handling call
0709 13:02:30.606589 (+ 35632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.624481 (+ 17892us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:30.824123 (+199642us) log.cc:844] Serialized 14 byte log entry
0709 13:02:31.383293 (+559170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:31.383408 (+   115us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:31.383511 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.383646 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.383921 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":316805,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":174,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":157,"thread_start_us":538949,"threads_started":2,"wal-append.queue_time_us":858409}
I20260709 13:02:31.735939  8824 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:31.706539 10498 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:31.680613  9380 rpcz_store.cc:276] 0709 13:02:30.225341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.239828 (+ 14487us) service_pool.cc:224] Handling call
0709 13:02:31.050431 (+810603us) spinlock_profiling.cc:243] Waited 8.03 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:31.535798 (+485367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18472192}
I20260709 13:02:31.752180 10329 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:31.710251 10652 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:31.736917  8756 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:31.758678  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 223) took 1590 ms. Trace:
I20260709 13:02:31.759148  8756 rpcz_store.cc:276] 0709 13:02:30.168134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.714878 (+1546744us) service_pool.cc:224] Handling call
0709 13:02:31.736742 (+ 21864us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.758365 (+ 21623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.758569 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.740649  8748 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:31.712421  9384 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:31.729768 10651 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:31.772140  8828 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:31.723093  9383 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:31.750046 10498 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:31.773608 10498 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:31.773701 10581 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:31.774693 10694 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:31.775290 10581 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:31.772336 10723 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:02:31.778678 10720 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:02:31.743647 10630 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:31.727478 10724 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.085s	user 0.001s	sys 0.001s
I20260709 13:02:31.773502 10651 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:31.758886 10652 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:31.800436  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.812577  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.782883 10329 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:31.817617  8647 leader_election.cc:400] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
I20260709 13:02:31.818086  8647 leader_election.cc:403] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:31.823287  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.828635 10729 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
I20260709 13:02:31.744961 10692 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:02:31.830818  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 603058us
W20260709 13:02:31.833201  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 115046us
I20260709 13:02:31.834107  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:31.831149  8751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1482) took 3427 ms (client timeout 2386 ms). Trace:
I20260709 13:02:31.828925  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:31.835023  8751 rpcz_store.cc:269] 0709 13:02:28.392617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.398708 (+  6091us) service_pool.cc:224] Handling call
0709 13:02:31.819378 (+3420670us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b540031d654. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:31.819622 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3403895680}
I20260709 13:02:31.836866  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:31.842145  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 225) took 1571 ms. Trace:
I20260709 13:02:31.842643  8756 rpcz_store.cc:276] 0709 13:02:30.271085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.821542 (+1550457us) service_pool.cc:224] Handling call
0709 13:02:31.841538 (+ 19996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.841765 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.841792 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:31.841836 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:31.841874 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.841929 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.842080 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.763315 10599 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:31.850867  8751 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:31.852003  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1904) took 1239 ms. Trace:
I20260709 13:02:31.852495  8751 rpcz_store.cc:276] 0709 13:02:30.612153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.839954 (+1227801us) service_pool.cc:224] Handling call
0709 13:02:31.850677 (+ 10723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.851712 (+  1035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.851935 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.373037  8749 rpcz_store.cc:276] 0709 13:02:29.923380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.490838 (+567458us) service_pool.cc:224] Handling call
0709 13:02:30.491213 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.491482 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:30.683616 (+192134us) log.cc:844] Serialized 14 byte log entry
0709 13:02:30.963283 (+279667us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:30.969439 (+  6156us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:30.976204 (+  6765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.980883 (+  4679us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.986517 (+  5634us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3165,"prepare.queue_time_us":247837,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":26436,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":15930,"spinlock_wait_cycles":9352192,"thread_start_us":377551,"threads_started":2,"wal-append.queue_time_us":280743}
I20260709 13:02:31.831765 10652 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:31.836802 10467 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:31.840371 10602 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:31.858476  8756 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:31.863499  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1905) took 1251 ms. Trace:
I20260709 13:02:31.863897  8756 rpcz_store.cc:276] 0709 13:02:30.612470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.843661 (+1231191us) service_pool.cc:224] Handling call
0709 13:02:31.858262 (+ 14601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.863171 (+  4909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.863416 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.821012 10329 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:31.848555 10729 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:31.886370  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 226) took 1238 ms. Trace:
W20260709 13:02:31.891464  8753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 1886) took 2163 ms (client timeout 2238 ms). Trace:
W20260709 13:02:31.893432  8753 rpcz_store.cc:269] 0709 13:02:29.728109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.948580 (+220471us) service_pool.cc:224] Handling call
0709 13:02:31.867174 (+1918594us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b5400138654. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:31.891359 (+ 24185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259801344}
I20260709 13:02:31.857613 10467 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:31.896257  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 60729.6us
I20260709 13:02:31.895366  8751 rpcz_store.cc:276] 0709 13:02:30.647839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.853324 (+1205485us) service_pool.cc:224] Handling call
0709 13:02:31.885453 (+ 32129us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.885765 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.885812 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:31.885886 (+    74us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:31.885942 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.886041 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.886275 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.904147  8830 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:31.857678 10737 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:02:31.852088 10583 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:31.897848  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:31.858038 10608 raft_consensus.cc:3055] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:02:31.912068  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.918875  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 227) took 1058 ms. Trace:
I20260709 13:02:31.919960  8753 rpcz_store.cc:276] 0709 13:02:30.860983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.895262 (+1034279us) service_pool.cc:224] Handling call
0709 13:02:31.918082 (+ 22820us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.918348 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.918392 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:31.918453 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:31.918509 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.918589 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.918793 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:31.922070  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:31.906630 10467 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:31.919468 10737 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:02:31.924405 10737 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:31.925457 10737 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:31.925900 10737 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:02:31.927358 10721 log.cc:927] Time spent T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:02:31.928606 10569 raft_consensus.cc:3055] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:31.929095 10569 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:31.913476 10608 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:31.930899  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:31.931640  9383 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:31.931486 10608 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:31.932585 10608 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:31.923111  8830 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:31.917496 10583 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:31.939199 10583 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:31.930047 10569 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:31.941908 10569 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:31.942937 10737 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:31.945776 10467 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:31.946811 10738 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:31.947407 10738 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:31.948321 10467 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:31.948484 10738 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:31.894721 10692 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:31.920548 10651 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:31.863615 10652 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:31.909498 10729 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:31.950361 10738 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:31.954075  8751 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:02:31.979382  8750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 1891) took 2173 ms (client timeout 2837 ms). Trace:
W20260709 13:02:32.006019  8750 rpcz_store.cc:269] 0709 13:02:29.805889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.106798 (+300909us) service_pool.cc:224] Handling call
0709 13:02:31.979082 (+1872284us) spinlock_profiling.cc:243] Waited 94.9 ms on lock 0x7b540018d8d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:31.979275 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218373504}
I20260709 13:02:31.981774  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1882) took 2257 ms. Trace:
I20260709 13:02:32.007963  8752 rpcz_store.cc:276] 0709 13:02:29.724246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.406082 (+681836us) service_pool.cc:224] Handling call
0709 13:02:30.406434 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.692036 (+285602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.981374 (+1289338us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b540034a654. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:31.981677 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":533254016}
W20260709 13:02:32.054751  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.056574 10728 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:32.059273  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.016564  8645 connection.cc:256] RPC call timeout handler was delayed by -1.2588097629999879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23860000000000003s for a response
W20260709 13:02:32.064589  8645 connection.cc:256] RPC call timeout handler was delayed by -1.1016380119999667s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19490000000000002s for a response
I20260709 13:02:32.067406  8830 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 4. Current term is 6. Ops: []
I20260709 13:02:32.068980  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1417) took 1089 ms. Trace:
I20260709 13:02:32.069510  8830 rpcz_store.cc:276] 0709 13:02:30.979762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.936649 (+956887us) service_pool.cc:224] Handling call
0709 13:02:32.067143 (+130494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.068594 (+  1451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.068897 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:32.071347  8645 connection.cc:256] RPC call timeout handler was delayed by -1.0960709949999909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
I20260709 13:02:32.073287  9386 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:02:32.073342  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 236557us
I20260709 13:02:31.976635  8749 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:02:31.974447 10692 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:31.957850 10560 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:31.976353 10651 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:31.975803 10707 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.053135 10583 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:31.982501 10652 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:31.968768 10599 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:32.063879 10640 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:02:32.109681  8793 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.187s (SENT)
I20260709 13:02:32.115451  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:32.076923 10531 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:32.088896 10701 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:32.119791  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 222036us
W20260709 13:02:32.079615 10669 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.371s	user 0.019s	sys 0.018s
I20260709 13:02:32.063185 10502 consensus_queue.cc:1059] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 13:02:32.125355 10669 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.418s	user 0.019s	sys 0.019s
I20260709 13:02:32.135594 10617 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:32.135797 10606 raft_consensus.cc:3055] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:32.127033 10599 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.082468 10498 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.137070  8824 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:32.138316 10498 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:32.141697 10638 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.127885 10633 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:32.145845  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.146039 10633 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.148478 10633 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:32.149009 10633 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.149488 10633 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.150770 10639 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:32.129884  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 240) took 1869 ms (client timeout 2087 ms). Trace:
I20260709 13:02:32.133440  8756 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:32.152524  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1910) took 1360 ms. Trace:
I20260709 13:02:32.152874  8756 rpcz_store.cc:276] 0709 13:02:30.792499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.864814 (+1072315us) service_pool.cc:224] Handling call
0709 13:02:32.132553 (+267739us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b5800083198. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:32.132877 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.152215 (+ 19338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.152459 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28304768}
I20260709 13:02:32.117712 10651 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:32.135110 10745 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:32.126217 10652 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:32.136224 10606 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:32.136677 10599 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.116575 10583 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.136204 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:60414 (request call id 77) took 4625 ms. Trace:
I20260709 13:02:32.133764 10707 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:32.151271 10729 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:32.159446 10730 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:02:32.057276 10728 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.084137 10672 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:32.172369  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 95161.5us
I20260709 13:02:32.118860  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:02:32.151463  8823 rpcz_store.cc:269] 0709 13:02:30.260057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.432145 (+172088us) service_pool.cc:224] Handling call
0709 13:02:32.129526 (+1697381us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400125f54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:32.129778 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2815779840}
I20260709 13:02:32.119534 10739 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:32.196607  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.119877 10531 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.227626  9355 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:32.153097  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:32.201125 10652 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.211099  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1422) took 1207 ms. Trace:
I20260709 13:02:32.232800  8830 rpcz_store.cc:276] 0709 13:02:31.004052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.070993 (+1066941us) service_pool.cc:224] Handling call
0709 13:02:32.085445 (+ 14452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:32.086135 (+   690us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:32.086182 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:32.086946 (+   764us) log.cc:844] Serialized 26 byte log entry
0709 13:02:32.199066 (+112120us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:32.199113 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.199191 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.210691 (+ 11500us) raft_consensus.cc:1685] finished
0709 13:02:32.210727 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.210987 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":39998,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"thread_start_us":69409,"threads_started":1,"wal-append.queue_time_us":84351}
I20260709 13:02:32.151302 10639 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.182253 10694 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.220183  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:32.121574 10640 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:32.224566  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.248991  8827 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:32.249758  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.123198  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1908) took 1331 ms. Trace:
I20260709 13:02:32.211727 10746 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:32.246217  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.255478 10531 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:32.256075 10599 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:32.256758  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:32.262681  9383 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:32.263909 10633 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:32.312238  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:32.313015  8681 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:32.279850  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.253252 10603 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:32.298136 10581 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
W20260709 13:02:32.285012  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 206116us
I20260709 13:02:32.285319 10755 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:32.285912 10713 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 1.454s	user 0.000s	sys 0.019s
I20260709 13:02:32.294332 10483 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:02:32.296437  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1202) took 2063 ms (client timeout 2076 ms). Trace:
I20260709 13:02:32.218007 10672 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:32.315915  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.320991  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 211388us
I20260709 13:02:32.226055 10747 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 13:02:32.324819  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:32.325297  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.325620  8679 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:32.321034  8645 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:32.327015  8827 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:32.329032  9355 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:32.330340  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:32.331094  8680 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:32.332183  8796 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:32.251174 10748 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:32.312844 10329 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:32.321786 10483 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.332183  8645 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:32.271735 10694 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.252919  8749 rpcz_store.cc:276] 0709 13:02:30.791799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.867705 (+1075906us) service_pool.cc:224] Handling call
0709 13:02:31.917407 (+ 49702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.082464 (+165057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.121892 (+ 39428us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.339866  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:32.335171 10531 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:32.342911  8681 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:32.325696  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:32.321640  8821 rpcz_store.cc:269] 0709 13:02:30.233536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.240320 (+  6784us) service_pool.cc:224] Handling call
0709 13:02:32.296205 (+2055885us) spinlock_profiling.cc:243] Waited 909 ms on lock 0x7b54002288d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:32.296358 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2089835904}
I20260709 13:02:32.326493 10633 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:02:32.351464 10652 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:32.330242 10622 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:32.352797 10639 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:32.357872 10743 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:02:32.358003 10639 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:32.358992 10639 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:32.359501 10639 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:32.360101 10651 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.360424 10639 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.363379 10728 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:32.364571 10622 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:32.366932 10550 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:32.367435 10583 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:32.334715 10581 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:32.367492 10550 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:32.271515 10746 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:32.377238  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 161878us
W20260709 13:02:32.363533  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.380187  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1898) took 2402 ms. Trace:
I20260709 13:02:32.365823 10622 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.226943 10739 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.353761  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:32.380780  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 546731us
W20260709 13:02:32.472096  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.481969  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.327930 10755 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.382130  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 224) took 2213 ms. Trace:
I20260709 13:02:32.486464  8722 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:32.470105  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:32.442247  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 52719.9us
I20260709 13:02:32.483444  8647 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:32.190451 10669 rpcz_store.cc:276] 0709 13:02:27.511230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.511490 (+   260us) service_pool.cc:224] Handling call
0709 13:02:28.331807 (+820317us) spinlock_profiling.cc:243] Waited 56.9 ms on lock 0x7b58002069d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24550aa 00007f32e24d7447 00007f32e24f470a 00007f32e24f576a 00007f32e24f491b 00007f32e250003c 00007f32e24ff6b3 00007f32e24f79fa 00007f32e74bf4c9 00007f32e0a3bc54
0709 13:02:28.449802 (+117995us) log.cc:844] Serialized 494 byte log entry
0709 13:02:30.105033 (+1655231us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b54001d3e20. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2472142 00007f32e2452eaa 00007f32e2459d53 00007f32e2459cd9 00007f32e2459c69 00007f32e2459c31 00007f32e2458dfd 000000000038f354 000000000038f199 00007f32df40c475
0709 13:02:32.135969 (+2030936us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7767,"raft.queue_time_us":18778,"raft.run_cpu_time_us":37742,"raft.run_wall_time_us":3717887,"spinlock_wait_cycles":150187392,"thread_start_us":85151,"threads_started":1,"wal-append.queue_time_us":180733,"wal-append.run_cpu_time_us":3775,"wal-append.run_wall_time_us":1121831}
W20260709 13:02:32.399668  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 60402.1us
I20260709 13:02:32.485867  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:32.484344  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:32.488369  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:32.493924  8753 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:32.495939  8747 rpcz_store.cc:276] 0709 13:02:29.977508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.523859 (+1546351us) service_pool.cc:224] Handling call
0709 13:02:31.524360 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.524651 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:31.758319 (+233668us) log.cc:844] Serialized 14 byte log entry
0709 13:02:31.945864 (+187545us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b1800078b20. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:02:32.379759 (+433895us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:32.379819 (+    60us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:32.379875 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.379950 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.380118 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":571868,"prepare.queue_time_us":370208,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":562462,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":70315,"spinlock_wait_cycles":30293888,"thread_start_us":266320,"threads_started":2,"wal-append.queue_time_us":161447}
I20260709 13:02:32.501675  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:32.524247  8751 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:32.501873  8823 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:32.502092  9389 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:32.502274 10692 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:32.481796  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 245) took 2192 ms. Trace:
I20260709 13:02:32.485672  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:32.503125  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:32.503784 10754 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:02:32.504806  9385 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 406) took 5758 ms (client timeout 1752 ms). Trace:
I20260709 13:02:32.383981 10591 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.506381 10396 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:32.487259  8754 rpcz_store.cc:276] 0709 13:02:30.168480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.717392 (+1548912us) service_pool.cc:224] Handling call
0709 13:02:31.717744 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.718484 (+   740us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:32.061300 (+342816us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.381586 (+320286us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:32.381659 (+    73us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:32.381735 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.381836 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.382031 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":152769,"prepare.queue_time_us":226094,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":153612,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"thread_start_us":492410,"threads_started":2,"wal-append.queue_time_us":192865}
W20260709 13:02:32.539479 10569 consensus_meta.cc:287] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.597s	user 0.015s	sys 0.001s
I20260709 13:02:32.511029  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:32.515156  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:32.658041  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:32.381986  9382 raft_consensus.cc:2410] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:02:32.517877  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 1.69178e+06us
I20260709 13:02:32.380996 10467 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:32.375353  8673 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:02:32.380259 10651 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:32.375262 10581 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:32.379760 10699 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:32.526912  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.530821  8676 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:32.533152  9388 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:02:32.689891  8722 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.057s (SENT) [suppressed 37 similar messages]
I20260709 13:02:32.383385 10470 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.534716  8749 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:02:32.535526  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:32.693858  8679 raft_consensus.cc:2368] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4.
I20260709 13:02:32.371093 10753 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:32.540205  8750 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:32.509330 10747 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:32.340137 10748 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.389006 10550 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.522364 10618 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.507319  8566 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet abd86f4e9b544531a7cbfa0042b4fa6c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:02:32.548280  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:32.719262  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:32.545877 10754 consensus_queue.cc:1243] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:02:32.530871 10692 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.552559  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:32.729197  8754 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.555176  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:32.730520  8751 raft_consensus.cc:2393] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:32.730208  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:32.537086 10599 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:02:32.579341  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:32.586555  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:32.739552  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:32.606993  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.532912 10652 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.649173 10729 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:32.651569 10619 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:32.536345  8822 rpcz_store.cc:276] 0709 13:02:30.288643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.549850 (+261207us) service_pool.cc:224] Handling call
0709 13:02:30.626482 (+ 76632us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.642897 (+ 16415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.642939 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:30.643561 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 13:02:30.695037 (+ 51476us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:30.698818 (+  3781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.766024 (+ 67206us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.146308 (+1380284us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:32.469381 (+323073us) raft_consensus.cc:1685] finished
0709 13:02:32.480695 (+ 11314us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.480970 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58436992,"thread_start_us":43731,"threads_started":1,"wal-append.queue_time_us":115793}
I20260709 13:02:32.651080  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:32.730185 10754 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:32.554381 10758 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:02:32.530548 10738 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:02:32.573202 10752 consensus_queue.cc:1059] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:02:32.531143 10436 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:02:32.658118  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:32.763861  8827 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:32.764820  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:32.766062  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:32.517444  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:32.658900  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.660043  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:32.659947  8719 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:32.502395 10286 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:32.654533 10640 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:32.694087  8753 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:32.651855 10483 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.651144 10672 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:32.544308  9385 rpcz_store.cc:269] 0709 13:02:26.745871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.749876 (+  4005us) service_pool.cc:224] Handling call
0709 13:02:27.357384 (+607508us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:32.504506 (+5147122us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b5400520b54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:32.504680 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4436342016}
I20260709 13:02:32.663108 10694 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:32.688684 10372 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.662344 10467 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:32.717939  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:32.720635  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:32.721747 10746 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:32.693905 10589 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.539076  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:32.731243  9386 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:02:32.795682  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:32.749122 10729 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.657202 10591 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.748780 10652 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.709900 10745 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:32.743182  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:32.744443  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:32.718238 10747 raft_consensus.cc:697] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:32.695389 10699 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.760844 10754 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.695873 10470 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.732697 10692 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:32.653750  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 436) took 2103 ms. Trace:
W20260709 13:02:32.733462  8566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 223920us
I20260709 13:02:32.811476 10752 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.829979 10752 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.831564  8826 raft_consensus.cc:2410] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:32.831686 10591 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:32.798575 10589 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.826386  9384 rpcz_store.cc:276] 0709 13:02:30.550353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.551729 (+  1376us) service_pool.cc:224] Handling call
0709 13:02:30.639122 (+ 87393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.765969 (+1126847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.653411 (+887442us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001a4854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:32.653665 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2442792832}
I20260709 13:02:32.833132 10752 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:32.834981 10589 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:32.835227 10470 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:32.806921 10372 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.817731  8823 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:32.838819 10653 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.839728 10653 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.838513 10372 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:32.758544  8754 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:32.726125 10618 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.762708  9388 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:02:32.768733  8681 raft_consensus.cc:2410] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:02:32.849581  9388 pending_rounds.cc:85] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting all ops after (but not including) 4
I20260709 13:02:32.849956  9388 pending_rounds.cc:107] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:02:32.850261  9388 raft_consensus.cc:2889] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:32.768687  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 108845us
I20260709 13:02:32.767252  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:32.752847  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:32.772871 10286 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:32.776875  8756 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:32.777614  8794 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:32.780843  8750 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:32.776219 10640 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.790858 10651 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:32.734398 10581 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.797003  8679 raft_consensus.cc:2393] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:32.795616 10467 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.798213 10531 raft_consensus.cc:3055] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:32.773556 10755 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:32.773317 10630 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:02:32.811836  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.818130  9386 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:32.798951  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:32.881139  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 71902.4us
W20260709 13:02:32.746834  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 56982.8us
I20260709 13:02:32.882359  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:32.828095  8751 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:32.848004 10606 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:32.855085  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 77547.3us
I20260709 13:02:32.881831  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:32.884335  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:32.885582  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:32.850472  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:32.887279  8681 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:32.806424 10729 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:32.803434  8822 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:32.861830  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:32.799312  8821 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:32.859794 10286 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.866531 10695 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:32.868538 10754 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:32.871296 10747 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.872668 10652 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:32.874347 10550 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.821424 10745 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:32.860461 10638 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.718s	user 0.010s	sys 0.012s
I20260709 13:02:32.799708 10753 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.916884 10753 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.861680 10591 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:02:32.907116  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:32.909861  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:32.864912 10752 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:32.865278  8673 raft_consensus.cc:2376] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5. Re-sending same reply.
I20260709 13:02:32.889906 10581 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.879669 10531 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:32.905426  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:32.945035  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:32.946151  8681 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:32.890100  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:32.935883  8677 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:32.905622 10396 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:32.941779  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:32.948616  8756 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.949167  8827 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:32.906308 10765 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:32.950321  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:32.912791 10671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.371s	user 0.015s	sys 0.031s
I20260709 13:02:32.879850 10651 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.890659 10766 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:32.979425 10766 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.982102 10766 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:32.919735 10606 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:32.983866 10591 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:32.955840  8672 raft_consensus.cc:2410] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:32.883630 10755 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:32.986959 10755 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:32.987280  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:32.987859  8679 raft_consensus.cc:2410] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:02:32.988154  8672 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:32.989331  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:32.989701 10755 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:32.939261 10752 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.992522 10752 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:32.996214 10618 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:32.890172 10589 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:02:32.999022  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1411) took 2050 ms. Trace:
I20260709 13:02:32.999487  8828 rpcz_store.cc:276] 0709 13:02:30.948850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.730469 (+781619us) service_pool.cc:224] Handling call
0709 13:02:31.730836 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.781258 (+ 50422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.998727 (+1217469us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x7b54003abbd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:32.998949 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70359808}
I20260709 13:02:33.001868  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:33.002554  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.003290  8752 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:32.865442 10640 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:32.864316 10470 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
W20260709 13:02:32.954770 10671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.413s	user 0.015s	sys 0.031s
I20260709 13:02:32.936617 10748 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:33.003422  8676 raft_consensus.cc:2393] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:33.031718  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:33.032853  8676 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:32.955075  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:33.011426  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.035768  8752 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:33.038007  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:32.906144 10762 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:33.061201  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:33.034264  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:33.063750  8677 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:33.064447  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.065467  8752 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:33.066761 10678 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.007203 10753 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:33.067991 10470 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:33.060690 10762 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:33.068852 10762 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:33.070056 10752 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:33.072638 10752 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:33.005774 10692 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:33.078917 10531 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:33.082252 10550 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:33.085435 10651 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:33.093400 10640 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:33.096692  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:33.140230  8680 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:32.950424 10745 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:33.019604  8756 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:33.068742 10470 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:33.107196  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:33.145681  8676 raft_consensus.cc:2393] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:33.115444  8681 raft_consensus.cc:2376] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3. Re-sending same reply.
I20260709 13:02:33.022176  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:33.150209  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 267171us
I20260709 13:02:33.131470 10531 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:33.154834  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:33.163324  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 281073us
I20260709 13:02:33.167456  8681 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:33.171211  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:33.067363 10678 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:33.176192  8681 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:33.142762  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:33.170823 10695 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:33.165951 10699 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:33.187578  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:33.189301  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:33.190039  8822 raft_consensus.cc:2376] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4. Re-sending same reply.
I20260709 13:02:33.191439  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:33.192198  8822 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:33.197484  8673 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:33.197822  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:33.199509 10768 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:02:33.199208 10550 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:33.175081 10286 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:33.200987 10550 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.201507 10550 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:33.202864  8822 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:33.196379 10692 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:33.204034  8756 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:33.195148 10470 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:33.204190  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:33.205122 10470 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.205340  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:33.205838  8680 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:02:33.206223  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.208698 10739 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:33.209384 10765 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:33.199271  8673 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:02:33.202800 10618 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:33.192260  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:33.205953  8756 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:33.202425 10286 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:33.210189  8672 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:33.250212  8680 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:33.187670 10467 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:33.229329 10762 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:33.205090  8822 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:33.230806 10765 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:33.248970 10777 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 13:02:33.275898  8826 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1187) took 4485 ms (client timeout 2057 ms). Trace:
W20260709 13:02:33.276592  8826 rpcz_store.cc:269] 0709 13:02:28.790644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.413741 (+623097us) service_pool.cc:224] Handling call
0709 13:02:33.275555 (+3861814us) spinlock_profiling.cc:243] Waited 3.42 s on lock 0x7b54002189d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:33.275785 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7857351552}
W20260709 13:02:33.280402  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.271404 10692 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.237943 10745 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:33.238426 10618 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:33.282578  8680 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:33.259675 10589 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:33.320323  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 128156us
I20260709 13:02:33.335364  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.307206  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:33.321537  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.326915  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:33.338171  8752 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.328102 10745 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:33.329334  8681 raft_consensus.cc:3055] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:33.343492  8681 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:02:33.295970  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.269248 10630 raft_consensus.cc:697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:33.282436 10765 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:33.331276  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.332084  8756 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:33.335215 10692 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:33.340137  8673 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:02:33.318367 10467 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:33.344321  8681 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:33.350544  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.373028  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.369158  8752 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:33.346887  8754 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:33.370661  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:33.350309  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:33.408197  8826 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:33.408420  9388 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:33.408641  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.348172  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:33.415468  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:33.416109  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:33.418120  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.418473  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.420604  8681 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:33.421823  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:33.422576  8826 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
W20260709 13:02:33.357244 10722 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.676s	user 0.004s	sys 0.001s
W20260709 13:02:33.361985 10761 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.543s	user 0.000s	sys 0.007s
I20260709 13:02:33.417886 10640 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:33.455657  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:33.424563  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:33.410177  8673 pending_rounds.cc:85] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 2
I20260709 13:02:33.464377  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:33.352296 10762 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:33.346592 10775 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.432387  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:33.467818  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:33.472692  8826 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:33.353915 10695 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:33.486761  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:33.486864 10695 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.489425  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:33.364408 10653 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:33.361377 10765 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:33.471118  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.359265 10639 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.108s
I20260709 13:02:33.508811  8750 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:33.478371  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.478919 10630 consensus_queue.cc:237] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:33.481526  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:33.340178 10699 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.120s
I20260709 13:02:33.527555  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:33.528462  8827 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:33.529987  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:33.473769 10640 raft_consensus.cc:697] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:33.485633  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:33.498716  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.500808  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.536944  8827 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:33.538002  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1433) took 1003 ms. Trace:
I20260709 13:02:33.360615 10286 consensus_queue.cc:1059] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.114s
I20260709 13:02:33.467350  8673 pending_rounds.cc:107] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:02:33.539580  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:33.544623  8828 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:33.546685  8828 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:33.547694  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1211) took 1031 ms. Trace:
I20260709 13:02:33.548221  8828 rpcz_store.cc:276] 0709 13:02:32.516352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.545711 (+1029359us) service_pool.cc:224] Handling call
0709 13:02:33.546137 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.547316 (+  1179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.547616 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:33.541409  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.551496  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 219) took 4132 ms. Trace:
W20260709 13:02:33.470839  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54805.4us
I20260709 13:02:33.551999  8755 rpcz_store.cc:276] 0709 13:02:29.418786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.420588 (+  1802us) service_pool.cc:224] Handling call
0709 13:02:29.431550 (+ 10962us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.432088 (+   538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.432135 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:30.930262 (+1498127us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b2c000f0558. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e0df2a11 00007f32e0e02372 00007f32e2534a01 00007f32e24f943a 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2
0709 13:02:30.930931 (+   669us) log.cc:844] Serialized 26 byte log entry
0709 13:02:31.202909 (+271978us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:31.202980 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.203090 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.309098 (+2106008us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:33.551226 (+242128us) raft_consensus.cc:1685] finished
0709 13:02:33.551253 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.551423 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":531,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":128,"spinlock_wait_cycles":18362496,"thread_start_us":233874,"threads_started":1,"wal-append.queue_time_us":650159}
I20260709 13:02:33.540270  8827 rpcz_store.cc:276] 0709 13:02:32.534721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.529634 (+994913us) service_pool.cc:224] Handling call
0709 13:02:33.537913 (+  8279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.474004 10550 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:33.362840 10771 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 13:02:33.557328  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:33.546373  8673 raft_consensus.cc:2889] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:33.560034  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1213) took 1043 ms. Trace:
I20260709 13:02:33.555199  8827 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:33.571578  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1212) took 1054 ms. Trace:
I20260709 13:02:33.572191  8827 rpcz_store.cc:276] 0709 13:02:32.516662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.554198 (+1037536us) service_pool.cc:224] Handling call
0709 13:02:33.554568 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.571092 (+ 16524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.571479 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.574232  8827 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:33.562361  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:33.575335  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.576800  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.578598  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1214) took 1061 ms. Trace:
I20260709 13:02:33.579187  8827 rpcz_store.cc:276] 0709 13:02:32.517204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.573342 (+1056138us) service_pool.cc:224] Handling call
0709 13:02:33.573860 (+   518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.578175 (+  4315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.578514 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.574721  8828 rpcz_store.cc:276] 0709 13:02:32.516926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.555786 (+1038860us) service_pool.cc:224] Handling call
0709 13:02:33.556098 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.556461 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.559612 (+  3151us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.559671 (+    59us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:33.559727 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.559802 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.559955 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":522}
I20260709 13:02:33.582222  8828 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:33.583279  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1426) took 1063 ms. Trace:
I20260709 13:02:33.583777  8828 rpcz_store.cc:276] 0709 13:02:32.519935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.581408 (+1061473us) service_pool.cc:224] Handling call
0709 13:02:33.581900 (+   492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.582859 (+   959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.583189 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.359149 10770 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.197s
I20260709 13:02:33.587755  8755 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:33.588853  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.591320  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:33.599834  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:33.601424  8755 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:33.602859  8826 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:33.611454  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1223) took 1077 ms. Trace:
I20260709 13:02:33.558338 10606 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.586256  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1427) took 1066 ms. Trace:
I20260709 13:02:33.612030  8826 rpcz_store.cc:276] 0709 13:02:32.534026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.423335 (+889309us) service_pool.cc:224] Handling call
0709 13:02:33.611381 (+188046us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.507611 10775 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:33.614705  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1416) took 2664 ms. Trace:
I20260709 13:02:33.621392  8824 rpcz_store.cc:276] 0709 13:02:30.950349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.750903 (+800554us) service_pool.cc:224] Handling call
0709 13:02:31.752066 (+  1163us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.137520 (+385454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.614364 (+1476844us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b54003ac0d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:33.614606 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":885028608}
I20260709 13:02:33.623471  8824 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:33.624184  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 265) took 1087 ms. Trace:
I20260709 13:02:33.624519  8824 rpcz_store.cc:276] 0709 13:02:32.537057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.622724 (+1085667us) service_pool.cc:224] Handling call
0709 13:02:33.623085 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.623896 (+   811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.624122 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.625352  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.609341 10590 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:33.617111 10286 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.618299 10678 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:33.610052  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 52808.4us
W20260709 13:02:33.638301  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 138760us
I20260709 13:02:33.618429  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1429) took 1097 ms. Trace:
I20260709 13:02:33.645522  8826 rpcz_store.cc:276] 0709 13:02:32.520853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.617376 (+1096523us) service_pool.cc:224] Handling call
0709 13:02:33.617746 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.617975 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.618009 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.618096 (+    87us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:33.618137 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.618203 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.618360 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.616092  8828 rpcz_store.cc:276] 0709 13:02:32.520236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.584941 (+1064705us) service_pool.cc:224] Handling call
0709 13:02:33.585422 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.585722 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.585770 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.585844 (+    74us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:33.585904 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.585990 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.586166 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.615345  8681 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.641100  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:33.649019  9386 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:33.623484 10640 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:33.664608 10603 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:33.668120  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.669701  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.671342 10618 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:33.637154 10590 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.679064 10590 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:33.685423 10775 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:33.670373  8749 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:33.693126  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.695464  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:33.704859  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.705689  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:33.717036  8749 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:33.718392  8749 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:33.714926 10637 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:02:33.723836  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:33.724403  8749 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:33.621434 10606 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:33.731029  8749 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:33.722702 10470 consensus_meta.cc:287] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.004s
I20260709 13:02:33.733150  8749 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:33.736397  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 258) took 1913 ms. Trace:
I20260709 13:02:33.736954  8830 rpcz_store.cc:276] 0709 13:02:31.823248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.236869 (+413621us) service_pool.cc:224] Handling call
0709 13:02:32.285360 (+ 48491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.337710 (+ 52350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:32.337741 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:32.337784 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:32.337814 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.736138 (+1398324us) spinlock_profiling.cc:243] Waited 52.1 ms on lock 0x7b5400126454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:33.736200 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.736330 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119912704}
W20260709 13:02:33.733322  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.743831  8828 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:02:33.746189  8749 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:33.747480  8828 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:33.749190  8828 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:33.725040 10717 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:33.654093 10286 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:33.724570 10745 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:33.757522  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.759007 10737 raft_consensus.cc:3055] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:33.759527 10737 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:33.760615 10737 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:33.761915 10737 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:33.765523  8748 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:33.776482  8748 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:33.723168 10694 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:02:33.770495  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:33.777930  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:33.784015  8748 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:02:33.753500  8749 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.751904 10717 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.788355  8748 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:33.789163  8755 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:33.789790  8749 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:33.801463  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:33.818271  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:33.778137 10622 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:33.825793  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:33.793752 10717 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:33.833832  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:33.837275  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:33.838366  8828 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:33.842391  8830 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:33.855406  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:33.859347  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.860538  8754 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:33.866987  8748 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:33.845148  8821 raft_consensus.cc:2410] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:33.865466  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:33.871727  8828 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:33.873111  8754 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:33.868824  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.874462  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:33.877560  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 90269.4us
I20260709 13:02:33.878248 10797 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:33.882601 10797 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:33.828912 10622 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:33.885845  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.889405  8749 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:33.887465  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:33.892146  8754 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:33.888732  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:33.893772  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.894205  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:33.896193  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:33.899462  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.902701  8749 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:33.876613  8748 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:33.887584  8828 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:33.895591  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:33.888106 10640 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.910810  8754 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:33.923233  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:33.960291  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 118085us
I20260709 13:02:33.961408  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:33.962872  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:33.923817  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:33.963557  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:33.958819  8673 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:33.964288  9388 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:33.965340  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:33.975800  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:33.976569  8673 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:33.902428 10599 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:33.904167 10765 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:33.883683 10797 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:33.980245  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:33.989411  8680 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:33.902149 10606 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:33.964098  8679 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:33.959071 10729 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:33.982481  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 86970.5us
I20260709 13:02:33.884433 10695 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:33.985777  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:33.985766  8676 raft_consensus.cc:1275] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:33.991640 10286 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:33.981539 10640 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:33.996614 10765 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:33.999235  8794 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:34.011428  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:02:34.011869  8677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1326) took 5107 ms (client timeout 1766 ms). Trace:
W20260709 13:02:34.039219  8677 rpcz_store.cc:269] 0709 13:02:28.904868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.171749 (+266881us) service_pool.cc:224] Handling call
0709 13:02:34.011600 (+4839851us) spinlock_profiling.cc:243] Waited 3.7 s on lock 0x7b54003cf854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:34.011753 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8517543808}
I20260709 13:02:34.028504  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.043061  8680 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:34.020776  8675 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:34.023852  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 288) took 1711 ms. Trace:
W20260709 13:02:34.002789 10792 log.cc:927] Time spent T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:02:34.070798  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 71647.3us
I20260709 13:02:34.074944  8678 rpcz_store.cc:276] 0709 13:02:32.312366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.312485 (+   119us) service_pool.cc:224] Handling call
0709 13:02:32.312803 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.732690 (+419887us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:32.732722 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:32.732769 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:32.732802 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.023485 (+1290683us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b54002562d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:34.023554 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.023762 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":965182208}
I20260709 13:02:34.083838  8674 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:34.084390  8794 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:34.086323  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:33.999687 10745 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:34.006644  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:34.004802 10653 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.021435  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:34.009511 10695 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:34.008172 10483 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:34.104329  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:34.015862  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1222) took 1496 ms (client timeout 1627 ms). Trace:
I20260709 13:02:34.012630 10692 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:34.129218  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:34.130137  8679 raft_consensus.cc:2376] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5. Re-sending same reply.
W20260709 13:02:34.079857  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 181722us
I20260709 13:02:34.008404 10622 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:34.024399 10640 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:34.022012  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.006171  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:34.038316 10796 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:34.026760 10765 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.112416  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:34.116385  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:34.151564  8828 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:34.152916  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:34.117285  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:34.153970  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 67753.7us
I20260709 13:02:34.154059  8830 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
W20260709 13:02:34.155956  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 134220us
I20260709 13:02:34.156697  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:34.122735  8674 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:34.124953 10483 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:34.128064  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:34.132581  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:34.168128  8681 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:34.131565  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:34.102777 10669 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:34.102083 10745 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.024659 10729 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:34.089169 10804 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:34.147439  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.179580  8680 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:34.148202  8673 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:34.170663  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:34.172297  8829 raft_consensus.cc:2410] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:34.174001  8794 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:34.176198  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1438) took 1240 ms. Trace:
I20260709 13:02:34.181780  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:34.183004  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:34.121498 10653 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:34.195225  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:34.188870  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.187780  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:34.182057  8826 rpcz_store.cc:276] 0709 13:02:32.936113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.647090 (+710977us) service_pool.cc:224] Handling call
0709 13:02:33.706976 (+ 59886us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.711252 (+  4276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.711290 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:33.711872 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:02:33.833808 (+121936us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:33.833878 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.833992 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.004425 (+170433us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b48000cd158. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:02:34.155330 (+150905us) raft_consensus.cc:1685] finished
0709 13:02:34.155365 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.176122 (+ 20757us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115,"raft.run_cpu_time_us":3320,"raft.run_wall_time_us":297120,"spinlock_wait_cycles":7418624,"thread_start_us":85466,"threads_started":1,"wal-append.queue_time_us":108377}
W20260709 13:02:34.151382  9380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 536) took 2080 ms (client timeout 1852 ms). Trace:
I20260709 13:02:34.178815 10745 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:34.151266 10803 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:34.169260 10796 raft_consensus.cc:697] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:34.153733  9382 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:34.148382 10622 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:02:34.206601  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.201517  8672 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 2: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:34.193665 10606 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:34.197075 10590 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.517s	user 0.015s	sys 0.000s
I20260709 13:02:34.187474 10804 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.213922  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:34.123353  8823 rpcz_store.cc:269] 0709 13:02:32.519614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.560215 (+ 40601us) service_pool.cc:224] Handling call
0709 13:02:34.015632 (+1455417us) spinlock_profiling.cc:243] Waited 71.7 ms on lock 0x7b54003de0d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:34.015782 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165006080}
I20260709 13:02:34.207194  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:34.229904  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 56004us
I20260709 13:02:34.232501  8830 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:34.244341  8830 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:02:34.233242  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.192986 10805 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.241057  8829 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:34.254004  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:34.255179  8826 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
W20260709 13:02:34.258176  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.263765 10801 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:02:34.267227 10798 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:02:34.269601 10797 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:34.272725 10790 log.cc:927] Time spent T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 13:02:34.280359  8674 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:34.282907  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.282796  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.284722  8681 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:34.286128 10812 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:34.286947 10812 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.287472 10812 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:34.288197  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 44411.5us
I20260709 13:02:34.289250  8794 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:34.292511 10805 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.296506 10805 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:34.297583 10805 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:34.311280  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:34.313331 10483 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.314255 10483 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:34.284166  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.208022 10803 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:34.239650 10804 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:34.349835  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.350344  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.351410  8679 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:34.354560  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:34.354830  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:34.356000  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.357535  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:34.358388  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.205607  9380 rpcz_store.cc:269] 0709 13:02:32.070525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.073120 (+  2595us) service_pool.cc:224] Handling call
0709 13:02:34.006467 (+1933347us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:34.148298 (+141831us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62236416}
I20260709 13:02:34.356289  8679 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:34.359402  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:34.328653 10669 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:34.361966  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:34.332077 10569 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:34.364454  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.364636 10569 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.365334 10569 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:34.366129  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.367065  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.367663 10591 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:34.370931 10591 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:34.371367 10591 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.371711 10591 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:34.372910  8822 consensus_meta.cc:287] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.084s	user 0.012s	sys 0.002s
I20260709 13:02:34.373346  8822 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:34.374068  8827 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:34.374810  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1228) took 1651 ms (client timeout 2102 ms). Trace:
I20260709 13:02:34.379634 10796 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:34.381213 10591 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.375178  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:34.375272 10729 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:34.329330 10803 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:34.383522 10286 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:34.384042  8722 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:34.385164 10630 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:34.377857  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.387136  8680 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:34.336246 10738 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:02:34.384089 10286 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:02:34.378860  8822 rpcz_store.cc:269] 0709 13:02:32.723575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.203902 (+480327us) service_pool.cc:224] Handling call
0709 13:02:34.374727 (+1170825us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.403141  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:34.355283 10717 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.512s	user 0.005s	sys 0.008s
I20260709 13:02:34.405906 10717 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:34.388218  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1338) took 4256 ms. Trace:
I20260709 13:02:34.389348 10591 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:34.389931 10583 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:34.424654 10583 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:02:34.341056 10694 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.050s	sys 0.045s
I20260709 13:02:34.244851 10590 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:34.391518  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1916) took 1901 ms. Trace:
I20260709 13:02:34.428354  8753 rpcz_store.cc:276] 0709 13:02:32.490040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.773598 (+283558us) service_pool.cc:224] Handling call
0709 13:02:32.773932 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.836106 (+ 62174us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:32.836140 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:32.836188 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:32.836228 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.391167 (+1554939us) spinlock_profiling.cc:243] Waited 61.7 ms on lock 0x7b5400156dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:34.391229 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.391444 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3601,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":142019456}
I20260709 13:02:34.388244 10286 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:34.395961 10669 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.406489  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1224) took 1865 ms. Trace:
W20260709 13:02:34.441610  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 151134us
I20260709 13:02:34.377535 10653 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:34.377555  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:34.445315  8751 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:34.407716  9386 raft_consensus.cc:2368] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4.
W20260709 13:02:34.421072  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.247182 10606 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:34.426338 10694 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.051s	sys 0.046s
I20260709 13:02:34.460595  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1520) took 1125 ms. Trace:
I20260709 13:02:34.440428  8824 rpcz_store.cc:276] 0709 13:02:32.541405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.640398 (+1098993us) service_pool.cc:224] Handling call
0709 13:02:33.640895 (+   497us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:33.641390 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.641426 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:33.666245 (+ 24819us) log.cc:844] Serialized 52 byte log entry
0709 13:02:33.824076 (+157831us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:33.901538 (+ 77462us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.041746 (+140208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.041981 (+   235us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.126393 (+ 84412us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b1800369460. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:02:34.405984 (+279591us) raft_consensus.cc:1685] finished
0709 13:02:34.406153 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.406395 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":163219,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":2057,"raft.queue_time_us":89297,"raft.run_cpu_time_us":485,"raft.run_wall_time_us":67979,"spinlock_wait_cycles":10150144,"thread_start_us":331006,"threads_started":3,"wal-append.queue_time_us":239183}
I20260709 13:02:34.463071  8751 rpcz_store.cc:276] 0709 13:02:33.335703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.377267 (+1041564us) service_pool.cc:224] Handling call
0709 13:02:34.460486 (+ 83219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:34.452972  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:34.451956  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:34.483719  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1425) took 1964 ms. Trace:
I20260709 13:02:34.484597  8827 rpcz_store.cc:276] 0709 13:02:32.519642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.587049 (+1067407us) service_pool.cc:224] Handling call
0709 13:02:34.373864 (+786815us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.374438 (+   574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.483325 (+108887us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b540032f850. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:34.483605 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1754222464}
W20260709 13:02:34.486963  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 50320.7us
W20260709 13:02:34.450590 10811 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.288s	user 0.000s	sys 0.006s
W20260709 13:02:34.506788  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 65465.5us
I20260709 13:02:34.461268 10653 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.511099  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:34.513322 10467 raft_consensus.cc:3055] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:34.467167 10483 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:34.466401 10531 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.439860 10329 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:02:34.495390 10717 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:34.532660  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 145750us
I20260709 13:02:34.466760  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:34.519336 10467 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:34.553687 10603 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:34.439347 10583 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.525000 10795 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:02:34.542806 10531 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.526629 10653 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.565907 10603 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:34.589334 10804 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.589725 10812 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.590217 10803 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.592134 10812 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:34.625083  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.408999  8675 rpcz_store.cc:276] 0709 13:02:30.132064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.840996 (+708932us) service_pool.cc:224] Handling call
0709 13:02:34.020322 (+3179326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.044805 (+ 24483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.387953 (+343148us) spinlock_profiling.cc:243] Waited 3.18 s on lock 0x7b54003cf850. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:34.388150 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7302580480}
I20260709 13:02:34.632213  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:34.427994  8748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.002s	sys 0.007s
W20260709 13:02:34.643275  8748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.002s	sys 0.007s
I20260709 13:02:34.643797  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:34.432722  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:34.645684  8672 raft_consensus.cc:2410] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:34.646153  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:34.647279  8749 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:34.646301  8751 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:34.432996  8793 connection.cc:256] RPC call timeout handler was delayed by -1.7978912319999836s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29209999999999997s for a response
W20260709 13:02:34.652603  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.653770  8793 connection.cc:256] RPC call timeout handler was delayed by -1.6264625019999812s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2979s for a response
I20260709 13:02:34.536693  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1418) took 3550 ms. Trace:
I20260709 13:02:34.658550  8825 rpcz_store.cc:276] 0709 13:02:30.980453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.983577 (+1003124us) service_pool.cc:224] Handling call
0709 13:02:31.984163 (+   586us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:31.984823 (+   660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.984871 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:31.985672 (+   801us) log.cc:844] Serialized 26 byte log entry
0709 13:02:32.187377 (+201705us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:32.191719 (+  4342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.207926 (+ 16207us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.425421 (+2217495us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b24001edfb8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df2929c4 00007f32df290acb 00007f32df44bdc9 00007f32df44ba55 00007f32df44afb0 00007f32e20c1488 00007f32e20c1be8 00007f32e2099398 00007f32e209476a 00007f32e2093bb6
0709 13:02:34.478830 (+ 53409us) raft_consensus.cc:1685] finished
0709 13:02:34.485888 (+  7058us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.530642 (+ 44754us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2566,"raft.queue_time_us":16608,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":50497,"spinlock_wait_cycles":5404032,"thread_start_us":195725,"threads_started":1,"wal-append.queue_time_us":276872}
I20260709 13:02:34.648380  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:34.664268  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.668160  8793 connection.cc:256] RPC call timeout handler was delayed by -1.3907266319999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
W20260709 13:02:34.669790  8793 connection.cc:256] RPC call timeout handler was delayed by -1.2783595639999703s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
W20260709 13:02:34.671123  8793 connection.cc:256] RPC call timeout handler was delayed by -1.1819266319999997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
W20260709 13:02:34.675100  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 166961us
I20260709 13:02:34.683393 10531 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:34.684625 10531 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.674240 10467 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:34.560838 10502 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:02:34.685708 10531 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.689864  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:34.692407 10531 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:34.703804 10583 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:34.704329  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.703965 10599 consensus_queue.cc:1059] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:34.705520 10653 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:34.705782 10590 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:34.706077 10606 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:34.708050 10603 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:34.694003  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:34.702068  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:34.697992 10483 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:34.730142  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 102494us
I20260709 13:02:34.740782  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:34.742203  8754 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:34.742830  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:34.743913  8749 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:34.592535 10803 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:34.760258  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 58565.9us
I20260709 13:02:34.761637  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:34.749480  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:34.763314 10820 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:34.762985 10795 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:02:34.764744 10820 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:34.763299 10805 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:34.765344 10820 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.592038 10804 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:34.765833 10820 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:34.764422  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 70618.4us
I20260709 13:02:34.775477  8793 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:34.758073 10286 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.811650 10531 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:34.811916 10569 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.815832 10569 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:34.829463  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:34.830776  8823 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:34.837563  8579 catalog_manager.cc:5697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:34.845894  8579 catalog_manager.cc:5697] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 3, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 3 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
W20260709 13:02:34.851658  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.868302  8751 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:02:34.872182  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.873034  8754 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
W20260709 13:02:34.880986  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 105751us
I20260709 13:02:34.885511  8579 catalog_manager.cc:5697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 3 to 4, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 4 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:02:34.887272 10820 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:34.888077  8579 catalog_manager.cc:5697] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 2 to 3, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:34.897245  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:34.894690  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:34.920420 10286 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:34.921768  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.921895  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:02:34.910077 10745 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.693s	user 0.008s	sys 0.008s
I20260709 13:02:34.890434 10820 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:34.904443 10531 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:34.913414  8794 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:34.890583  8579 catalog_manager.cc:5697] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 2 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:34.927210 10745 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.934134 10745 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:34.907307  8823 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
W20260709 13:02:34.935746 10816 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.380s	user 0.004s	sys 0.001s
I20260709 13:02:34.937283  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:34.939216  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.940377  8754 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:34.944175  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:34.945786  8751 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:34.946178  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1220) took 2426 ms (client timeout 2040 ms). Trace:
I20260709 13:02:34.899712 10809 raft_consensus.cc:2955] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:02:34.953598  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 465311us
I20260709 13:02:34.954262  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:34.953648  8828 raft_consensus.cc:1275] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
W20260709 13:02:34.957792  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.958573 10755 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.974330  8756 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:02:34.974521 10755 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.975157  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:34.957679  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:34.977459  8675 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:34.977566  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 100669us
I20260709 13:02:34.978899 10820 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:34.959419  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1513) took 1761 ms. Trace:
I20260709 13:02:34.962486 10728 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:34.972795  8828 pending_rounds.cc:85] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 1
I20260709 13:02:34.987739 10820 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:34.998802 10669 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:34.999346 10728 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.999668 10669 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:35.000967  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:35.002096 10728 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:35.002849  8829 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:02:34.979828  8755 rpcz_store.cc:276] 0709 13:02:33.197797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.608515 (+410718us) service_pool.cc:224] Handling call
0709 13:02:34.958962 (+1350447us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b540034ba54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:34.959182 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374698880}
I20260709 13:02:34.988387  8828 pending_rounds.cc:107] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:02:34.996503 10820 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:34.977666 10755 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:35.051492  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 46087.5us
I20260709 13:02:34.997007  8823 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:35.053936  8794 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:35.035632  8749 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:35.170688 10805 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:35.074720  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:35.192431  8754 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:35.181651  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.167878  8826 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:35.185261  8828 raft_consensus.cc:2889] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:35.189567  8756 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:35.107142  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 241) took 1218 ms. Trace:
I20260709 13:02:35.051721  8579 catalog_manager.cc:5697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:02:34.997056  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:35.004153  8722 leader_election.cc:336] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.959s (SENT)
I20260709 13:02:35.091539  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:35.049162  8680 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 1.
W20260709 13:02:35.203263  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 149497us
I20260709 13:02:35.090387  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:35.184828 10755 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.997792  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:35.056658 10531 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:35.110458 10603 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:35.110858 10669 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:35.213936  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 276863us
I20260709 13:02:35.107077 10802 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:35.004444  8672 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:35.193912  8748 rpcz_store.cc:276] 0709 13:02:33.906092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.907376 (+  1284us) service_pool.cc:224] Handling call
0709 13:02:33.907795 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.908473 (+   678us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.908518 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:33.909241 (+   723us) log.cc:844] Serialized 26 byte log entry
0709 13:02:34.644404 (+735163us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:34.644558 (+   154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.644744 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.902606 (+257862us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:35.106701 (+204095us) raft_consensus.cc:1685] finished
0709 13:02:35.106723 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.107023 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2507,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":15762,"spinlock_wait_cycles":4408192,"thread_start_us":734331,"threads_started":1,"wal-append.queue_time_us":593929}
I20260709 13:02:35.105676 10467 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:35.056273 10756 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.095582  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:35.209920  8722 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:35.217947  8579 catalog_manager.cc:5697] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 2 to 4, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 4 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:35.229411  8751 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:35.228613  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:35.177784  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:35.214996  8825 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:35.233918 10329 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:02:34.969524  8821 rpcz_store.cc:269] 0709 13:02:32.519110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.939410 (+420300us) service_pool.cc:224] Handling call
0709 13:02:33.770230 (+830820us) spinlock_profiling.cc:243] Waited 5.22 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:34.945584 (+1175354us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b540009ddd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:34.945903 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69984000}
I20260709 13:02:35.234754  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:35.238837 10590 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.239974  9387 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:35.243466  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:35.244423 10470 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.245111 10771 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:02:35.245684  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.246133 10599 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:35.246062  9352 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:35.247222  8827 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:35.251817  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:35.252650  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:35.257107  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:35.246085  8679 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:35.267601  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:35.275564  8794 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:35.284811  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.270017  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:35.274788 10802 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:35.279930 10590 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:35.291100 10802 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:35.292256  8645 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:35.295176  8719 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:35.302881 10756 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:35.302956 10795 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:35.304324 10795 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:35.304566 10771 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:35.305872  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:35.305613 10795 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:35.304365 10470 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:35.307335  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:35.309259 10470 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:35.310981 10502 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:35.311290  8749 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
W20260709 13:02:35.312140  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 46416.1us
I20260709 13:02:35.313019  8749 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:35.314034 10653 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:35.279707 10805 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:35.287752 10755 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:35.314560  8794 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:35.321959 10755 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:35.314915  8752 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:35.345695 10771 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:35.347645  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1241) took 1232 ms. Trace:
I20260709 13:02:35.348165  8828 rpcz_store.cc:276] 0709 13:02:34.121603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.200849 (+ 79246us) service_pool.cc:224] Handling call
0709 13:02:34.952666 (+751817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.227454 (+274788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.347347 (+119893us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b540009ddd0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:35.347542 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1723260800}
I20260709 13:02:35.294229  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:35.310985 10591 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:35.306375 10756 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:35.277026 10599 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.314739 10653 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:35.351157 10590 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:35.351426 10599 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:35.352134 10653 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:35.323506  8675 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:35.355345  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:35.325630  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.316944 10805 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:35.359309  8748 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:35.359869 10805 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:35.316404 10839 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:02:35.326632 10738 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:35.318919  8579 catalog_manager.cc:5697] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:02:35.362221 10738 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.362673 10738 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.362939  8579 catalog_manager.cc:5697] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:35.364756  8579 catalog_manager.cc:5697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 3, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 3 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:35.327855  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 70922.5us
I20260709 13:02:35.366696 10583 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:35.328953  8750 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:35.367254  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 133619us
I20260709 13:02:35.367264  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:35.369004  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:35.370019  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:35.370076 10812 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:35.371012  8793 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:35.371230 10812 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:35.329031 10795 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:35.371114 10329 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:35.372493 10812 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:35.330250  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 326200us
I20260709 13:02:35.331091 10483 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:35.314891  8755 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
W20260709 13:02:35.375586  8722 leader_election.cc:336] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.040s (SENT)
I20260709 13:02:35.375805  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:35.376003  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:35.376794  8755 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:35.315475 10796 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:35.377576 10531 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:35.377897  9384 raft_consensus.cc:2410] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:35.379791  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:35.335949  8679 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:35.339092 10833 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:35.338351  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 43506.9us
I20260709 13:02:35.361704  8748 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:35.366850 10728 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:35.388990  8752 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:35.319742  8796 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:35.328027 10737 raft_consensus.cc:2764] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 13:02:35.393292  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 73665.6us
I20260709 13:02:35.394234  8750 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:35.395033  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:35.361588 10599 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:35.396175  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.389235 10728 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:35.382536 10833 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.389906  8579 catalog_manager.cc:5697] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:35.389549 10531 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.432747 10599 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:35.426754  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.431092  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.431331  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 137291us
I20260709 13:02:35.383765 10755 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:35.462007  8754 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:02:35.478857  8749 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:02:35.482218  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 111335us
I20260709 13:02:35.397348 10590 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:35.397094 10669 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:35.394773 10756 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:35.519055 10846 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:35.519863 10756 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:35.520439 10756 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.523067 10824 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:02:35.524333  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:35.525512 10630 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:35.525990 10630 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:35.526443 10630 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:35.526873 10843 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:02:35.528460 10765 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:35.422572 10833 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:35.530032  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:35.532328 10745 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:35.533107 10745 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.533641 10745 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:35.534991 10765 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:35.371112 10583 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.112s
I20260709 13:02:35.538552  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:35.511063 10590 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.422775 10728 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:35.536859  8719 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:35.543751 10765 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:35.541613 10590 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.562583 10839 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:35.563383 10569 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:35.484177 10755 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.584517  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:35.586167  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:35.550411 10630 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:35.578118 10569 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:35.558197 10483 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
W20260709 13:02:35.598927  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 72384.7us
I20260709 13:02:35.582789 10755 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.600805  8679 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:35.588663 10502 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:35.606330  8750 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:35.633028  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 121068us
I20260709 13:02:35.604446 10669 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:35.637642  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 391814us
W20260709 13:02:35.647670  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 270866us
I20260709 13:02:35.687705  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:35.689754  8749 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:35.692883  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 116570us
I20260709 13:02:35.687500  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:35.695771  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:35.697209  8748 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:02:35.700774  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 58092.5us
W20260709 13:02:35.701303  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 322979us
I20260709 13:02:35.695940  8794 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:35.703944  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.694303 10745 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:35.709197  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:35.710078  8677 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 1.
I20260709 13:02:35.709847 10738 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:35.705605  9352 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:35.653082 10583 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.727741  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:35.729981  8753 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:35.642305 10839 consensus_queue.cc:1048] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 13:02:35.730159  9355 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:35.712077 10591 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:35.690814 10737 consensus_queue.cc:1059] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:02:35.745061 10483 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:35.753129 10591 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:35.759289 10603 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:35.794777 10591 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.752192 10502 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:35.761850 10569 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:35.754959 10771 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:02:35.746002 10669 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
W20260709 13:02:35.800144 10855 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:02:35.839329  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 93099.9us
I20260709 13:02:35.784188 10802 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:35.816875  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:35.831569  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 69539.2us
I20260709 13:02:35.758016 10583 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:35.791819 10483 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:35.852783 10738 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:35.853688 10849 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:02:35.854002 10846 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:35.874277 10846 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.874897 10846 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:35.874979 10861 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:35.875500 10861 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:35.876622 10868 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:35.854482 10840 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.243s	user 0.001s	sys 0.001s
I20260709 13:02:35.877396 10868 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:35.877682 10483 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:35.880945 10868 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:35.859676  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 58855.1us
I20260709 13:02:35.859496  8679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
W20260709 13:02:35.864300  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 96480.4us
I20260709 13:02:35.847129 10771 raft_consensus.cc:3055] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:35.857250 10859 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:35.883936  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 185229us
W20260709 13:02:35.861831 10851 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:02:35.882814  8679 raft_consensus.cc:606] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:35.891819  8749 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:35.845448 10802 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:35.896555  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 166640us
I20260709 13:02:35.870846 10797 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:35.865818 10630 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:35.916543  8722 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:35.932497  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:35.936517  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 67868.5us
W20260709 13:02:35.943823  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.884220 10771 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:35.948464 10809 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:35.952545  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.904577 10802 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.976960 10797 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:35.979033 10797 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:35.979492 10797 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:35.980564 10797 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:35.982002 10797 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:35.989562  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:35.999961  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 55077.3us
W20260709 13:02:36.014772  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 88643.7us
W20260709 13:02:36.015270  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 104027us
I20260709 13:02:36.017486  8794 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:35.903414 10738 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
W20260709 13:02:36.024874  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 52299.9us
I20260709 13:02:36.022940  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1952) took 1212 ms. Trace:
I20260709 13:02:36.027860  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:36.028805  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:36.029657  8753 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:36.030915  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.027809  8751 rpcz_store.cc:276] 0709 13:02:34.820549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.308592 (+488043us) service_pool.cc:224] Handling call
0709 13:02:35.309132 (+   540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:35.337850 (+ 28718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:35.337883 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:35.338509 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 13:02:35.577817 (+239308us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:35.577862 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.577953 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:36.022603 (+444650us) raft_consensus.cc:1685] finished
0709 13:02:36.022643 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.022848 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14195,"raft.queue_time_us":28250,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":127,"thread_start_us":238786,"threads_started":1,"wal-append.queue_time_us":353157}
I20260709 13:02:36.040184  8753 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:35.917450 10737 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.097s
W20260709 13:02:36.047905  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 135953us
I20260709 13:02:35.975461 10630 raft_consensus.cc:2764] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:02:36.053328  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1951) took 1252 ms. Trace:
I20260709 13:02:36.051227  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1941) took 1245 ms. Trace:
I20260709 13:02:36.053952  8756 rpcz_store.cc:276] 0709 13:02:34.801438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.295252 (+493814us) service_pool.cc:224] Handling call
0709 13:02:35.295790 (+   538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:35.296455 (+   665us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:35.296498 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:35.297427 (+   929us) log.cc:844] Serialized 26 byte log entry
0709 13:02:35.583933 (+286506us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:35.583974 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.584049 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:36.053000 (+468951us) raft_consensus.cc:1685] finished
0709 13:02:36.053025 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.053245 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":285488,"threads_started":1,"wal-append.queue_time_us":400743}
W20260709 13:02:36.055930  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 150674us
I20260709 13:02:35.876686 10861 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:35.895826 10765 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.136s
I20260709 13:02:36.018169 10329 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:36.054518  8747 rpcz_store.cc:276] 0709 13:02:34.636872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.640697 (+  3825us) service_pool.cc:224] Handling call
0709 13:02:34.641628 (+   931us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:34.878776 (+237148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.878817 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:34.964759 (+ 85942us) log.cc:844] Serialized 26 byte log entry
0709 13:02:35.379231 (+414472us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:35.379270 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.379353 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:35.882198 (+502845us) raft_consensus.cc:1685] finished
0709 13:02:35.882223 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.882397 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":134381,"raft.queue_time_us":196467,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":273,"thread_start_us":275464,"threads_started":2,"wal-append.queue_time_us":601382}
I20260709 13:02:36.051534 10771 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:36.077524  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 195268us
I20260709 13:02:36.094285  9352 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:36.079569 10876 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
W20260709 13:02:36.142812 10864 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:02:36.183162 10876 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:36.183931 10876 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:36.185267 10876 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:36.092643 10881 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.109s
I20260709 13:02:36.187038 10824 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:36.196221  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.198041 10854 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.326s	user 0.000s	sys 0.002s
I20260709 13:02:36.137130 10859 raft_consensus.cc:697] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:36.209556 10861 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:36.254134  9386 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:36.271865  8719 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:02:36.273021  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 85342.7us
I20260709 13:02:36.281442 10872 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:36.283188  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 215608us
I20260709 13:02:36.349848 10876 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:36.351366  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.282683 10771 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:36.358042  8679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:36.359054  8679 raft_consensus.cc:606] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:36.359503  8679 raft_consensus.cc:614] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:36.211524 10738 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:36.168845 10630 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 13:02:36.487455 10630 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:36.488575 10483 raft_consensus.cc:3055] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:36.489104 10483 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:36.493326  8750 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:02:36.496508  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 68076.7us
W20260709 13:02:36.495901  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 424684us
I20260709 13:02:36.498814  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:36.516799  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.516280 10738 raft_consensus.cc:697] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:36.513046 10583 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:36.541848 10286 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:36.527961 10483 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:36.542704  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 44018.7us
I20260709 13:02:36.543391  8753 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:36.543874 10852 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:36.543629  8719 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:02:36.545017 10886 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
W20260709 13:02:36.547142  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.552831  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 461325us
W20260709 13:02:36.556945  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 529548us
I20260709 13:02:36.560226  8794 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:36.560590  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.571645  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:36.558140 10286 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:36.576361  8680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:36.562404 10852 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:36.577561 10879 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:36.535702 10737 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:02:36.578807  8680 raft_consensus.cc:606] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:36.591401  8680 raft_consensus.cc:614] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:36.586141 10852 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:36.587918 10738 consensus_queue.cc:237] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:36.586956 10888 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:02:36.574196 10881 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:36.593055  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:36.594250 10502 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:36.578887 10286 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:36.520488 10859 consensus_queue.cc:237] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:36.593364 10846 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.718s	user 0.024s	sys 0.000s
I20260709 13:02:36.581524 10900 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:36.620290 10531 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.188s	user 0.014s	sys 0.002s
I20260709 13:02:36.615551 10900 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:36.623281  8647 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:36.603487 10502 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:36.625885  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 58649.2us
I20260709 13:02:36.623624 10900 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:36.635677 10897 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:36.623347 10531 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:36.592864 10894 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:02:36.637480 10897 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:36.638245 10894 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:36.639221 10796 raft_consensus.cc:3055] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:36.639689 10796 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:36.640726 10796 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:36.597131 10583 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:02:36.616226 10846 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:36.642474 10796 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:36.643600 10833 consensus_meta.cc:287] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.113s	user 0.012s	sys 0.024s
I20260709 13:02:36.646474 10833 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:36.644997 10863 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:36.614235 10881 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:02:36.620625 10756 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.100s	user 0.021s	sys 0.007s
W20260709 13:02:36.675239  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.592622 10879 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:36.649988 10802 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.632s	user 0.011s	sys 0.011s
I20260709 13:02:36.629451 10502 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:36.658888 10888 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:36.716234 10888 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.716843 10888 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:36.717289 10833 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:36.720391  9352 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:36.721812 10871 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:36.722373 10871 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:36.722822 10871 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:36.703703  8749 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:36.650069 10900 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:36.658896 10846 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:36.680166 10795 consensus_meta.cc:287] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.308s	user 0.015s	sys 0.001s
I20260709 13:02:36.712014  8680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
W20260709 13:02:36.676162  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.011s	sys 0.018s
W20260709 13:02:36.710297  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 87163.1us
W20260709 13:02:36.738255  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.011s	sys 0.025s
I20260709 13:02:36.666188 10531 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:36.738919  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 736318us
W20260709 13:02:36.736166  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 142022us
W20260709 13:02:36.729568  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 133467us
W20260709 13:02:36.709226 10590 consensus_meta.cc:287] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.126s	user 0.013s	sys 0.003s
W20260709 13:02:36.727026 10797 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.745s	user 0.011s	sys 0.007s
W20260709 13:02:36.710106 10728 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.152s	user 0.014s	sys 0.002s
I20260709 13:02:36.763139 10881 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:36.764087  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:36.764331 10900 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:36.765537  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:36.766410  8678 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:36.676558 10756 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:36.766572  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:36.768103  8679 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:02:36.776782  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 141886us
W20260709 13:02:36.771118 10591 consensus_meta.cc:287] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.941s	user 0.006s	sys 0.007s
I20260709 13:02:36.791160  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:36.707935 10802 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:36.772965 10755 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.139s	user 0.029s	sys 0.026s
I20260709 13:02:36.797014  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:36.757336 10590 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:36.807067  8676 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:36.794997 10737 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:36.756644 10531 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:36.808822 10590 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:36.810475 10531 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:36.800223 10900 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:36.811107 10802 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:36.800830 10755 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:36.812940  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:36.813524 10755 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:36.814350 10901 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.135s	user 0.000s	sys 0.002s
I20260709 13:02:36.815390  8680 raft_consensus.cc:606] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:36.815568  8722 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:36.815850  8680 raft_consensus.cc:614] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:36.815907 10904 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:36.816468 10904 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:36.816901 10904 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:36.819408  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:36.819625 10858 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:36.821195  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:36.823524  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 47992.8us
I20260709 13:02:36.785458 10820 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:36.825876  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:36.801474  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:36.833559  8675 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:02:36.834357  8645 connection.cc:256] RPC call timeout handler was delayed by -1.8842846289999784s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1529s for a response
W20260709 13:02:36.834823  8645 connection.cc:256] RPC call timeout handler was delayed by -1.8374940919999858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24280000000000002s for a response
I20260709 13:02:36.762169 10728 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:36.837111 10905 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:02:36.840299 10907 log.cc:927] Time spent T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:02:36.771073 10852 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:36.841022  8678 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
W20260709 13:02:36.859480  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.860117  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.842070 10870 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:02:36.771627 10889 consensus_queue.cc:1059] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.244s
W20260709 13:02:36.842402 10861 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.625s	user 0.020s	sys 0.008s
I20260709 13:02:36.849543  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:36.880116  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 83411.6us
I20260709 13:02:36.870844  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:36.871915  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:36.881637  8749 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:36.882428  8645 connection.cc:256] RPC call timeout handler was delayed by -1.4495846290000145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20120000000000002s for a response
I20260709 13:02:36.796554 10591 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:36.882951  8645 connection.cc:256] RPC call timeout handler was delayed by -1.3244846289999828s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
I20260709 13:02:36.884495  8672 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:36.879592 10846 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:36.886498  8645 connection.cc:256] RPC call timeout handler was delayed by -1.3183713029999922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19130000000000003s for a response
I20260709 13:02:36.891878 10755 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:36.877805 10833 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:36.902837 10898 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:36.903438 10898 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.903813 10898 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:36.905975 10846 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:36.842507 10502 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:36.901144  8676 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
W20260709 13:02:36.909004  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 93572.7us
I20260709 13:02:36.845057 10820 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:36.924299  8749 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:36.907511 10286 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:36.925928  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:36.927081  8826 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:36.928351 10286 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:36.929210  8722 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:36.932199  8722 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:36.932299 10797 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.891759 10728 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:36.934973  9355 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:36.909198 10569 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:36.947762  8827 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:36.892091 10809 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:02:36.906896 10591 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:36.909000 10502 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:36.908628 10846 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:36.968924  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:36.933069 10797 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:36.973003  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:36.949045 10824 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:36.977349 10756 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:36.942852 10881 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:02:36.977972  8722 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:36.948741 10903 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:36.978314  8749 raft_consensus.cc:2376] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2. Re-sending same reply.
I20260709 13:02:36.906594 10871 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:36.951383 10833 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:36.999629  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:36.948344 10728 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.003487  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 125083us
W20260709 13:02:37.002648  8645 leader_election.cc:336] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.529s (SENT)
I20260709 13:02:37.005039  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.003693  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:36.941332 10755 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:36.989078 10824 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.019565  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:36.964968 10900 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:02:37.023576  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:37.025697  8752 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:36.999063 10846 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:02:37.026190 10737 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:37.026538 10728 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.027670  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.028499  8747 raft_consensus.cc:2368] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4.
I20260709 13:02:37.028869 10728 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:37.028486 10856 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.978s	user 0.009s	sys 0.012s
W20260709 13:02:37.030140  8645 leader_election.cc:336] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.707s (SENT)
I20260709 13:02:37.030687  8645 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:37.031687  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 67860.4us
I20260709 13:02:37.030802 10872 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:37.032318 10872 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:37.033407 10872 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.034768 10876 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:37.035360 10872 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:37.035781 10876 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:37.003716 10599 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.036741  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 58862.7us
I20260709 13:02:37.015461 10286 raft_consensus.cc:2764] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
W20260709 13:02:37.038213  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 78934.8us
I20260709 13:02:37.021298 10824 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.068418  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.037010 10879 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:37.075032  8752 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:37.076013  8747 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:37.079756  8749 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:37.082976  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.048666 10599 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:37.107163 10599 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:37.049809 10738 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:02:37.081693 10888 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:37.116396  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 57341.6us
I20260709 13:02:37.120882  8749 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:37.122200  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 90467.8us
I20260709 13:02:37.126096  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:37.129206  8645 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.913s (SENT)
I20260709 13:02:37.129757  8645 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.127400  8747 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 1.
I20260709 13:02:37.151404  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:37.153746  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:02:37.155802  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.158574  8793 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:37.160821 10914 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:02:37.161514 10917 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:37.162426 10917 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:37.164911 10756 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:37.165983 10756 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:37.166163  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.161448  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:37.076555 10286 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
W20260709 13:02:37.211951  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 90896us
I20260709 13:02:37.208997  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.184688  8756 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:37.212258  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 46199.6us
I20260709 13:02:37.216048  8752 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:37.216437  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.210211  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.123848 10876 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:02:37.220209  8747 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:37.220587  8755 raft_consensus.cc:2368] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
I20260709 13:02:37.166568 10756 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:37.223234  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 100238us
I20260709 13:02:37.114710 10919 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.132s
I20260709 13:02:37.199314  9384 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
W20260709 13:02:37.229210  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 77488.2us
I20260709 13:02:37.208999 10889 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:37.183104 10888 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:37.189158 10591 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:37.216790 10880 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:37.232396  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.215035 10820 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:37.233536  8645 leader_election.cc:336] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 2.076s (SENT)
I20260709 13:02:37.233451  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:37.235965  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.237340 10911 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.350s	user 0.000s	sys 0.002s
I20260709 13:02:37.242347  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:02:37.241191  8722 connection.cc:256] RPC call timeout handler was delayed by -1.7624405640000305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
I20260709 13:02:37.244000  9380 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:37.244478  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.244746  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:37.245253  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.252334  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:37.253230 10873 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.715s	user 0.003s	sys 0.002s
I20260709 13:02:37.255082  8748 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:37.255340  8751 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:37.255142  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.256108 10910 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:37.226856 10797 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:37.256377  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:02:37.265677  8722 connection.cc:256] RPC call timeout handler was delayed by -1.5364714230000232s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 13:02:37.267107  8749 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:37.272506  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.298077  8756 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:37.228102 10756 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:37.275244  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.279798  8750 raft_consensus.cc:2410] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:02:37.279798  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.286911  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.246147 10898 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:37.287931  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54414.4us
I20260709 13:02:37.263947 10591 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:37.307252 10591 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:37.311590 10888 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:37.312465 10852 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:37.312856 10889 raft_consensus.cc:3055] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:02:37.313275  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 79965.6us
I20260709 13:02:37.314358  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:37.314463 10888 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:37.309307 10630 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:37.315761 10630 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.316002 10888 consensus_queue.cc:1059] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:37.317055 10858 raft_consensus.cc:3055] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:37.317451 10858 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:37.318297 10858 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:37.319021 10910 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:37.319715 10858 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:37.296803  8754 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:37.300671  8752 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:02:37.301240  8722 connection.cc:256] RPC call timeout handler was delayed by -1.4509714230000554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
I20260709 13:02:37.292021 10882 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:37.313279 10889 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:37.316424 10904 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:37.325121  8645 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.387881 10897 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:37.387974  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 62956.8us
I20260709 13:02:37.388895  8645 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:37.388788 10756 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.390568 10863 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:37.391023 10863 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.391242 10756 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:37.391408 10863 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:37.325701  8722 connection.cc:256] RPC call timeout handler was delayed by -1.4275714230000176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1653s for a response
W20260709 13:02:37.392524  8722 connection.cc:256] RPC call timeout handler was delayed by -1.3816714230000571s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
W20260709 13:02:37.392853  8722 connection.cc:256] RPC call timeout handler was delayed by -1.1707218480000279s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
I20260709 13:02:37.392030  8796 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:37.327651  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.386596  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:37.395009  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:37.395542 10820 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:37.397204 10820 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.322340 10910 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:37.342393 10904 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:37.343552 10889 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.397997 10820 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:37.398764 10889 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:37.346886 10880 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:37.347060  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.395556  8645 leader_election.cc:400] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
I20260709 13:02:37.359122 10591 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.401549  9388 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:37.403048 10909 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.647s	user 0.005s	sys 0.000s
I20260709 13:02:37.403358 10591 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:37.360421  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:37.362653  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:37.363301  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.365609 10871 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:37.407229  9381 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:37.366539 10861 raft_consensus.cc:3055] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:37.407843 10861 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:37.408792  8824 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:37.408753 10861 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.410477 10861 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:37.367373 10583 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:37.411085 10583 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:37.412009 10583 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.413291 10583 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:37.415198  8722 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.532s (SENT)
I20260709 13:02:37.415311 10880 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:37.415844  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.416678 10875 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:37.417233 10875 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:37.291829 10755 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:37.418120 10286 consensus_queue.cc:1059] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:37.418422 10875 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:37.420612 10904 raft_consensus.cc:3055] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:37.377151  9384 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:37.426998 10904 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:37.428378 10894 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:37.429808  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:37.381122 10630 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:37.382305 10882 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.353837 10824 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:37.385239 10852 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:37.434712 10882 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.294575  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.395632  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:37.436359 10755 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:37.436501  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 142057us
I20260709 13:02:37.437212 10882 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:37.438150  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.337857  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.350921 10569 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:37.439193 10824 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:37.439489 10569 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.440815 10904 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:37.441962 10904 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.442139 10569 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:37.444356  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 305) took 1847 ms. Trace:
I20260709 13:02:37.444767  8672 rpcz_store.cc:276] 0709 13:02:35.597197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.606822 (+  9625us) service_pool.cc:224] Handling call
0709 13:02:35.698935 (+ 92113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.891666 (+1192731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.444096 (+552430us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54001f9354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:37.444290 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2718348032}
W20260709 13:02:37.451027  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.420792 10875 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.395008  9389 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:37.400358  8645 leader_election.cc:403] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:37.410377  8827 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:37.416749 10591 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:37.375639  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:37.367226  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 52999.6us
I20260709 13:02:37.374541 10898 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:37.425880  9355 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:37.435228  9382 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:37.396860  8750 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:37.293964  8755 raft_consensus.cc:2393] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:02:37.457309  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 64743.9us
I20260709 13:02:37.461381  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:37.462287  9355 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:37.465454  8793 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:37.463588 10858 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:02:37.457690 10755 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:37.481845  8645 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:37.487259  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:37.470305  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 49114.8us
I20260709 13:02:37.463550 10824 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:37.466593 10871 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:37.498824 10920 consensus_queue.cc:1059] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:02:37.479712 10756 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:37.502741 10728 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:37.501591  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:37.495364 10919 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:37.512531 10919 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.512457  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.513121 10919 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:37.497040 10824 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.509614  8645 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:37.514125  8722 leader_election.cc:336] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.627s (SENT)
I20260709 13:02:37.514920  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.490392  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.513731 10920 raft_consensus.cc:3055] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:37.511096 10897 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:37.527552 10920 raft_consensus.cc:740] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:37.527797 10591 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:37.486227 10876 consensus_queue.cc:1059] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:02:37.530493 10920 consensus_queue.cc:260] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.533408 10920 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:37.503890 10871 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:37.531314  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.519239 10728 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.519663 10569 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:37.521963 10824 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:37.540962 10876 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:37.541517  8647 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:37.542682  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.543459  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:37.539122 10728 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.542040 10756 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:37.545938  8827 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:37.546345  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.537326 10871 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:37.569700  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:37.569883  8579 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43462 (request call id 265) took 12183 ms (client timeout 15000 ms). Trace:
I20260709 13:02:37.570592  8821 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:37.571559  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.571741 10900 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:37.572214 10900 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:37.545975  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.549634  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.573447  8823 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:37.573202 10900 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:37.574424  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.575271 10859 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:37.575736 10859 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.575827  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.576174 10859 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.577077  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:37.552380 10917 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:37.578869 10888 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:37.579339 10888 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:37.579730 10888 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:37.568069  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:37.580451  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.581053  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.581620  8827 raft_consensus.cc:2376] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3. Re-sending same reply.
I20260709 13:02:37.582801  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:37.555020  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.556399 10483 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.585209  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:37.585541  8829 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:37.583576  8647 leader_election.cc:400] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4.
I20260709 13:02:37.586901  8676 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:37.557444  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 60802.5us
I20260709 13:02:37.578269  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.567998  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.583227  8675 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:37.582799 10876 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.582552  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:37.586979  8647 leader_election.cc:403] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:02:37.588482  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 74433.3us
I20260709 13:02:37.535422  8825 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.589519  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.564917  8828 raft_consensus.cc:2410] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:37.539846 10861 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:37.553372 10919 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.596182 10755 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4.
I20260709 13:02:37.596850  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:37.597045  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.600834  8828 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:37.574512 10900 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:37.603613  8722 leader_election.cc:336] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.653s (SENT)
I20260709 13:02:37.608146  8722 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.609401 10889 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:37.610086  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:37.598166 10876 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.604185  8830 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:37.612181  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:37.612960  8675 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.613448 10876 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:37.614197  8678 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
W20260709 13:02:37.615419  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.616441  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:37.621029  8828 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:37.624081  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.617849  8680 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:37.641985  8752 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:37.643229  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.643888 10797 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:37.646080  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 56602.4us
I20260709 13:02:37.601879 10861 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:37.597563 10728 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:37.655511  8756 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:37.594233 10910 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:02:37.659674  8647 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:37.660019  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.594558 10795 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:02:37.673162  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:37.570349  8579 rpcz_store.cc:269] 0709 13:02:25.386531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.386774 (+   243us) service_pool.cc:224] Handling call
0709 13:02:25.744381 (+357607us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54002d7dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2463090 00007f32e2501f3a 00007f32e82b81cc 00007f32e83b637e 00007f32e68772a4 00007f32e6877232 00007f32e6877161 00007f32e68770dc 00007f32e6876392 00007f32e083448c 00007f32e08338c6
0709 13:02:37.569761 (+11825380us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:36.143033 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:36.199245 (+ 56212us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:36.199329 (+    84us) write_op.cc:435] Acquired schema lock
0709 13:02:36.199359 (+    30us) tablet.cc:662] Decoding operations
0709 13:02:36.200821 (+  1462us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:36.200908 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:36.200947 (+    39us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:02:36.550877 (+349930us) tablet.cc:701] Row locks acquired
0709 13:02:36.550921 (+    44us) write_op.cc:260] PREPARE: finished
0709 13:02:36.551126 (+   205us) write_op.cc:270] Start()
0709 13:02:36.551292 (+   166us) write_op.cc:276] Timestamp: P: 1783602156551044 usec, L: 0
0709 13:02:36.551315 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:02:36.577869 (+ 26554us) log.cc:844] Serialized 4374 byte log entry
0709 13:02:37.111963 (+534094us) op_driver.cc:464] REPLICATION: finished
0709 13:02:37.246741 (+134778us) write_op.cc:301] APPLY: starting
0709 13:02:37.247010 (+   269us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:37.247311 (+   301us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:37.247462 (+   151us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:37.286566 (+ 39104us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:37.286934 (+   368us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:02:37.287002 (+    68us) write_op.cc:312] APPLY: finished
0709 13:02:37.293436 (+  6434us) log.cc:844] Serialized 83 byte log entry
0709 13:02:37.307024 (+ 13588us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:37.307242 (+   218us) write_op.cc:454] Released schema lock
0709 13:02:37.307343 (+   101us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:37.307923 (+   580us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:37.307968 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21242,"reported_tablets":22,"spinlock_wait_cycles":249818368,"child_traces":[["op",{"apply.queue_time_us":118256,"apply.run_cpu_time_us":5612,"apply.run_wall_time_us":77251,"num_ops":11,"prepare.queue_time_us":279052,"prepare.run_cpu_time_us":18917,"prepare.run_wall_time_us":678417,"raft.queue_time_us":200415,"raft.run_cpu_time_us":45431,"raft.run_wall_time_us":107660,"replication_time_us":560419,"thread_start_us":373909,"threads_started":4,"wal-append.queue_time_us":165631}]]}
I20260709 13:02:37.662603  8749 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:37.644901 10286 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:37.664829  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.681682  8750 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:02:37.667141  8722 leader_election.cc:336] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.704s (SENT)
I20260709 13:02:37.683427  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.644567 10797 raft_consensus.cc:697] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:37.682725  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.680150  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.686234  8754 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:37.656823 10755 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:37.683743  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.598451 10917 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:02:37.660243 10861 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:37.688848  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:37.692600  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.662485  8675 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:37.704656  8828 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
W20260709 13:02:37.699725  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.698917  8751 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:37.713534 10756 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.713997 10797 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:37.714293 10756 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.715111 10286 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:37.715914  8672 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:37.716924 10756 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:37.706305  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:37.717978  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:37.714390  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.719271  8749 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:02:37.720038  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 52943.7us
W20260709 13:02:37.720650  8722 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.736s (SENT) [suppressed 30 similar messages]
I20260709 13:02:37.679612 10920 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:37.736191  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:37.763304 10919 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:37.762844  8575 catalog_manager.cc:5697] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:37.764187  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:37.766975  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:02:37.775056 10937 log.cc:927] Time spent T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:02:37.780660  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:37.788043  8829 raft_consensus.cc:2410] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:02:37.788972  8722 leader_election.cc:336] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.865s (SENT)
I20260709 13:02:37.789613  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:37.798307  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.803179 10897 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:37.803164  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:37.804258 10897 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:37.804519  8673 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:37.805622 10897 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:37.808140  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:37.809913  8828 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:37.811233 10852 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:37.811676 10852 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:37.813700  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:37.815071  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:37.815128  8645 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:37.819378  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.821017  8754 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:02:37.766279  8575 catalog_manager.cc:5697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 2 to 4, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 4 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:37.822817  8575 catalog_manager.cc:5697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 2 to 4, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 4 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:37.825531  8575 catalog_manager.cc:5697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:37.829761  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.829443  8575 catalog_manager.cc:5697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:37.830579  8823 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:37.828269  8825 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:37.832067  8749 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:37.833479  8749 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:37.835011  8750 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:37.840234  8647 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:37.842022  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.845041  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:37.846101  8672 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:37.846517  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:37.847597 10852 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:37.850070  8823 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:37.853638 10863 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:37.856876 10936 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:02:37.858148 10820 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:37.808209 10934 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:02:37.867898  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:37.868301 10935 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:02:37.868848  8756 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:37.862241  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:37.861682  8673 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:37.870713  9382 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:37.874112  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.875309  8826 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:37.876995  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:37.878800  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:37.880339  8828 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:02:37.893404  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:37.904899  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 64827.1us
I20260709 13:02:37.906081  8647 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:37.908773  8575 catalog_manager.cc:5697] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 2 to 4, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 4 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:37.915232 10920 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:37.942853  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:37.967957  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:37.969051  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 180138us
I20260709 13:02:38.079074  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:38.215963  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.393541  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.713620  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 807664us
W20260709 13:02:38.069070  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.719552  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.720829  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 905785us
W20260709 13:02:38.071852  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.723134  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.742987  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:38.744031  8828 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:38.725538  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 313) took 1261 ms. Trace:
W20260709 13:02:38.549945  8747 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.003s	sys 0.006s
I20260709 13:02:38.162986  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:38.438146  8830 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.821s	user 0.000s	sys 0.027s
I20260709 13:02:38.245440  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:38.899915  8754 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:38.909925  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:38.877558  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:02:38.841517  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 133116us
W20260709 13:02:38.929086  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.875233  8575 catalog_manager.cc:5697] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:38.812590  8752 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.035s	user 0.020s	sys 0.027s
W20260709 13:02:38.978243  8752 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.201s	user 0.021s	sys 0.037s
I20260709 13:02:38.940716  8750 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:38.972083  8575 catalog_manager.cc:5697] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:38.393071 10912 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:38.703825 10889 consensus_meta.cc:287] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.094s	user 0.009s	sys 0.002s
I20260709 13:02:38.951308  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1385) took 1353 ms. Trace:
I20260709 13:02:38.942695  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1387) took 1297 ms. Trace:
W20260709 13:02:38.692893 10824 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.126s	user 0.012s	sys 0.001s
W20260709 13:02:38.905772 10861 consensus_meta.cc:287] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.171s	user 0.010s	sys 0.000s
W20260709 13:02:38.843180 10888 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.263s	user 0.014s	sys 0.001s
I20260709 13:02:39.002650 10861 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:38.991285 10912 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.003556 10888 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.005584 10861 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:39.005852 10888 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:38.903169 10728 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:38.989114  8676 rpcz_store.cc:276] 0709 13:02:37.598224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.598547 (+   323us) service_pool.cc:224] Handling call
0709 13:02:37.603369 (+  4822us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.606419 (+  3050us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.606458 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:37.642098 (+ 35640us) log.cc:844] Serialized 26 byte log entry
0709 13:02:38.186742 (+544644us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:38.186803 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.186919 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:38.950923 (+764004us) raft_consensus.cc:1685] finished
0709 13:02:38.950957 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.951205 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8683,"raft.queue_time_us":3072,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":3268,"thread_start_us":468521,"threads_started":1,"wal-append.queue_time_us":1231710}
I20260709 13:02:39.006450 10888 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.874336  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:39.007436 10888 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.007835  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:39.008131 10889 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:39.008625  8676 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:39.009181  8679 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:38.813536 10875 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.356s	user 0.010s	sys 0.005s
I20260709 13:02:38.747184  8824 rpcz_store.cc:276] 0709 13:02:37.464160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.464359 (+   199us) service_pool.cc:224] Handling call
0709 13:02:37.464900 (+   541us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:37.480039 (+ 15139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.480091 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:37.494582 (+ 14491us) log.cc:844] Serialized 52 byte log entry
0709 13:02:37.535556 (+ 40974us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:37.535615 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.535736 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.867419 (+331683us) raft_consensus.cc:1685] finished
0709 13:02:37.867453 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.725439 (+857986us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22596,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":5965,"thread_start_us":23274,"threads_started":1,"wal-append.queue_time_us":90491}
W20260709 13:02:38.430678 10569 consensus_meta.cc:287] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.866s	user 0.010s	sys 0.002s
I20260709 13:02:38.938191  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:38.985623  8749 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:39.016166  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1389) took 1328 ms. Trace:
I20260709 13:02:39.016572  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1472) took 1049 ms. Trace:
I20260709 13:02:39.017055  8827 rpcz_store.cc:276] 0709 13:02:37.975873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.811301 (+835428us) service_pool.cc:224] Handling call
0709 13:02:39.016503 (+205202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:39.016700  8678 rpcz_store.cc:276] 0709 13:02:37.687380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.693408 (+  6028us) service_pool.cc:224] Handling call
0709 13:02:37.707854 (+ 14446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.758441 (+ 50587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.758491 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:37.773900 (+ 15409us) log.cc:844] Serialized 26 byte log entry
0709 13:02:38.224562 (+450662us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:38.224622 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.224726 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:39.015876 (+791150us) raft_consensus.cc:1685] finished
0709 13:02:39.015904 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.016078 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1886,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":7854,"thread_start_us":450051,"threads_started":1,"wal-append.queue_time_us":1232689}
I20260709 13:02:39.010838 10912 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:39.019867  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:39.020061 10888 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:39.021036 10920 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.045s	user 0.008s	sys 0.003s
I20260709 13:02:39.021287  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:39.006440 10861 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:02:39.023396  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:38.939487  8751 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:39.025355  8678 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:39.025455  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 272) took 1032 ms. Trace:
I20260709 13:02:39.025923  8751 rpcz_store.cc:276] 0709 13:02:37.993384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.772598 (+779214us) service_pool.cc:224] Handling call
0709 13:02:38.938972 (+166374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.025125 (+ 86153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.025385 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:39.026496  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:39.026949  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 318) took 1190 ms. Trace:
W20260709 13:02:38.911293 10900 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.305s	user 0.010s	sys 0.002s
I20260709 13:02:39.027400  8825 rpcz_store.cc:276] 0709 13:02:37.836190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.966110 (+129920us) service_pool.cc:224] Handling call
0709 13:02:38.822552 (+856442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.952889 (+130337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:38.968682 (+ 15793us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:38.981009 (+ 12327us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:38.991893 (+ 10884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.009035 (+ 17142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.026874 (+ 17839us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5376}
I20260709 13:02:37.915493 10820 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:39.028806  8753 raft_consensus.cc:2410] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:39.029196 10820 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.027909  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:39.029855 10820 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.030824  8678 raft_consensus.cc:2410] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:39.032414  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:39.032996 10820 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:38.907987 10859 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.331s	user 0.012s	sys 0.001s
I20260709 13:02:39.033814  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.033756 10859 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.034548  8678 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:39.035102  8821 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:39.035593  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:39.036209 10859 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:39.036939  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:38.746860 10286 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.038164  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:39.038404  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1265) took 1480 ms. Trace:
I20260709 13:02:39.038735 10286 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.039358  8821 rpcz_store.cc:276] 0709 13:02:37.558013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.611401 (+ 53388us) service_pool.cc:224] Handling call
0709 13:02:37.611808 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.035614 (+1423806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.038104 (+  2490us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54003de5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:39.038334 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3272738048}
I20260709 13:02:39.041184  8821 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:39.041393 10286 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:37.976027 10583 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:39.041994 10286 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:38.682197 10852 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.042814 10286 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.046082 10286 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:38.810153  8673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.001s	sys 0.011s
W20260709 13:02:39.046842  8673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.053s	user 0.003s	sys 0.013s
W20260709 13:02:38.189229 10871 consensus_meta.cc:287] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.620s	user 0.009s	sys 0.003s
I20260709 13:02:39.047806 10871 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:38.489311 10483 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.904s	user 0.012s	sys 0.000s
I20260709 13:02:38.991251  8680 rpcz_store.cc:276] 0709 13:02:37.644961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.645140 (+   179us) service_pool.cc:224] Handling call
0709 13:02:37.666439 (+ 21299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.678051 (+ 11612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.678086 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:37.686832 (+  8746us) log.cc:844] Serialized 26 byte log entry
0709 13:02:38.168866 (+482034us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:38.168935 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.169052 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:38.942290 (+773238us) raft_consensus.cc:1685] finished
0709 13:02:38.942325 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.942580 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11220,"raft.queue_time_us":556,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":18870,"thread_start_us":410108,"threads_started":1,"wal-append.queue_time_us":1186181}
I20260709 13:02:39.051290  8822 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:39.051489  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.041976  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1470) took 1167 ms. Trace:
I20260709 13:02:39.052476  8680 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:39.052965  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:39.053617  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:38.991521 10824 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.007722  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1460) took 1137 ms. Trace:
W20260709 13:02:38.875291  9353 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.000s	sys 0.012s
W20260709 13:02:38.984740  8830 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.368s	user 0.007s	sys 0.038s
I20260709 13:02:39.070806  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1266) took 1512 ms. Trace:
I20260709 13:02:39.021795 10920 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:39.077577 10920 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.068753  8823 rpcz_store.cc:276] 0709 13:02:37.870363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.870507 (+   144us) service_pool.cc:224] Handling call
0709 13:02:39.007644 (+1137137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:39.078266 10920 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.078919 10728 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.079486 10728 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.080720 10920 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:39.081605 10728 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:39.023710 10888 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:39.029318  8825 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:38.996721  8674 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:38.663081 10917 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.925s	user 0.007s	sys 0.038s
W20260709 13:02:38.701229 10858 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.197s	user 0.011s	sys 0.000s
I20260709 13:02:39.028710  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:39.084978  8754 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:02:38.985862  8747 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.003s	sys 0.011s
I20260709 13:02:37.994742 10591 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:38.047626 10876 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:38.047070 10897 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:39.053949 10871 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:39.066381  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:39.067695  8680 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:02:39.072324  9353 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.017s	user 0.002s	sys 0.012s
I20260709 13:02:39.068598 10583 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.052552  8821 rpcz_store.cc:276] 0709 13:02:37.874110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.040469 (+1166359us) service_pool.cc:224] Handling call
0709 13:02:39.040895 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.041643 (+   748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.041908 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:39.075711 10889 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.111478 10889 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.079754  8823 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:02:39.112864  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1471) took 1238 ms. Trace:
I20260709 13:02:39.113276  8823 rpcz_store.cc:276] 0709 13:02:37.874470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.079007 (+1204537us) service_pool.cc:224] Handling call
0709 13:02:39.079456 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.112506 (+ 33050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.112793 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:39.024538  8824 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:39.113812 10863 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:39.114006 10889 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:39.077656 10824 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:39.071182  8830 rpcz_store.cc:276] 0709 13:02:37.558283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.612630 (+ 54347us) service_pool.cc:224] Handling call
0709 13:02:37.613751 (+  1121us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.615344 (+  1593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.615375 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:37.616479 (+  1104us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.070394 (+1453915us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:39.070440 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.070514 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:39.070553 (+    39us) raft_consensus.cc:1685] finished
0709 13:02:39.070564 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.070744 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":539,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":97,"thread_start_us":1453457,"threads_started":1,"wal-append.queue_time_us":1291156}
I20260709 13:02:39.082242 10888 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:39.083519  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1469) took 1209 ms. Trace:
I20260709 13:02:39.086764  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1979) took 1415 ms. Trace:
I20260709 13:02:39.099753  8828 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
W20260709 13:02:39.102012  9382 consensus_meta.cc:287] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.231s	user 0.006s	sys 0.006s
I20260709 13:02:39.101840  8750 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:39.104974  8827 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:39.110363 10583 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.111850  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 319) took 1204 ms. Trace:
I20260709 13:02:39.114681  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1466) took 1241 ms. Trace:
I20260709 13:02:39.145533  8824 rpcz_store.cc:276] 0709 13:02:37.872751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.022542 (+1149791us) service_pool.cc:224] Handling call
0709 13:02:39.024205 (+  1663us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.114352 (+ 90147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.114583 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:39.042124  8678 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:02:39.148643  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 309) took 1784 ms (client timeout 1897 ms). Trace:
W20260709 13:02:39.084146 10917 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.351s	user 0.011s	sys 0.044s
I20260709 13:02:39.066097 10569 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:39.142956 10286 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.128319  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1468) took 1254 ms. Trace:
I20260709 13:02:39.130118  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:39.159530 10949 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:39.159895 10286 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.160338  8749 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:02:39.121760  8825 rpcz_store.cc:276] 0709 13:02:37.873763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.028729 (+1154966us) service_pool.cc:224] Handling call
0709 13:02:39.029078 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.083131 (+ 54053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.083446 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:39.160977 10949 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:39.161335  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:39.161728  8828 rpcz_store.cc:276] 0709 13:02:37.873397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.954959 (+ 81562us) service_pool.cc:224] Handling call
0709 13:02:39.128205 (+1173246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:39.162290  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:39.162863 10910 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:39.143357  9382 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:39.145299 10824 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:39.164587  9382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 568) took 1303 ms. Trace:
I20260709 13:02:39.165030 10910 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:39.145259  8821 rpcz_store.cc:276] 0709 13:02:37.907850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.110758 (+1202908us) service_pool.cc:224] Handling call
0709 13:02:39.111160 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.111396 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.111425 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.111485 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:39.111528 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.111604 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.111781 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:39.162290 10286 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:39.167371 10943 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:39.167899 10943 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:39.157090  8755 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:39.132189 10888 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.132773  8680 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:39.175967  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:39.176966  9389 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 2: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:39.173875  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:39.150354  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:39.153445  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1976) took 1505 ms. Trace:
I20260709 13:02:39.103673 10858 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 2. Current term is 3.
I20260709 13:02:39.221771  8752 rpcz_store.cc:276] 0709 13:02:37.648444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.650714 (+  2270us) service_pool.cc:224] Handling call
0709 13:02:37.651057 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.674480 (+ 23423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.674518 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:37.697022 (+ 22504us) log.cc:844] Serialized 26 byte log entry
0709 13:02:38.978909 (+1281887us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:38.978953 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.979044 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:39.153181 (+174137us) raft_consensus.cc:1685] finished
0709 13:02:39.153205 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.153377 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":273,"raft.queue_time_us":9982,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":22810,"thread_start_us":1201249,"threads_started":1,"wal-append.queue_time_us":1354388}
W20260709 13:02:39.149348  8822 rpcz_store.cc:269] 0709 13:02:37.372193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.375184 (+  2991us) service_pool.cc:224] Handling call
0709 13:02:39.148391 (+1773207us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b540016b554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:39.148553 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3661266816}
I20260709 13:02:39.172394  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1388) took 1503 ms. Trace:
I20260709 13:02:39.176268 10943 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.160995  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:39.165057 10824 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.208511  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 270) took 1236 ms. Trace:
I20260709 13:02:39.244290  8755 rpcz_store.cc:276] 0709 13:02:37.972173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.078692 (+106519us) service_pool.cc:224] Handling call
0709 13:02:39.208178 (+1129486us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b540018ced4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:39.208411 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":986532096}
I20260709 13:02:39.123960  8747 rpcz_store.cc:276] 0709 13:02:37.671438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.671953 (+   515us) service_pool.cc:224] Handling call
0709 13:02:37.698823 (+ 26870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.699060 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:37.967488 (+268428us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.086160 (+1118672us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.086224 (+    64us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:39.086288 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.086382 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.086701 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2168,"prepare.queue_time_us":17276,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":409,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":9503,"thread_start_us":1371414,"threads_started":2,"wal-append.queue_time_us":932619}
I20260709 13:02:39.165040  9382 rpcz_store.cc:276] 0709 13:02:37.861603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.861828 (+   225us) service_pool.cc:224] Handling call
0709 13:02:39.164485 (+1302657us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:39.213073  8674 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:39.251466  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1396) took 1082 ms. Trace:
I20260709 13:02:39.252005  8674 rpcz_store.cc:276] 0709 13:02:38.168920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.244820 (+ 75900us) service_pool.cc:224] Handling call
0709 13:02:39.251373 (+1006553us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:39.258219  9381 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:39.260217 10796 raft_consensus.cc:3055] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:39.260848 10796 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:02:39.267627 10954 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:02:39.271085  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1393) took 1366 ms. Trace:
I20260709 13:02:39.271683  8673 rpcz_store.cc:276] 0709 13:02:37.904985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.905181 (+   196us) service_pool.cc:224] Handling call
0709 13:02:37.905615 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.992485 (+ 86870us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.992540 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:37.993381 (+   841us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.047323 (+1053942us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:39.047372 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.047448 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:39.270736 (+223288us) raft_consensus.cc:1685] finished
0709 13:02:39.270775 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.270995 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31019,"raft.queue_time_us":82731,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":21708,"thread_start_us":1053352,"threads_started":1,"wal-append.queue_time_us":1129534}
I20260709 13:02:39.278980  9388 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:39.281065 10824 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:39.194796  8753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1562) took 1545 ms (client timeout 2012 ms). Trace:
W20260709 13:02:39.282064  8753 rpcz_store.cc:269] 0709 13:02:37.649850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.649990 (+   140us) service_pool.cc:224] Handling call
0709 13:02:39.185572 (+1535582us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54002fd5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:39.194682 (+  9110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3115292032}
I20260709 13:02:39.193882  8829 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:39.284507  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.285090  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1465) took 1412 ms. Trace:
I20260709 13:02:39.184696  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:39.220903  8756 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
W20260709 13:02:39.224612  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 74346.9us
I20260709 13:02:39.289470  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:39.225853  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.211655  9353 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:39.228448  8679 rpcz_store.cc:276] 0709 13:02:37.668596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.668745 (+   149us) service_pool.cc:224] Handling call
0709 13:02:37.685816 (+ 17071us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.009766 (+1323950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.172020 (+162254us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54001f9854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:39.172303 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3042993408}
I20260709 13:02:39.243955 10876 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:39.291533  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 79988.4us
I20260709 13:02:39.285712  8829 rpcz_store.cc:276] 0709 13:02:37.872413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.992344 (+119931us) service_pool.cc:224] Handling call
0709 13:02:39.183674 (+1191330us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:39.193541 (+  9867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.284695 (+ 91154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.285008 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55340288}
I20260709 13:02:39.296970  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:39.252957 10920 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:39.294895  8826 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1467) took 1421 ms (client timeout 1501 ms). Trace:
I20260709 13:02:39.300819  8676 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
I20260709 13:02:39.298998 10863 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:39.304661  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:39.300459  8826 rpcz_store.cc:269] 0709 13:02:37.873081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.931971 (+ 58890us) service_pool.cc:224] Handling call
0709 13:02:39.184525 (+1252554us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:39.294453 (+109928us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55460096}
I20260709 13:02:39.306051  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:39.309932  8677 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:39.270458 10583 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:39.308405 10824 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:39.332683 10824 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.333222 10824 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:39.296762 10943 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:39.303524 10889 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:02:39.309846 10920 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.310075 10728 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:39.292269 10900 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:39.342978  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.019s	sys 0.064s
I20260709 13:02:39.330192 10876 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:39.299291 10858 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:02:39.329229 10863 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:39.262050 10796 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:39.346479  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.020s	sys 0.064s
W20260709 13:02:39.352517  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 630429us
W20260709 13:02:39.352187  8675 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.280s	user 0.045s	sys 0.134s
I20260709 13:02:39.351280  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1395) took 1280 ms. Trace:
W20260709 13:02:39.353824  8675 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.282s	user 0.045s	sys 0.134s
W20260709 13:02:39.355137  8748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.447s	user 0.010s	sys 0.043s
W20260709 13:02:39.355868  8748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.447s	user 0.010s	sys 0.043s
I20260709 13:02:39.357129  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1978) took 1695 ms. Trace:
I20260709 13:02:39.354126  8677 rpcz_store.cc:276] 0709 13:02:38.071006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.071183 (+   177us) service_pool.cc:224] Handling call
0709 13:02:39.350983 (+1279800us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b54001ff4d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:39.351188 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68667776}
I20260709 13:02:39.350135 10858 raft_consensus.cc:3055] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:39.347195 10889 consensus_queue.cc:1243] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:39.397565  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:39.366366 10863 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.349529 10900 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:39.403384  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.404394  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.405367  9388 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:39.398388 10569 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.374464 10858 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:39.401170 10889 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:39.408465 10876 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.414491 10917 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.401180 10728 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:02:39.427126 10876 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.424546 10889 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.417996 10863 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.431564 10897 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.423099 10569 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.435281 10728 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.487874 10920 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:39.492540  8720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.028s	sys 0.033s
W20260709 13:02:39.495429  8720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.028s	sys 0.033s
I20260709 13:02:39.491519 10583 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:39.462703 10889 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:39.499009 10728 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.430068 10917 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:39.498134  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.490579 10888 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.499133 10591 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.499454 10583 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:02:39.533975  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.494871 10897 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.524327 10286 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:39.477480 10858 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.536242 10591 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.549640 10286 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:39.578464  8794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.008s	sys 0.014s
W20260709 13:02:39.579036  8794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.008s	sys 0.014s
I20260709 13:02:39.566140 10583 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:39.598256  8722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.011s	sys 0.038s
W20260709 13:02:39.614044  8722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.011s	sys 0.038s
I20260709 13:02:39.567879 10876 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:39.617574  9354 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.428s	user 0.001s	sys 0.050s
W20260709 13:02:39.618120  9354 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.428s	user 0.001s	sys 0.050s
I20260709 13:02:39.357717  8748 rpcz_store.cc:276] 0709 13:02:37.661401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.661648 (+   247us) service_pool.cc:224] Handling call
0709 13:02:37.661989 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.662231 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:37.908067 (+245836us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.356601 (+1448534us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.356681 (+    80us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:39.356773 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.356867 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.357044 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16554,"prepare.queue_time_us":53860,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":132,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":3798,"thread_start_us":1655372,"threads_started":2}
I20260709 13:02:39.619518  8796 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:39.621232  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.611456  8719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.012s	sys 0.020s
W20260709 13:02:39.622568  8719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.012s	sys 0.020s
W20260709 13:02:39.622958  9352 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.008s	sys 0.024s
I20260709 13:02:39.621390 10728 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:39.635732  8646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.000s	sys 0.035s
W20260709 13:02:39.655921  8646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.001s	sys 0.035s
W20260709 13:02:39.632412  8795 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.032s	sys 0.021s
W20260709 13:02:39.657833  8795 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.032s	sys 0.023s
W20260709 13:02:39.654884  9352 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.009s	sys 0.024s
I20260709 13:02:39.641248 10863 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:39.652804  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.006s	sys 0.015s
W20260709 13:02:39.665616  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.006s	sys 0.015s
I20260709 13:02:39.619762  8722 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:39.652190 10943 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:39.664863  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.665038  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 349575us
W20260709 13:02:39.663488  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 1.04673e+06us
I20260709 13:02:39.670003  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:39.671604  9388 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:39.672745  9352 connection.cc:256] RPC call timeout handler was delayed by -1.0513672030000407s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19920000000000002s for a response
W20260709 13:02:39.673246  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.654323 10871 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:39.675159  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1390) took 1968 ms. Trace:
I20260709 13:02:39.664649 10728 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.664801 10876 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.664741 10569 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:39.678745  8675 rpcz_store.cc:276] 0709 13:02:37.707193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.707933 (+   740us) service_pool.cc:224] Handling call
0709 13:02:37.708212 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.796433 (+ 88221us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:39.355403 (+1558970us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.674119 (+318716us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.674236 (+   117us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:39.674419 (+   183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.674727 (+   308us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.675079 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2733,"prepare.queue_time_us":1286908,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":10370,"thread_start_us":1544110,"threads_started":2}
W20260709 13:02:39.689426  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.693185  9352 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:39.673815 10959 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.701088  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:39.685672 10583 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.704414  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:39.705925  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:39.678891 10917 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:39.707263  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.681387 10910 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:39.689390 10861 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:39.719063 10963 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.686735 10871 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.726956 10963 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.728929 10871 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:39.729782 10963 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:39.695566 10876 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.719182  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.720175  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.709290  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:39.688947 10967 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:39.723126  8681 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.009s	sys 0.016s
I20260709 13:02:39.730516 10963 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:39.759912 10963 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.760725 10824 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.732944 10591 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:39.751304  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.763774 10591 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:39.764539  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:39.765533  8679 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:39.765707  8680 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:39.755337 10861 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:39.733331 10910 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.767970 10861 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.716435 10897 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:39.719098 10962 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:39.724117 10956 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:02:39.695154 10728 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.724771 10965 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:39.673766 10943 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.730432 10889 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.793345 10943 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:39.762897 10967 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.796671 10967 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:39.796969 10824 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:39.797804 10967 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:39.798856  8679 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:39.800171  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.774886  8673 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:39.801685  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:39.732674 10286 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:39.731386 10900 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:39.803589 10962 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.697160 10569 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.804912 10569 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.807267 10569 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:39.807825 10569 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:39.807852  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.808364 10569 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:39.808858  8678 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
W20260709 13:02:39.809337  8719 connection.cc:256] RPC call timeout handler was delayed by -1.0001631839999732s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
I20260709 13:02:39.809451 10569 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.810048  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:39.755987 10858 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:39.711544 10959 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:39.778741  8681 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.925s	user 0.010s	sys 0.017s
I20260709 13:02:39.700080 10920 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:39.733522 10964 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.714444 10960 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:02:39.781208 10863 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.783026  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:39.855993  8680 raft_consensus.cc:2410] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:39.793574  8674 raft_consensus.cc:2410] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:02:39.772951 10796 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:39.754978  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1394) took 1777 ms. Trace:
I20260709 13:02:39.813628 10286 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.814523 10900 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:39.776242  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.764542 10591 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:39.819927  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:39.825373  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:39.826098  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:39.834972  8679 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:39.823608  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:39.841814  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1397) took 1564 ms. Trace:
I20260709 13:02:39.845273  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:39.845454  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:39.846148  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:39.852006  8673 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:39.857311  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:39.770300 10910 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:39.862138  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.818145  8677 raft_consensus.cc:2410] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:39.861126 10964 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.868033  8755 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
W20260709 13:02:39.884434  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 291309us
I20260709 13:02:39.868655  8749 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.882992  8675 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:39.844170  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.888823  8747 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:39.869750  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.869571  8751 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:39.872181  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.882541  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.868929  8753 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:39.892426  8748 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 2: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:39.867827  8752 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:39.897411  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.898972  8678 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:39.899914  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.903429  8680 raft_consensus.cc:2410] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:39.903509  8676 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:39.908133  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.909343  8680 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:39.916337  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.931926  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.932713  8677 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:39.868171  8681 rpcz_store.cc:276] 0709 13:02:38.278062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.389990 (+111928us) service_pool.cc:224] Handling call
0709 13:02:38.390577 (+   587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.390924 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:38.853651 (+462727us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.841281 (+987630us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.841362 (+    81us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:39.841442 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.841556 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.841734 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":692731,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":145,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":11963,"thread_start_us":1431795,"threads_started":2,"wal-append.queue_time_us":868839}
I20260709 13:02:39.934664  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.879843 10963 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:39.938395  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 245456us
I20260709 13:02:39.938683  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:39.936895  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:39.942987  8752 raft_consensus.cc:2376] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2. Re-sending same reply.
I20260709 13:02:39.943332  8748 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:39.940033  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.861016 10920 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:39.860405 10863 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.944401  8673 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:39.864593 10969 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:39.945778  8679 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:39.897804 10889 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:39.946758  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:39.954849  8674 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:02:39.955310  8674 pending_rounds.cc:85] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 1
I20260709 13:02:39.908133  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:39.955575  8674 pending_rounds.cc:107] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:02:39.955852  8674 raft_consensus.cc:2889] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:39.955901  8676 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:39.956085  9352 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:39.957158  9352 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:39.948779  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:39.950438  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:39.958706  8748 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:39.947341  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 327979us
I20260709 13:02:39.951167  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:39.951817  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:39.959801  8751 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:39.864042 10286 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:39.961160  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:39.962096  9381 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:39.962718 10286 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:39.963716  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.879639 10900 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:39.951321  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 124071us
I20260709 13:02:39.929762 10964 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:39.952345  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.970746  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.972972  9352 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:39.975122  8749 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:39.980619  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:39.981657  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:39.983446  8679 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
W20260709 13:02:39.984926  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 365259us
I20260709 13:02:39.991479  8674 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:39.993028  8673 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:02:39.995023  8675 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:39.996368  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:39.997779  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:39.946074 10963 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.001050  8676 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:02:40.006454  8826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:58932
I20260709 13:02:40.006174  8754 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:40.007462  8826 raft_consensus.cc:606] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:40.006932  8676 pending_rounds.cc:85] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 3
I20260709 13:02:39.999842  8645 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:40.009194  8676 pending_rounds.cc:107] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:02:40.009635  8676 raft_consensus.cc:2889] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:39.971561 10824 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:39.964382 10910 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:40.013033  9388 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:39.983989 10286 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.979511 10943 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:39.989856 10981 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.030745  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:40.042275  9388 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:40.037750  8829 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:39.999959 10977 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.003055 10871 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:40.054288  8673 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 1. Current term is 2. Ops: [1.4-1.4]
I20260709 13:02:40.005210 10964 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:40.057396  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:40.010828 10980 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:40.058054  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 130320us
I20260709 13:02:40.018005 10861 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:40.058138  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:40.064116  8677 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:39.966303 10969 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:40.055640  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:40.076411  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 70382.8us
I20260709 13:02:40.083841  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:40.051678 10960 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:40.012979 10963 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:40.053143 10889 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:40.095147  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.093587  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 112011us
I20260709 13:02:40.057842 10964 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:40.104017  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:40.111219  8673 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:40.104238  8719 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:40.107539  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:40.027177 10990 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 13:02:40.043453 10286 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:40.119879  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:40.119231  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:40.115001  8678 raft_consensus.cc:2376] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2. Re-sending same reply.
I20260709 13:02:40.123201  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:40.124226  8719 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:40.048477 10981 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:40.109925 10897 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.136008 10897 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:40.136449 10998 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:40.139155 10900 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:40.139395 10975 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:02:40.040050 10591 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:40.057580 10983 consensus_queue.cc:1059] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
W20260709 13:02:40.155086  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.157745  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:40.158754  8826 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:40.130532  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 56836.5us
I20260709 13:02:40.125728 10982 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:40.132418  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:40.116240  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:40.093514 10960 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:40.080471 10986 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:02:40.140916 10897 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:40.073167 10977 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:40.182133  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.142692  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:40.075579 10980 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:40.131480  8678 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:40.116510 10985 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:40.171890  8829 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:40.140544 10959 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:40.194993  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 70892.2us
I20260709 13:02:40.115269 10889 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:40.192890  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:40.171108 10984 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.144002 10900 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:40.154318 10998 consensus_queue.cc:1059] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:02:40.196928  8678 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:40.157104 10993 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.195811  8719 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:40.190590 10960 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:40.171667 10583 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:40.179643  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 63586.1us
I20260709 13:02:40.200488  8673 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:40.204694  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:40.179833 10991 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.194388 10985 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:40.206103  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 198064us
I20260709 13:02:39.856710  8672 rpcz_store.cc:276] 0709 13:02:37.977731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.995466 (+ 17735us) service_pool.cc:224] Handling call
0709 13:02:38.249188 (+253722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.162486 (+913298us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:39.162944 (+   458us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.642511 (+479567us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.652160 (+  9649us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:39.654292 (+  2132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.754555 (+100263us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b5400175054. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:39.754667 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.754888 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":244,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":1112,"raft.run_cpu_time_us":3303,"raft.run_wall_time_us":8545,"spinlock_wait_cycles":1990158592,"thread_start_us":458033,"threads_started":1,"wal-append.queue_time_us":600405}
I20260709 13:02:40.212198  8678 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:40.195811 10863 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:40.213956 10286 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:40.214155 10988 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:02:40.214399 10977 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:40.219111  8680 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:02:40.195592  8722 connection.cc:256] RPC call timeout handler was delayed by -1.2017966679999859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
I20260709 13:02:40.223451  8673 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:02:40.216159 10981 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:40.197593 10996 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:40.179243 10982 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:40.232635  8829 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:40.232867 10995 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:40.198691  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:40.229851  9353 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:40.145704 10999 consensus_queue.cc:1059] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:02:40.203356 10900 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:40.171697 10964 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:40.256381  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.267920  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:40.240015  8748 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:02:40.266763  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:40.276522  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 66027us
I20260709 13:02:40.341953  8796 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:40.307778  8722 connection.cc:256] RPC call timeout handler was delayed by -1.2005381170000078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
I20260709 13:02:40.208709 10959 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:40.343351  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 113589us
W20260709 13:02:40.343972  8722 connection.cc:256] RPC call timeout handler was delayed by -1.1801966680000078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17450000000000002s for a response
W20260709 13:02:40.344391  8722 connection.cc:256] RPC call timeout handler was delayed by -1.1604277300000376s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 13:02:40.344693  8722 connection.cc:256] RPC call timeout handler was delayed by -1.1564381169999933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
I20260709 13:02:40.344578  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:02:40.345228  8722 connection.cc:256] RPC call timeout handler was delayed by -1.1450966680000079s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1784s for a response
W20260709 13:02:40.345644  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0496966679999673s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.189s for a response
I20260709 13:02:40.264863  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:40.345927 11004 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:40.316221 11005 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:40.316399 10995 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:02:40.345995  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0091966680000155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1935s for a response
I20260709 13:02:40.347857 10889 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:40.229882 10985 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:40.348928  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.349344  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.349866  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.267132 10286 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:02:40.350718 10987 log.cc:927] Time spent T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Append to log took a long time: real 0.166s	user 0.002s	sys 0.000s
I20260709 13:02:40.350494 10286 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:40.233832 10583 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:40.329387  8826 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:40.332454  8674 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:02:40.364698  8675 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:40.208038 10984 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:40.365537  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 204194us
I20260709 13:02:40.339617  8672 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:40.341279  8828 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:40.306308  8673 pending_rounds.cc:85] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 6
I20260709 13:02:40.368254  8673 pending_rounds.cc:107] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:02:40.368665  8673 raft_consensus.cc:2889] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:40.345952 10796 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:40.316323  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.347554 10995 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:40.349234 10900 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:40.229998 10991 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:40.323678 10999 raft_consensus.cc:3055] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:40.358520  8678 raft_consensus.cc:2410] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:40.334719 10989 raft_consensus.cc:3055] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:40.364706  8674 pending_rounds.cc:85] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 4
I20260709 13:02:40.389189  8674 pending_rounds.cc:107] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:02:40.389492  8674 raft_consensus.cc:2889] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:40.278982 10998 raft_consensus.cc:3055] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:40.379142  8675 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:02:40.380793  9353 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:40.367929 10967 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:40.392807  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 44264.7us
I20260709 13:02:40.383289 10796 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:40.387557 11004 raft_consensus.cc:697] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:40.423154  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:40.424185  9382 raft_consensus.cc:2393] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:40.423743 11004 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:40.424467 10991 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:40.425093 10969 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:40.393781 10998 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:40.426734 10998 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:40.428093 10998 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:40.428685 10959 consensus_queue.cc:1048] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:40.309332 11001 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.130s
I20260709 13:02:40.266927 10993 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:40.435493 10980 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:40.387539 11005 raft_consensus.cc:697] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:40.384313 10963 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:40.437474 11005 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:40.388949 10989 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:40.387321 10999 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:40.412389 10967 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.472481  8722 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:40.417889 10871 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:40.408190 10996 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:40.421376 10995 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:40.435475  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:40.442314 10796 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:40.471589 11007 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:40.505016 11007 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:40.507031 10286 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:40.508949  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47312.4us
I20260709 13:02:40.510115  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:40.515703  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 287338us
I20260709 13:02:40.490099 10967 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:40.518524  8722 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:40.511638 11011 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:02:40.487452 10991 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:02:40.538048  8719 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:40.521710 11009 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:40.521252 10286 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:40.516418  9381 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:40.524739 10989 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:40.675835  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:40.722263  9388 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:40.639755  8681 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:40.685891  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 137723us
W20260709 13:02:40.718567  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.540300 10858 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.697s	user 0.021s	sys 0.000s
W20260709 13:02:40.628737 10861 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.554s	user 0.013s	sys 0.003s
I20260709 13:02:40.724982  8676 raft_consensus.cc:2410] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:40.725180 10861 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:40.629240  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:40.726105  9355 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:40.726989  8672 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 4: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:40.727169 10943 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:40.602638 10824 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.573s	user 0.014s	sys 0.004s
I20260709 13:02:40.728358 10824 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:40.680608 10889 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:40.715608  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:40.729645  9355 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:40.729957 10889 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:40.686546  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:40.733453  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:40.550431  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:40.739176  8678 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:40.630923 11007 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:40.536734 10871 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:40.540799 10999 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:40.565811 10969 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.544301 10981 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:40.787125  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 51328.5us
W20260709 13:02:40.788213  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 51856us
I20260709 13:02:40.788539  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:40.682049  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.635993 10993 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:40.720199 10989 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:40.565850 11001 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:40.688746 11010 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:02:40.566140 11000 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:40.627542 11009 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:40.627873 10984 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:40.604990  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:40.804900  8675 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:40.745064  9355 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:40.749032  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:40.786209 10965 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:40.751103  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.576834  8722 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:40.772936  8796 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:40.822647  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 67758.5us
W20260709 13:02:40.774901  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 231502us
I20260709 13:02:40.823997  8719 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:40.780038  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 412726us
I20260709 13:02:40.791746 10910 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:40.789300  8681 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:40.791935  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:40.841144  8674 raft_consensus.cc:2393] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:40.657755 10583 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:40.887116  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 61409.4us
I20260709 13:02:40.670711 10982 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:40.625253 10569 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:40.798822  8678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:40.924991  8678 raft_consensus.cc:606] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:40.658463 10962 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:40.928793  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:40.930197  8749 raft_consensus.cc:2410] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:40.684391 10977 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:40.808238  8676 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:40.749902  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:40.806187  8647 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:40.604131 10286 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.194s
W20260709 13:02:40.630167 10976 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.672s	user 0.000s	sys 0.005s
I20260709 13:02:40.565749 10980 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:40.828871 11013 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:40.960006  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:40.960265  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:40.960763  8754 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
W20260709 13:02:40.878232  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.888545  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 64677.6us
I20260709 13:02:40.961817  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:40.744517 10960 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:40.802992 10917 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.795346 10981 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:40.803936 10861 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.925406  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:40.925401  8678 raft_consensus.cc:614] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:40.927706  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.927747  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 386807us
I20260709 13:02:40.987954  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:40.931264  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:40.550966 10959 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:40.931893  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 178112us
I20260709 13:02:40.827085 10999 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:40.936437  8750 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:40.940199  8752 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:02:40.792439 10969 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:40.942159  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 136083us
I20260709 13:02:40.809070 11000 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:40.943746  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 133752us
I20260709 13:02:40.891264 10993 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:40.602272 10996 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:40.803376 10986 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.126s
I20260709 13:02:40.919775 10943 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:40.954264  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:40.946525  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:40.959142  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 214189us
I20260709 13:02:40.936154 10569 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:40.922956 10990 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:40.969214  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:40.971280  8751 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:40.971280  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:41.021270  8826 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:40.994134 10858 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:40.945878 10977 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:40.926023 10583 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:40.927384 10983 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.943830  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:40.996658  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:40.998342  8756 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:41.029739  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.999787  8754 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:41.001325  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:41.001686  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 180287us
I20260709 13:02:40.962749  8829 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:41.015991  8753 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:41.032346  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.033528  8829 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
W20260709 13:02:41.014783  8793 leader_election.cc:336] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.207s (SENT)
I20260709 13:02:41.034578  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:40.930050 10965 raft_consensus.cc:697] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:41.037190  8823 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:02:41.037427 10889 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:40.993294 10959 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:41.038688  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.039688  8823 raft_consensus.cc:2410] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:02:40.994768 10728 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:41.040558 10994 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:41.040676  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:40.816902 11001 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:40.997861  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:41.042335  8747 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:40.979012 10980 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.044555 11001 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:41.044730 10980 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:41.045063 10900 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.000s
I20260709 13:02:41.038302 10965 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.045583 10900 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.046811  8825 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:41.047979  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.048153 10969 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:41.025118  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.038858 10959 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:41.031067  8752 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:41.035995  8756 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:41.012146 10993 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:41.053399  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:41.037282  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:41.054498  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:41.054637  8647 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:41.054782  8829 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.056469  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.000571 10910 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:41.059140  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:41.028661 10962 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.054864 10943 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:41.056564 10876 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:40.946121 10871 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:41.064615 10796 consensus_meta.cc:287] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.002s
I20260709 13:02:41.065263 10920 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:40.996374 10861 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:40.994601 10917 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.012732 11038 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:41.082918  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 46736.5us
I20260709 13:02:41.083423  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:41.065095  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.022560 11039 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.049099 10858 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.052313 10980 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:02:41.024529 10983 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.024437 10583 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.027654 10967 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.027839 10981 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.023129 11009 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.084697  8647 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:41.151635  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 323) took 1068 ms. Trace:
I20260709 13:02:41.077351  8826 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
W20260709 13:02:41.332515  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 247934us
I20260709 13:02:41.332641  8822 rpcz_store.cc:276] 0709 13:02:40.083746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.657346 (+573600us) service_pool.cc:224] Handling call
0709 13:02:40.682885 (+ 25539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.020349 (+337464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.020384 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.020446 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:41.020483 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.151368 (+130885us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400126454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:41.151422 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.151561 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":775510272}
I20260709 13:02:41.334038  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.334806  8826 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:41.335201  9386 raft_consensus.cc:2410] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:02:41.080602 10728 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:41.335848  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.336511 10728 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.082918 11002 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:41.337293 10796 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.337383  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:41.145650  8672 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:41.252051  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1894) took 1371 ms. Trace:
I20260709 13:02:41.338302 10861 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:41.085399 10962 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.339296  8828 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:41.342413 10960 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:41.189194 10889 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.132746  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:41.058306 10993 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:41.133255  9387 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:02:41.341600  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 287249us
I20260709 13:02:41.347522  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:41.341519  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:41.374454  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:41.374764  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:41.375222  8830 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:41.358879 11002 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:41.381501  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.348353  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:41.349237  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.114845 10998 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.686s	user 0.012s	sys 0.005s
I20260709 13:02:41.241369  8829 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:02:41.382412 11002 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.383247  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.385175  8830 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:41.386188  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.387058  8827 raft_consensus.cc:2410] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:41.383443  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:41.387807 10977 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:41.388355 10977 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:41.388856  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:41.360224  9388 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
W20260709 13:02:41.206204 10995 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.704s	user 0.005s	sys 0.009s
I20260709 13:02:41.275629  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:41.384292  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.391075  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:41.392861  8823 raft_consensus.cc:2410] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:41.356133 11039 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.397317 11039 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:41.386377 10989 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.003s
I20260709 13:02:41.341570 10900 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:41.342948 11038 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:41.086038 10985 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.737s	user 0.012s	sys 0.006s
I20260709 13:02:41.085386 10969 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.363683  8680 rpcz_store.cc:276] 0709 13:02:39.880524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.307509 (+426985us) service_pool.cc:224] Handling call
0709 13:02:40.357623 (+ 50114us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.371555 (+ 13932us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.371610 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.385837 (+ 14227us) log.cc:844] Serialized 26 byte log entry
0709 13:02:40.528467 (+142630us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:40.826135 (+297668us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.251486 (+425351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.251617 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.251706 (+    89us) raft_consensus.cc:1685] finished
0709 13:02:41.251726 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.251947 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":128795,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":170942,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":26794,"thread_start_us":817706,"threads_started":3,"wal-append.queue_time_us":289472}
I20260709 13:02:41.108595  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:41.341444 11000 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:41.156556  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:41.114163 10876 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:41.356277 10920 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:41.356375 10728 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:41.335850  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:41.376698  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.141607 10910 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.376993 10861 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.377507  9381 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:41.371558 10965 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:41.373154 10889 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.351159 10959 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.518266 10861 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.525524 10959 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:41.525664  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.386937  8830 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:41.366438  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:41.527199  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 160916us
I20260709 13:02:41.528605 10796 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:41.529357 10796 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:41.529764 10796 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:41.530141 10796 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:41.343914 10949 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.008s	sys 0.011s
I20260709 13:02:41.381217  9384 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:02:41.530826 10949 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.000s	user 0.008s	sys 0.011s
I20260709 13:02:41.403277 10977 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:41.402269  8824 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:41.418546 10964 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:41.407109 11035 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:41.535372  8827 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:41.438839 11002 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.442826 10876 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.436846  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:41.443905 10569 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:41.435141 10985 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.436846  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.416026 11038 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:41.423128 10917 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:41.404023 10980 raft_consensus.cc:2764] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:41.433911 10999 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:02:41.532061 10977 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:41.503731 10998 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:41.504385 10583 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:41.435784 10994 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:02:41.506096  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:41.505914 11001 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:41.424685 10995 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:02:41.424503 10982 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:02:41.115307 10943 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.424324 11046 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:41.436936 11027 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:02:41.427130 11008 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:41.510083  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.506132 10967 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:41.527281  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:41.598003  8830 raft_consensus.cc:2376] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4. Re-sending same reply.
W20260709 13:02:41.414340  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 425673us
I20260709 13:02:41.533831  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.600152  8828 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:41.602010  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:41.536845  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:41.536986  8673 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:41.532096 10889 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:41.556425 10985 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:41.558734 10876 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:41.559497 11020 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.518496 10965 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.563320  9355 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:41.532258 10861 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:41.425477  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:41.572738  8826 raft_consensus.cc:2376] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7. Re-sending same reply.
W20260709 13:02:41.643674  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 218314us
I20260709 13:02:41.574823  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:41.578891  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:41.580539  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 164000us
I20260709 13:02:41.563968  9380 raft_consensus.cc:2368] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7.
W20260709 13:02:41.655531  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 92865.6us
W20260709 13:02:41.594762  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 88744.2us
W20260709 13:02:41.597433  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.438182 10920 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.666678 10920 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:41.618059 10977 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.608536  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.668341  8828 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:41.669255 10980 raft_consensus.cc:3055] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:41.669520  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:41.670249  8828 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.611790  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:41.671090 11050 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:41.671319 11047 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:02:41.671886  8830 raft_consensus.cc:2410] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:41.607271  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.617622  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.612176 10983 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:41.625113 10917 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:41.626027 11018 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.995s	user 0.004s	sys 0.000s
I20260709 13:02:41.588303 10979 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:02:41.594595 11046 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.681537  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.629148 10583 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.629957  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:02:41.682837  8822 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:41.648674  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:41.663679  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:41.595745 11007 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:02:41.665012  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:41.665915  8796 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:41.667598 11014 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.669727 10980 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:41.672868  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.719060 11014 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.719573 10980 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.720285  8830 raft_consensus.cc:2410] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:41.720918 10980 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:41.721545 11014 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:41.667636  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:41.692873 11001 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:02:41.723037  9387 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:41.604462 11035 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:41.724443 11035 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.724866 11035 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:41.620628 10962 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:41.673065  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.727736 10969 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:41.678720  8647 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.501s (SENT)
I20260709 13:02:41.728703  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:41.729763  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51154.8us
I20260709 13:02:41.730945  8647 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:41.681701  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:41.628376 10889 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:41.732317 10858 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:41.732079 10889 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.732947 10858 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:41.733739 10858 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:41.734109 10858 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:41.648416  8678 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.675643 10861 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:02:41.644991 10965 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:41.696195  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 47602.6us
I20260709 13:02:41.692770  8821 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:41.707063  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:41.693547 11046 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:41.707553 10964 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:41.704231 10583 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.679987 10876 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.795267 10876 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.796278 10583 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:41.718393 10920 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.797683 10876 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:41.760246 11003 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:41.711905  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:41.784642 10964 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:41.799199 10964 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:41.799294  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:41.799708  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 200054us
I20260709 13:02:41.800302 10920 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:41.800527  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:41.802002  8753 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:41.803938 11009 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:41.773034 10990 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:41.804507 11009 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:41.711026  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:41.735724  8825 raft_consensus.cc:2410] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:41.809347  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:41.802547  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:41.808017  8678 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
W20260709 13:02:41.732344  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50790.1us
I20260709 13:02:41.742587  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:41.812824  8829 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:41.645393 11020 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.757009  8796 leader_election.cc:400] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7.
I20260709 13:02:41.813655  8796 leader_election.cc:403] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:41.813943  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:41.814164  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 109427us
I20260709 13:02:41.757432 10591 raft_consensus.cc:3055] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:41.734920 10824 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:41.737484 11052 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 13:02:41.758513 10993 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:41.759500  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 790348us
I20260709 13:02:41.701859 11027 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:02:41.701786 10983 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:41.741365 10995 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:41.765033  8794 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:41.832434  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 67533.2us
I20260709 13:02:41.833524  8794 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:41.773581  8828 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:02:41.777848  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:41.692061 10910 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:41.734426 10889 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:41.793002  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:41.734517 10858 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:41.805300  9383 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:41.744616  8830 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:41.812242  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:41.859833  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:41.860054  8756 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:41.857417  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:41.822301 11014 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:41.811452  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:41.821115  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:41.825723  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:41.826884  9389 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:41.840108  9355 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:41.841153 10982 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:41.842521  8754 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:41.854307  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:41.798262  8751 raft_consensus.cc:3055] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:41.818287 10591 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:41.858253  8747 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:41.817592  8748 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:41.867487  8751 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:41.861313  8749 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:41.828719 10991 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:41.842593 10967 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:41.809590 10796 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.868484  8751 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.843456 10910 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.882197  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:41.884938 10871 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:02:41.885131  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 82663.6us
I20260709 13:02:41.885586 11003 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:41.824940 10969 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:02:41.886287 11055 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:41.886811  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 63860.9us
I20260709 13:02:41.891943  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:41.892217 11020 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:41.893168 10889 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:41.874554 10982 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.843861 11007 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7.
I20260709 13:02:41.913444  8751 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:41.831036 10998 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:02:41.859133 10994 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.874727  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:41.857957 11004 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:02:41.896552 10986 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:41.828410 10824 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:41.836721 11054 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.139s
I20260709 13:02:41.855768 11058 consensus_queue.cc:1059] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.114s
I20260709 13:02:41.897110 10967 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:41.994544 10983 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:41.999081  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 107270us
I20260709 13:02:41.906353 10910 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:41.997656  8719 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:42.058828 11064 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:42.059381 11064 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:42.059792 11064 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:42.060488 10983 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.054069 10967 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:42.002884 11035 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:41.970454 11050 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.970002  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:41.894528 10728 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.007977 10982 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:42.017984 11003 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:42.025708  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.861553 11009 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:42.033501  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 179388us
I20260709 13:02:41.896656 10991 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.011624  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:42.027132 10591 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.078850 11050 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.080904 11042 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.029512 10994 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.079255 10959 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:42.081418  9383 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:42.055069 11038 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.089782 10991 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:42.104715 11046 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.067941 10986 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:41.860257 10990 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.049273  9384 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:42.012321 11055 raft_consensus.cc:697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:02:42.065336  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 246872us
I20260709 13:02:42.067989  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:42.115191  8751 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:02:42.115470  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47586.6us
I20260709 13:02:42.116449  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:42.116544  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:42.070327  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 72799.9us
I20260709 13:02:42.080672  8722 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:42.117856  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.118759  8722 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:42.044063 11063 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:42.124425  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:42.118521  8751 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:42.128954  8672 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:42.131299 10861 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:42.133044  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:42.133889  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:42.134037 10861 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:42.136811  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:42.137411  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:42.138238  8677 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:42.141883  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:42.102485 10959 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.146507  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:42.094743 10591 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:42.112622 10969 raft_consensus.cc:3055] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:42.151350  8677 raft_consensus.cc:2376] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3. Re-sending same reply.
I20260709 13:02:42.146900  8722 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:42.102025 10994 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:42.098256 11050 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:42.147984 10861 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:42.138520 10889 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:42.096994 10993 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:42.139319 10991 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:42.193078  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:42.193353 10991 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.164906  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:42.173262  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.173743  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:42.194917  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:42.138376 10583 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:42.179672  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:42.182152  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:42.145318  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:42.149247 10963 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:42.185496  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:42.188012  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:42.147841 10728 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:42.156023 10959 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:42.150242 10920 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:42.239017 10920 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.239629 10920 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:42.243391  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:42.244865  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:42.191293  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 73392.3us
I20260709 13:02:42.246938  8719 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:02:42.247635 11070 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:02:42.247932  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:42.249151  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:42.193835 11001 raft_consensus.cc:3055] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:42.092664 10965 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:42.207340  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.160717 10969 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:42.192080  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 335) took 1176 ms. Trace:
I20260709 13:02:42.292338  8822 rpcz_store.cc:276] 0709 13:02:41.015605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.705335 (+689730us) service_pool.cc:224] Handling call
0709 13:02:41.784129 (+ 78794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.784444 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:41.991259 (+206815us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.191342 (+200083us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.191564 (+   222us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.191609 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:42.191675 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.191755 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.192010 (+   255us) inbound_call.cc:177] Queueing success response
0709 13:02:42.280968 (+ 88958us) spinlock_profiling.cc:243] Waited 62.6 ms on lock 0x7b1800360d60. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
Metrics: {"mutex_wait_us":24423,"prepare.queue_time_us":211486,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":85,"raft.queue_time_us":29040,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":13832,"spinlock_wait_cycles":159488512,"thread_start_us":344613,"threads_started":2,"wal-append.queue_time_us":184520}
I20260709 13:02:42.214447  8675 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:42.179934 10981 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:42.298903 10981 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.299363 10981 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.216177  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:42.162766 10983 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:42.217885  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:42.202714  8681 raft_consensus.cc:2376] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4. Re-sending same reply.
I20260709 13:02:42.224228  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 1999) took 1245 ms. Trace:
I20260709 13:02:42.189033 10889 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:42.235045  9380 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:42.245728  8672 raft_consensus.cc:2376] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3. Re-sending same reply.
I20260709 13:02:42.188763 10876 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.271677 11001 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:42.183665 10861 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.272368  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:42.142189 10986 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.331482  8822 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:42.333192  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:42.278797  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.192203 11038 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:42.193858 10991 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:42.179651 11008 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.203733 11050 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:42.355952 11027 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:42.356410 11050 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.356729 10871 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:42.357789 10871 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:42.357551 10981 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.358211  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:42.303331  8750 rpcz_store.cc:276] 0709 13:02:40.978940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.994192 (+ 15252us) service_pool.cc:224] Handling call
0709 13:02:40.994557 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.425730 (+431173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.425779 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.452739 (+ 26960us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.808044 (+355305us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:41.808121 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.808202 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.223955 (+415753us) raft_consensus.cc:1685] finished
0709 13:02:42.223981 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.224146 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":400471,"raft.queue_time_us":763836,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":309976,"threads_started":2,"wal-append.queue_time_us":351155}
I20260709 13:02:42.331338 10858 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.290199 11054 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:42.358234 10871 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:42.359665 11054 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:42.360147 11054 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:42.360088 10981 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:42.360772 10858 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:42.361366  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:42.320801  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:42.322237  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 156842us
I20260709 13:02:42.290905 11002 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:42.327495  8680 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:42.360723 11027 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:42.363993 11002 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.364471 11002 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:42.364542  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:42.364663 11027 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:42.365474  8680 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 4: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:42.097512 11042 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:42.366770 11071 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.287s	user 0.002s	sys 0.000s
I20260709 13:02:42.367866  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:42.363152  8722 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.736s (SENT)
I20260709 13:02:42.368568 11042 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:42.275622 10996 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:42.369316 10996 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.369705 10996 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:42.370041  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:42.371349  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:42.373114 11027 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:42.374122 11027 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:42.278499 10583 raft_consensus.cc:697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:02:42.375414 11073 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:02:42.376673  8793 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:42.335258  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:42.379760  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:42.335981  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:42.336299  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:42.380882  8827 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:42.339478  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:42.362922  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:42.366438  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:42.382602  8680 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:42.381500  8679 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:42.319656 11046 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:42.317921 10889 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:42.342103  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:42.345661 11056 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:42.336526 10995 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.351572 10965 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:42.335004 10964 consensus_meta.cc:287] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.535s	user 0.012s	sys 0.000s
I20260709 13:02:42.353482  9389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 575) took 1582 ms. Trace:
I20260709 13:02:42.298368 10982 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.327862 10876 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.355746  9384 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:42.362226 10993 raft_consensus.cc:2764] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:02:42.360286 10871 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.380803  8681 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:42.337284  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:42.422662  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:42.331005 10861 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:42.426759  9383 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
W20260709 13:02:42.428174  8781 heartbeater.cc:646] Failed to heartbeat to 127.7.89.190:45099 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.7.89.190:45099 timed out after 15.000s (SENT)
I20260709 13:02:42.383714 11004 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.396798  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:42.401618  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:42.445788  8680 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:42.396288  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:42.336797 11003 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:42.413213  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.399271 11056 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:42.413422  9389 rpcz_store.cc:276] 0709 13:02:40.771063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.771184 (+   121us) service_pool.cc:224] Handling call
0709 13:02:40.771544 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.866168 (+1094624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.353166 (+486998us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400520b54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:42.353371 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2426622720}
I20260709 13:02:42.457302  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.423527  8830 raft_consensus.cc:1813] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Handling vote request from an unknown peer 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:42.458509  8830 raft_consensus.cc:2368] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:42.403384 11013 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:42.421306  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.460383  8750 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:42.340058  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:42.443703  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.444195  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:42.444307  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:42.461714  8755 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.462174  8678 raft_consensus.cc:2410] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:42.432251  8796 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:02:42.368647 11075 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 13:02:42.449074  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:42.419699  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:02:42.430513 11076 log.cc:927] Time spent T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.215s	user 0.000s	sys 0.003s
I20260709 13:02:42.457777  8673 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:42.475680  8681 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 4: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:42.319578 11055 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.414743  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.483455  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.465754  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:42.485204  8750 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:42.418779 11050 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:42.446667 11035 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:42.478050  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.459065 11004 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.471405 10998 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.481648  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.403734 11008 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.486385  8753 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:42.387243 10969 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.417367 10995 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.399820 10569 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:42.491950  9382 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.424912 10964 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.375445 10583 consensus_queue.cc:237] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.530583  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.517761 11064 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.476883 10871 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:42.525725  8825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1495) took 1455 ms (client timeout 1927 ms). Trace:
I20260709 13:02:42.534494  8755 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
W20260709 13:02:42.555563  8825 rpcz_store.cc:269] 0709 13:02:41.070310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.390723 (+320413us) service_pool.cc:224] Handling call
0709 13:02:42.525455 (+1134732us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b540041c154. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:42.525639 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":790240384}
I20260709 13:02:42.526057  8673 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:42.556586  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.506583  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.515678  8823 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:42.522449 10981 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.557859  8756 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:42.560093  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.563325  8825 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:42.468022 11002 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.568097  8822 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:42.467712 11003 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.576668  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.473052 11082 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:42.578351 11056 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.596637 10982 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:42.597204 11056 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:42.597621 11064 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:42.571897 10910 consensus_meta.cc:287] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.512s	user 0.013s	sys 0.000s
I20260709 13:02:42.608091  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.609387  8756 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:42.524127 11013 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:02:42.584488 10969 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:42.626490  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.564136 10981 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.510973 10998 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.630108 11002 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:42.631681 11003 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:42.633845 11004 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:42.643006  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:42.648521  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.615294 11082 raft_consensus.cc:697] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:42.656406  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.598421 10920 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.688146  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.618238 10910 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.676247  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:42.675840  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:42.690546 10996 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.732757 11000 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:42.733383 11000 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:42.733541  8755 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:42.734560 10996 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:42.734486 11000 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:42.750821  8830 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:42.689983 11082 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.705873  8677 raft_consensus.cc:2368] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5.
I20260709 13:02:42.971685  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:42.971992  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.973264  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:42.973580  8645 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.641s (SENT) [suppressed 47 similar messages]
I20260709 13:02:42.974105  8748 raft_consensus.cc:2410] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:02:42.806346  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 951731us
I20260709 13:02:42.691841  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.978266  8750 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:42.972767  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:42.741928  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:42.981914  8825 raft_consensus.cc:2410] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:42.982421  9389 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:42.980880  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.736646  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:42.989405  9382 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:02:42.990077  8752 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:42.992812  8722 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:43.000473  8794 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:42.762413 11086 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:02:42.959429 11000 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:42.932022 10967 consensus_meta.cc:287] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.868s	user 0.001s	sys 0.014s
I20260709 13:02:43.005852  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:43.028440  8822 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:42.735965 10998 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:43.023554  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:42.984925  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:42.738632  8821 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:43.035749  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:43.036676  8821 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:43.021445 10983 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:43.039148 10983 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.039572 10983 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:43.040970 10591 consensus_meta.cc:287] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.881s	user 0.012s	sys 0.000s
W20260709 13:02:43.011643  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 275104us
I20260709 13:02:42.961532 10910 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:42.999220 11007 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.978678 11008 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:43.032258  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:42.746953  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:42.851135 10876 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:43.031373  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:43.004756 10999 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.005090  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:42.707196 11054 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:43.056360  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:42.961808 10991 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:42.810297 10917 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.220s
I20260709 13:02:43.059471  8748 raft_consensus.cc:2376] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6. Re-sending same reply.
I20260709 13:02:43.021243 11003 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:43.061378  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:42.706809 10861 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:43.062209  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:43.026465 11064 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:43.048136 10995 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:43.067011  8822 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:43.026598 11014 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:02:43.075975 10796 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:43.076478  8823 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:43.072943  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:43.064517  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:43.052101 10990 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.097543  8749 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:43.061925 10981 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:43.105932  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:43.108286  8751 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:02:43.071169 10917 consensus_queue.cc:1243] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:02:43.043123 10967 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:43.094314 10876 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:43.093364 11064 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.062556 11007 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:43.160451  8676 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:43.095700 11003 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:43.184930 10861 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:43.185396 11003 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:43.069787 10999 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:43.175658  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:43.096470 10920 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:43.071242 11050 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
W20260709 13:02:43.112603 10889 consensus_meta.cc:287] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.700s	user 0.007s	sys 0.007s
I20260709 13:02:43.181394  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:43.188992  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:43.036703 11001 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:43.197238  8828 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:43.211853  8830 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:43.205785  8822 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:43.218844  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:43.181123 11064 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:43.222281  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:43.245901  8830 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:43.224697  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:43.111763 10990 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:43.230500 11091 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:02:43.231449 11090 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:02:43.193053 10876 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:43.241139  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:43.191911 10861 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:43.187743 11088 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.114s	user 0.002s	sys 0.007s
W20260709 13:02:43.223965 10969 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.588s	user 0.011s	sys 0.003s
W20260709 13:02:43.246460  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 47374.9us
I20260709 13:02:43.247318  8749 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:43.223397 10920 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:43.215137 10889 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:43.238795 11001 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:43.407872  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.279307  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:43.284205  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:43.558470  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:43.530557  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1939) took 1181 ms. Trace:
I20260709 13:02:43.292063 10824 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:43.557206  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1938) took 1209 ms. Trace:
W20260709 13:02:43.416646  8673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 1933) took 1192 ms (client timeout 1529 ms). Trace:
I20260709 13:02:43.459398  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:43.412941  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:43.261999  8830 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:43.289827  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:43.543596  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 380429us
I20260709 13:02:43.573721  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:43.420922  8722 leader_election.cc:400] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:43.261264  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:02:43.547421  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 474542us
I20260709 13:02:43.453948  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 1922) took 1292 ms. Trace:
I20260709 13:02:43.287575  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 313) took 1194 ms. Trace:
I20260709 13:02:43.280184  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:43.529259  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:43.578493  8647 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:43.569296  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:43.256309 10959 consensus_meta.cc:287] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.012s	user 0.019s	sys 0.000s
I20260709 13:02:43.289024  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:43.261425  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:43.459540  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:43.237327  8823 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:43.400218  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:43.535446  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1937) took 1203 ms. Trace:
I20260709 13:02:43.553391  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:43.415930  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.574687  8722 leader_election.cc:403] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:43.528115  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.289073  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:43.551736 10917 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.156s
I20260709 13:02:43.667647  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:43.872419  8825 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:43.719561  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:43.572364  8673 rpcz_store.cc:269] 0709 13:02:42.224024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.320480 (+ 96456us) service_pool.cc:224] Handling call
0709 13:02:43.416366 (+1095886us) spinlock_profiling.cc:243] Waited 94.8 ms on lock 0x7b54002a8fd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:43.416553 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":217991936}
I20260709 13:02:43.874109  8830 raft_consensus.cc:2410] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:43.572137  8675 rpcz_store.cc:276] 0709 13:02:42.348374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.693642 (+345268us) service_pool.cc:224] Handling call
0709 13:02:42.705104 (+ 11462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.705609 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.705638 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.707903 (+  2265us) log.cc:844] Serialized 26 byte log entry
0709 13:02:43.095214 (+387311us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:43.095270 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.095350 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.556882 (+461532us) raft_consensus.cc:1685] finished
0709 13:02:43.556915 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.557113 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":929,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":2595,"thread_start_us":331068,"threads_started":1,"wal-append.queue_time_us":667841}
I20260709 13:02:43.701995  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:43.890529  8753 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:43.890976  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:43.579809  8677 rpcz_store.cc:276] 0709 13:02:42.161897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.164617 (+  2720us) service_pool.cc:224] Handling call
0709 13:02:43.417072 (+1252455us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b54002413d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:43.453841 (+ 36769us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1176574208}
I20260709 13:02:43.529713 11050 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.570084  8749 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:43.827219  8829 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:43.291316 11009 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:43.629487  8674 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 4: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:02:43.640218  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 111042us
I20260709 13:02:43.941354  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:43.569325  8680 rpcz_store.cc:276] 0709 13:02:42.348745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.955322 (+606577us) service_pool.cc:224] Handling call
0709 13:02:42.955840 (+   518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.956504 (+   664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.956562 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.957261 (+   699us) log.cc:844] Serialized 26 byte log entry
0709 13:02:43.245969 (+288708us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:43.246031 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.246163 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.530231 (+284068us) raft_consensus.cc:1685] finished
0709 13:02:43.530267 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.530466 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":140,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":23654,"thread_start_us":220788,"threads_started":1,"wal-append.queue_time_us":466346}
W20260709 13:02:43.543182  8645 leader_election.cc:336] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.718s (SENT)
I20260709 13:02:43.944893  8753 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:43.945356  8645 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:43.945567  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:43.946357  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 403224us
I20260709 13:02:43.563191 10996 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.581905  8647 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:43.783934 11055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.011s	sys 0.010s
W20260709 13:02:43.880889  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 324720us
W20260709 13:02:43.272065 11094 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:02:43.277964 11056 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.680s	user 0.002s	sys 0.009s
I20260709 13:02:43.830893 10910 raft_consensus.cc:2764] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:43.881475  8751 rpcz_store.cc:276] 0709 13:02:42.092838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.133308 (+ 40470us) service_pool.cc:224] Handling call
0709 13:02:42.133733 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:43.109080 (+975347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.287113 (+178033us) spinlock_profiling.cc:243] Waited 969 ms on lock 0x7b5400139054. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:43.287452 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2228004096}
I20260709 13:02:43.897457 10994 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.973443  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:43.590456  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1934) took 1258 ms. Trace:
W20260709 13:02:43.455375 10871 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.896s	user 0.018s	sys 0.001s
W20260709 13:02:43.980471  8676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 1928) took 1788 ms (client timeout 1639 ms). Trace:
I20260709 13:02:43.572703 10861 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:43.947554 10591 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:02:43.953989 11007 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:43.853333  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.721868  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:43.992300  8748 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:43.880743  8678 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:43.874549  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:43.886725  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:43.888502  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:43.704299  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:43.994313  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:43.995002  8822 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:43.995919  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:44.017026  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:43.286093 10999 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:43.458818 10991 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:43.884851  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:43.484551  9383 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:43.375722 10981 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:43.944646  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.573161  8826 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:43.879801  8755 raft_consensus.cc:2410] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:02:43.703254 10983 consensus_meta.cc:287] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.663s	user 0.009s	sys 0.004s
I20260709 13:02:44.035446 10983 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:43.899539  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 478841us
I20260709 13:02:43.961963 10996 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:43.491108  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:43.453378 11014 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.523000 10967 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:43.946321  8680 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:43.955315  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:44.043118  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:44.044633  8749 raft_consensus.cc:2410] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:43.953891  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:44.044752  8678 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:44.044914  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:43.701913 10583 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.045784  8680 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:43.847003 11101 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:43.271188 11092 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:02:43.831269  9380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 586) took 1443 ms. Trace:
I20260709 13:02:43.640112 10824 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:43.971827  8754 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:43.974701  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:43.974774  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:43.880724  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:44.063731 11096 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.284423 10876 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.069962  8680 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:44.071357  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.072515 10876 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:44.072642  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:44.073184 10876 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:44.073472  8680 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:44.073730 10876 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:44.074330  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:43.981063  8676 rpcz_store.cc:269] 0709 13:02:42.192097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.217582 (+ 25485us) service_pool.cc:224] Handling call
0709 13:02:43.980165 (+1762583us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b54004303d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:43.980364 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1867816832}
I20260709 13:02:44.049417 11014 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:44.075556  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.075664  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.076262  8680 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:44.076434  8676 raft_consensus.cc:2368] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:44.077121 10996 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:44.077230  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.077342  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.078024  8827 raft_consensus.cc:2376] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7. Re-sending same reply.
W20260709 13:02:43.959152 11055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.011s	sys 0.016s
I20260709 13:02:44.042582 10889 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:44.079169 10889 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.944164 11009 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:44.079663 10889 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:44.080340 11009 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.081432 11009 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:44.082163 10889 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:43.882834  8750 raft_consensus.cc:2376] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7. Re-sending same reply.
I20260709 13:02:43.989778  9389 raft_consensus.cc:2410] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:02:44.083349  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:44.083978  8750 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:44.084872  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:44.085525  8750 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:44.070096 11096 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.090866 11101 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:43.988345 10994 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.093698 11101 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:44.046339 10981 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.095036 10994 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:44.095556 10964 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:44.098026 10981 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:44.028829 11107 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.976266  8672 rpcz_store.cc:276] 0709 13:02:42.317520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.334664 (+ 17144us) service_pool.cc:224] Handling call
0709 13:02:42.362305 (+ 27641us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.398419 (+ 36114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.400923 (+  2504us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.500254 (+ 99331us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.686243 (+185989us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:43.026850 (+340607us) log.cc:844] Serialized 14 byte log entry
0709 13:02:43.035174 (+  8324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.035273 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.551643 (+516370us) raft_consensus.cc:1685] finished
0709 13:02:43.563142 (+ 11499us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.576194 (+ 13052us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21353,"prepare.queue_time_us":478791,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":35925,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":8312,"thread_start_us":429614,"threads_started":2,"wal-append.queue_time_us":538806}
I20260709 13:02:43.975831  8821 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:43.881534 10959 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:43.960209 11100 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:44.115295  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 533492us
I20260709 13:02:44.118175  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:44.118849  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 165636us
I20260709 13:02:44.116358  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:44.125056 11101 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:44.125689 11101 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.126181 11101 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:44.128518  8719 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:44.129447  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:44.130170  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:43.891673  8675 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:44.006342  8753 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
W20260709 13:02:44.132490  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 185140us
I20260709 13:02:44.132587  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:02:44.133565  8645 leader_election.cc:336] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.853s (SENT)
I20260709 13:02:44.134099  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:43.991658 11103 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:02:43.947309 11050 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.026340  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:02:44.027415  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.142706  8678 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:44.142825  8826 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
W20260709 13:02:44.143241  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.992810 11099 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:44.033162  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 148403us
I20260709 13:02:44.149540  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:44.034078 10999 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:43.279593 11054 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:44.043170  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:44.045166  8830 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:44.119939 10983 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:44.008919 11007 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:43.648711 10920 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:44.012221 11056 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:44.039237 10969 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:44.177153  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:44.137818 10889 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.051344  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:44.184141  8749 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:44.184978  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:43.882352  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:44.187067  8824 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:44.187564  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1515) took 1214 ms. Trace:
I20260709 13:02:44.187938  8824 rpcz_store.cc:276] 0709 13:02:42.973287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.578474 (+605187us) service_pool.cc:224] Handling call
0709 13:02:44.187494 (+609020us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.189072  8824 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:44.190220  8824 pending_rounds.cc:85] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 6
I20260709 13:02:44.190480  8824 pending_rounds.cc:107] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:02:44.086283  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:44.107038  8680 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:44.069941  9382 raft_consensus.cc:2368] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6.
I20260709 13:02:44.112879  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:43.892370  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.150588 11100 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:44.131029 10871 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:02:44.136778  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:44.148576 11082 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:44.143391  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1522) took 1155 ms. Trace:
I20260709 13:02:44.174454 10920 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.132335 11108 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.147239  8747 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:44.150507  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:44.225814  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 75397.5us
I20260709 13:02:44.180289 11007 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.153518  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1511) took 1181 ms. Trace:
I20260709 13:02:44.047401  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:44.234472  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1940) took 1885 ms. Trace:
I20260709 13:02:44.068742 10583 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:44.139495 11000 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.106s	user 0.014s	sys 0.000s
I20260709 13:02:44.173221  8823 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:44.250322  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:44.251238  8823 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:44.252103 10583 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:44.252254  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:44.158557  9355 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:44.253324  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 94879.3us
I20260709 13:02:44.254051 11007 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:44.174463  8673 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:44.175448  8719 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:44.225503 10920 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:44.255726  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 80377us
I20260709 13:02:44.111061 10824 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.077526 11014 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:43.873807  8681 rpcz_store.cc:276] 0709 13:02:42.332096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.476459 (+144363us) service_pool.cc:224] Handling call
0709 13:02:42.513421 (+ 36962us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.528383 (+ 14962us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.528434 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.558063 (+ 29629us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.642731 (+ 84668us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:42.642778 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.642852 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.535106 (+892254us) raft_consensus.cc:1685] finished
0709 13:02:43.535133 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.535333 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14514,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":32468,"thread_start_us":42500,"threads_started":1,"wal-append.queue_time_us":457000}
W20260709 13:02:44.179212  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 290857us
I20260709 13:02:44.135149  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1521) took 1142 ms. Trace:
I20260709 13:02:44.100358  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:44.265035  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1422) took 1032 ms. Trace:
I20260709 13:02:44.265507  8676 rpcz_store.cc:276] 0709 13:02:43.232102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.078436 (+846334us) service_pool.cc:224] Handling call
0709 13:02:44.264949 (+186513us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.264890  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.266556  8681 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:44.266788  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.267421  8676 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:02:44.267792  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:44.268282  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:44.268814  8676 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:44.268921  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:44.192193  8677 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:44.269582  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1409) took 1060 ms. Trace:
I20260709 13:02:44.270041  8677 rpcz_store.cc:276] 0709 13:02:43.209063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.891996 (+682933us) service_pool.cc:224] Handling call
0709 13:02:44.269519 (+377523us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.271128  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:44.271342  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 227999us
I20260709 13:02:44.271850  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1411) took 1062 ms. Trace:
I20260709 13:02:44.272207  8677 rpcz_store.cc:276] 0709 13:02:43.209725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.270858 (+1061133us) service_pool.cc:224] Handling call
0709 13:02:44.271790 (+   932us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.157172 10999 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.131111  8672 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:44.274595  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 359) took 1071 ms. Trace:
I20260709 13:02:44.199504 10889 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:44.147768  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:44.171010 10964 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:44.203632  8753 raft_consensus.cc:2410] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:44.250161 11008 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:44.171022 10949 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.223675 11082 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.206646  8826 rpcz_store.cc:276] 0709 13:02:42.988242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.989298 (+  1056us) service_pool.cc:224] Handling call
0709 13:02:43.412688 (+423390us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:44.143333 (+730645us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10295680}
I20260709 13:02:44.290807  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:44.291528  8826 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:44.143777  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.292685  8678 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:44.293203  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 361) took 1084 ms. Trace:
I20260709 13:02:44.293509  8678 rpcz_store.cc:276] 0709 13:02:43.208582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.143481 (+934899us) service_pool.cc:224] Handling call
0709 13:02:44.293140 (+149659us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.211367  8749 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:44.211951  8675 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:44.177668 10969 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.171849 10994 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:44.200697 11100 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.215000 10861 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:44.215464 10991 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:44.216387 10876 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:44.218938 11114 consensus_queue.cc:1059] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:44.222249 11116 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:44.151108  8825 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:44.182211 11056 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:44.239275  8679 rpcz_store.cc:276] 0709 13:02:42.349023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.105869 (+756846us) service_pool.cc:224] Handling call
0709 13:02:43.106291 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:43.106639 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:43.106667 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:43.107338 (+   671us) log.cc:844] Serialized 26 byte log entry
0709 13:02:43.483092 (+375754us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:43.483155 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.483252 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.234181 (+750929us) raft_consensus.cc:1685] finished
0709 13:02:44.234205 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.234389 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":375207,"threads_started":1,"wal-append.queue_time_us":542182}
I20260709 13:02:44.286827 11008 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:44.235834  8796 leader_election.cc:400] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6.
I20260709 13:02:44.321978  8796 leader_election.cc:403] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:02:44.247501  8750 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:44.248804  8752 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
W20260709 13:02:44.134382  8645 leader_election.cc:336] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.898s (SENT)
I20260709 13:02:44.272927  8821 rpcz_store.cc:276] 0709 13:02:42.987871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.288691 (+300820us) service_pool.cc:224] Handling call
0709 13:02:44.130259 (+841568us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:44.323267  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 188878us
I20260709 13:02:44.269646  8681 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:44.194095  8824 raft_consensus.cc:2889] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:44.246600 11064 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.974s	user 0.015s	sys 0.000s
I20260709 13:02:44.274976  8672 rpcz_store.cc:276] 0709 13:02:43.203479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.110873 (+907394us) service_pool.cc:224] Handling call
0709 13:02:44.274511 (+163638us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.154420 11099 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.243062 10967 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:44.279397  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.137552  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:44.250485 10983 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.115371 11107 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.254832 10871 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:44.295758  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1421) took 1063 ms. Trace:
I20260709 13:02:44.273466 11054 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:44.277381 10999 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:44.278225 10964 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.232534 11108 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.309219  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:44.298717  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:44.311836  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:44.374207  8678 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:44.312518  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:44.313720  8676 raft_consensus.cc:2410] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:44.376212  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.315048  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:44.377861  8756 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:44.318444  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:44.297253 10949 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.345913 11099 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:44.319355  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 65180.6us
I20260709 13:02:44.405045  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:44.299918 10969 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:44.350749 11116 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:44.332508  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:44.277822 11014 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 13:02:44.337394  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 79298.5us
I20260709 13:02:44.418919  8719 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:44.066234  9380 rpcz_store.cc:276] 0709 13:02:42.376552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.376735 (+   183us) service_pool.cc:224] Handling call
0709 13:02:42.377170 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.377611 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.377640 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.378191 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.650202 (+272011us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:42.674716 (+ 24514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.707508 (+ 32792us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.794932 (+1087424us) raft_consensus.cc:1685] finished
0709 13:02:43.794989 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.819951 (+ 24962us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13839,"raft.queue_time_us":8264,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":9306,"thread_start_us":257800,"threads_started":1,"wal-append.queue_time_us":695406}
W20260709 13:02:44.343856  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 206384us
I20260709 13:02:44.309990 10991 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:44.355005  8675 rpcz_store.cc:276] 0709 13:02:43.231870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.890687 (+658817us) service_pool.cc:224] Handling call
0709 13:02:44.295695 (+405008us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.423666  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.424556  8675 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:02:44.310774 11100 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:44.341158  8645 leader_election.cc:336] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.590s (SENT)
I20260709 13:02:44.367794 10959 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:44.369064 11110 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:02:44.365302 10964 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:44.370728  9381 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:02:44.372885 11108 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:44.379312  8679 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:44.240639  8830 rpcz_store.cc:276] 0709 13:02:42.972187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.898336 (+926149us) service_pool.cc:224] Handling call
0709 13:02:44.153412 (+255076us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.377808  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.448216  8676 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:44.449550  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:44.385946  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.451287  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.361611 11001 consensus_meta.cc:287] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.787s	user 0.009s	sys 0.005s
I20260709 13:02:44.338166 11124 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:44.310945 10994 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.462742 11056 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:44.463578 11056 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.405923  8672 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:44.406756  8750 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:44.354228 10876 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:44.277535 10996 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:02:44.427554 10991 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:44.472033  8823 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:44.411185  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.473786  8678 raft_consensus.cc:2410] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:44.360716 11054 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:44.339313 11064 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.428012  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:44.403918  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:44.378964 11125 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:44.360337 10983 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.453404  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:44.498905 11125 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.499553  8679 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:44.500015 11116 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.501567 11055 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:44.501802  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:44.502169 11055 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:44.487416 11056 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:44.502794 11055 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:44.504344 11056 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:44.504987 11056 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:44.505324 11056 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:44.505684 11056 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:44.502231 10889 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:44.489542  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1412) took 1276 ms. Trace:
I20260709 13:02:44.519863  8673 rpcz_store.cc:276] 0709 13:02:43.212933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.874254 (+661321us) service_pool.cc:224] Handling call
0709 13:02:44.489291 (+615037us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54003cf854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:44.489458 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416194304}
W20260709 13:02:44.493103  8828 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1284) took 2160 ms (client timeout 2088 ms). Trace:
I20260709 13:02:44.495639 10861 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:44.496239  8821 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:02:44.521252  8828 rpcz_store.cc:269] 0709 13:02:42.332754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.332898 (+   144us) service_pool.cc:224] Handling call
0709 13:02:44.492823 (+2159925us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b54003ddbd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:44.492957 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1985102208}
I20260709 13:02:44.524868  8753 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:44.497675  8681 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:44.472231 10824 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:44.442694 11050 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:44.450397  8676 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:44.488267  9389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 380) took 1427 ms. Trace:
I20260709 13:02:44.566874  9389 rpcz_store.cc:276] 0709 13:02:43.060860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.061075 (+   215us) service_pool.cc:224] Handling call
0709 13:02:44.487948 (+1426873us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b5400477a54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:44.488175 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2110023936}
W20260709 13:02:44.465142  8829 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1290) took 2112 ms (client timeout 1865 ms). Trace:
W20260709 13:02:44.568320  8829 rpcz_store.cc:269] 0709 13:02:42.352577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.379494 (+ 26917us) service_pool.cc:224] Handling call
0709 13:02:44.464916 (+2085422us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002297d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:44.465057 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2749485440}
I20260709 13:02:44.566323  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.570786  8672 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:02:44.502794  8678 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:44.552552 11082 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:44.439491 11123 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:44.553047 11056 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:44.556393  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 256487us
W20260709 13:02:44.556591  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.481858 11124 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.463824 10994 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:44.481451 11007 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.522221 11096 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:44.417806 11008 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:44.481317 11054 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:44.508275 10591 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.513s	user 0.011s	sys 0.002s
W20260709 13:02:44.556921  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 151298us
W20260709 13:02:45.000007  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 581177us
W20260709 13:02:45.182204  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 841088us
I20260709 13:02:45.191687  8719 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:44.614496  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.192886  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.907354  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.999850  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:44.491262 11000 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6.
W20260709 13:02:44.670032  8674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 1925) took 2485 ms (client timeout 2120 ms). Trace:
W20260709 13:02:44.999598  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 667146us
W20260709 13:02:45.200464  8674 rpcz_store.cc:269] 0709 13:02:42.184135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.185166 (+  1031us) service_pool.cc:224] Handling call
0709 13:02:44.669719 (+2484553us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54001af9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:44.669922 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3254231296}
I20260709 13:02:45.204263  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:44.799600  8753 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:02:44.640230  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 404550us
W20260709 13:02:45.007387  8754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1598) took 2049 ms (client timeout 1725 ms). Trace:
I20260709 13:02:45.047478  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:45.190332  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:45.055552  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:44.511632 10876 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:02:45.036763  8747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1584) took 2550 ms (client timeout 2047 ms). Trace:
I20260709 13:02:45.009035  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:45.207358  8722 leader_election.cc:336] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.782s (SENT)
W20260709 13:02:45.231405  8747 rpcz_store.cc:269] 0709 13:02:42.413088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.414349 (+  1261us) service_pool.cc:224] Handling call
0709 13:02:44.963627 (+2549278us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b540031bd54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:44.963855 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3823812864}
I20260709 13:02:45.232141  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:45.234463  8826 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:45.234990  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:45.178306 11104 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.161s	user 0.006s	sys 0.001s
I20260709 13:02:45.235054  8830 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:02:45.232499  8754 rpcz_store.cc:269] 0709 13:02:42.459556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.481306 (+ 21750us) service_pool.cc:224] Handling call
0709 13:02:44.508512 (+2027206us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54001581d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:44.508640 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3383034624}
W20260709 13:02:45.239835  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 53695.5us
I20260709 13:02:45.244388  8794 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:44.805629 11009 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.724s	user 0.010s	sys 0.006s
W20260709 13:02:44.970481 11111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.012s	sys 0.009s
W20260709 13:02:45.254894 11111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.014s	sys 0.032s
I20260709 13:02:45.254940  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1299) took 1013 ms. Trace:
I20260709 13:02:45.189497 11125 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:44.927188 11107 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:45.256764 11133 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:45.257273 11133 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.257607 11133 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:45.257783 10983 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:45.258347 11099 consensus_meta.cc:287] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.840s	user 0.010s	sys 0.007s
I20260709 13:02:44.667994  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:45.155411 11118 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:45.253118 10990 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:45.253839 10920 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.979s	user 0.017s	sys 0.003s
W20260709 13:02:45.261545 11100 consensus_meta.cc:287] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.819s	user 0.012s	sys 0.004s
I20260709 13:02:44.809338 11108 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.462s
I20260709 13:02:45.229245 11008 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:45.262863 11064 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:44.809859 11119 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:44.885486 10969 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.521378 10991 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:45.196048 11106 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
W20260709 13:02:45.266145 10964 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.814s	user 0.013s	sys 0.003s
I20260709 13:02:45.267804 11056 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:45.249181 10994 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.198248 11007 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:45.314590 10994 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:45.318161 10994 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:45.280508 11118 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:45.282253 11099 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:44.578919 10999 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:45.293066 11119 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:02:45.345041 11101 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.218s	user 0.020s	sys 0.005s
I20260709 13:02:45.289523 11100 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:45.356815 11101 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:45.362550 11105 raft_consensus.cc:3055] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:45.363125 11105 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:45.363903 11101 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:45.363092 11133 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.114444  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:45.409051  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 216974us
I20260709 13:02:44.641289  8576 catalog_manager.cc:5697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 4, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 4 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:02:45.413092  8645 leader_election.cc:336] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.999s (SENT)
I20260709 13:02:45.414038  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:45.417230  8645 leader_election.cc:336] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 2.040s (SENT)
I20260709 13:02:45.418071  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:45.423512  8647 connection.cc:256] RPC call timeout handler was delayed by -1.4044995149999977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
W20260709 13:02:45.424391  8647 connection.cc:256] RPC call timeout handler was delayed by -1.3618055879999815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16510000000000002s for a response
W20260709 13:02:45.424805  8647 connection.cc:256] RPC call timeout handler was delayed by -1.15425694999999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17s for a response
W20260709 13:02:45.425207  8647 connection.cc:256] RPC call timeout handler was delayed by -1.0072055879999766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20450000000000002s for a response
I20260709 13:02:45.429237  8755 raft_consensus.cc:2410] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:02:45.435290  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 243541us
I20260709 13:02:45.437832  8719 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:45.438284  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2017) took 1884 ms. Trace:
W20260709 13:02:45.441076  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 233625us
I20260709 13:02:45.432160  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 357) took 2229 ms. Trace:
I20260709 13:02:45.440630  8672 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:45.442731  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 324) took 1260 ms. Trace:
I20260709 13:02:45.450788  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1946) took 3061 ms. Trace:
I20260709 13:02:45.451424  8672 rpcz_store.cc:276] 0709 13:02:42.389051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.439228 (+3050177us) service_pool.cc:224] Handling call
0709 13:02:45.440021 (+   793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.450414 (+ 10393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.450709 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.448675  8674 rpcz_store.cc:276] 0709 13:02:43.202870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.201696 (+1998826us) service_pool.cc:224] Handling call
0709 13:02:45.430754 (+229058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.431360 (+   606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.431404 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.431504 (+   100us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:45.431572 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.431667 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.431846 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.255429  8823 rpcz_store.cc:276] 0709 13:02:44.241238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.251970 (+ 10732us) service_pool.cc:224] Handling call
0709 13:02:45.254724 (+1002754us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b5400125f54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:45.254856 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503826304}
W20260709 13:02:45.454511  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 210072us
I20260709 13:02:45.455930  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:45.456537  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:45.457667  8829 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:45.457834  8823 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:45.268584  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 357) took 1124 ms. Trace:
I20260709 13:02:45.277552  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 363) took 1125 ms. Trace:
I20260709 13:02:45.446919  8748 rpcz_store.cc:276] 0709 13:02:43.553924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.142822 (+588898us) service_pool.cc:224] Handling call
0709 13:02:44.171940 (+ 29118us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.187133 (+ 15193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.187166 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.672346 (+485180us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b2c000f0558. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e0df2a11 00007f32e0e02372 00007f32e2534a01 00007f32e24f943a 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2
0709 13:02:44.799676 (+127330us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.175104 (+375428us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:45.175163 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.175275 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.437379 (+262104us) raft_consensus.cc:1685] finished
0709 13:02:45.437422 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.437936 (+   514us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2268,"raft.queue_time_us":47977,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":180,"spinlock_wait_cycles":13014784,"thread_start_us":321084,"threads_started":1,"wal-append.queue_time_us":427233}
I20260709 13:02:45.269953  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:45.459168  8822 rpcz_store.cc:276] 0709 13:02:44.144377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.153484 (+  9107us) service_pool.cc:224] Handling call
0709 13:02:44.179857 (+ 26373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.321363 (+141506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.321394 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.321438 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:44.321471 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.268260 (+946789us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b540032df54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:45.268313 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.268489 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324931328}
I20260709 13:02:45.460456  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 362) took 1314 ms. Trace:
I20260709 13:02:45.460883  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1307) took 1119 ms. Trace:
I20260709 13:02:45.461476  8828 rpcz_store.cc:276] 0709 13:02:44.342179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.266221 (+924042us) service_pool.cc:224] Handling call
0709 13:02:45.460623 (+194402us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.462729  8826 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:45.462877  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.463440  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:45.463871  8674 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:45.459302  8825 rpcz_store.cc:276] 0709 13:02:44.152196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.276327 (+1124131us) service_pool.cc:224] Handling call
0709 13:02:45.276783 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.277058 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.277097 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.277159 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:45.277218 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.277294 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.277463 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.464851  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1418) took 2233 ms. Trace:
I20260709 13:02:45.466173  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 365) took 1287 ms. Trace:
I20260709 13:02:45.465941  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:45.230949  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 360) took 2027 ms. Trace:
I20260709 13:02:45.467360  8825 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:45.467473  8679 rpcz_store.cc:276] 0709 13:02:43.203797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.229716 (+2025919us) service_pool.cc:224] Handling call
0709 13:02:45.230151 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.230400 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.230468 (+    68us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.230526 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:45.230587 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.230689 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.230853 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.458379  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1303) took 1215 ms. Trace:
W20260709 13:02:45.461480  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.445576  8678 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:02:45.425118  8680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 351) took 2224 ms (client timeout 2142 ms). Trace:
I20260709 13:02:45.465449  8674 rpcz_store.cc:276] 0709 13:02:43.231039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.462924 (+2231885us) service_pool.cc:224] Handling call
0709 13:02:45.463541 (+   617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.464451 (+   910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.464765 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.247298  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1301) took 1005 ms. Trace:
I20260709 13:02:45.482595  8830 rpcz_store.cc:276] 0709 13:02:44.241798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.039031 (+797233us) service_pool.cc:224] Handling call
0709 13:02:45.247184 (+208153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.485765 10999 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:45.441059  8576 catalog_manager.cc:5697] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 3 to 4, term changed from 1 to 2, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to <none>, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New cstate: current_term: 2 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:45.263564  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:45.489104  9383 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:45.196062 10591 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:45.364713 11101 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:45.519332 11100 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:45.519152 11101 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:45.520599  9352 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:45.524161  8823 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:45.525238  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1308) took 1182 ms. Trace:
I20260709 13:02:45.525727  8823 rpcz_store.cc:276] 0709 13:02:44.342565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.509319 (+1166754us) service_pool.cc:224] Handling call
0709 13:02:45.523347 (+ 14028us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.524812 (+  1465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.525128 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.528398 11099 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:45.287750 10920 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:45.529874 10876 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:44.833868 10871 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:45.189828 10949 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:44.807901 10965 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.464s
I20260709 13:02:45.485615  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:45.486083  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 354) took 2284 ms. Trace:
W20260709 13:02:45.262187  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 1.05909e+06us
I20260709 13:02:45.491189  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1944) took 3107 ms. Trace:
I20260709 13:02:45.471151  8829 rpcz_store.cc:276] 0709 13:02:44.242425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.454862 (+1212437us) service_pool.cc:224] Handling call
0709 13:02:45.458238 (+  3376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.461840  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:45.471959  8826 rpcz_store.cc:276] 0709 13:02:44.178363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.236373 (+1058010us) service_pool.cc:224] Handling call
0709 13:02:45.251579 (+ 15206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.463318 (+211739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.465716 (+  2398us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b5400267fd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:45.466050 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":484042496}
I20260709 13:02:45.478668  8674 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:45.460976  8824 rpcz_store.cc:276] 0709 13:02:44.145795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.189725 (+ 43930us) service_pool.cc:224] Handling call
0709 13:02:44.190057 (+   332us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:44.555295 (+365238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.555335 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:44.556023 (+   688us) log.cc:844] Serialized 52 byte log entry
0709 13:02:45.186250 (+630227us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b18002e77a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:02:45.459505 (+273255us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:45.459629 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.459749 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.459818 (+    69us) raft_consensus.cc:1685] finished
0709 13:02:45.459827 (+     9us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.460194 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1211,"raft.queue_time_us":159309,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":46066,"spinlock_wait_cycles":363955968,"thread_start_us":498161,"threads_started":2,"wal-append.queue_time_us":815244}
W20260709 13:02:45.505904  8827 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1519) took 2531 ms (client timeout 2045 ms). Trace:
I20260709 13:02:45.289610  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:45.507004 11137 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:45.472838  8680 rpcz_store.cc:269] 0709 13:02:43.200758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.076959 (+876201us) service_pool.cc:224] Handling call
0709 13:02:45.424442 (+1347483us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b54000da2d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:45.424783 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66652544}
I20260709 13:02:45.627599  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1419) took 2396 ms. Trace:
I20260709 13:02:45.627780 10969 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:45.628013  8674 rpcz_store.cc:276] 0709 13:02:43.231323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.477706 (+2246383us) service_pool.cc:224] Handling call
0709 13:02:45.478293 (+   587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.627268 (+148975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.627524 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.630050  8822 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:45.632254  8822 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:45.634007  8829 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:45.634754  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1311) took 1291 ms. Trace:
I20260709 13:02:45.635177  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:45.635143  8829 rpcz_store.cc:276] 0709 13:02:44.343416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.633050 (+1289634us) service_pool.cc:224] Handling call
0709 13:02:45.633513 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.634423 (+   910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.634667 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:45.515193 11055 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.011s	user 0.013s	sys 0.008s
I20260709 13:02:45.364307 11105 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:45.637322 11055 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:45.637907 11105 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:45.502556  9384 raft_consensus.cc:2410] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:02:45.640991  9384 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 590) took 2160 ms (client timeout 1877 ms). Trace:
W20260709 13:02:45.641602  9384 rpcz_store.cc:269] 0709 13:02:43.480451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.483306 (+  2855us) service_pool.cc:224] Handling call
0709 13:02:44.667715 (+1184409us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:45.640833 (+973118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4927232}
W20260709 13:02:45.622318  8827 rpcz_store.cc:269] 0709 13:02:42.974536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.261202 (+286666us) service_pool.cc:224] Handling call
0709 13:02:45.505192 (+2243990us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b540016ba54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:45.505520 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":457934208}
I20260709 13:02:45.620793  8678 rpcz_store.cc:276] 0709 13:02:42.383503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.444243 (+3060740us) service_pool.cc:224] Handling call
0709 13:02:45.444961 (+   718us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.472238 (+ 27277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.491096 (+ 18858us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.644495  9383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 478) took 1327 ms. Trace:
I20260709 13:02:45.644944  9383 rpcz_store.cc:276] 0709 13:02:44.316840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.319791 (+  2951us) service_pool.cc:224] Handling call
0709 13:02:44.360870 (+ 41079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.489765 (+1128895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.644188 (+154423us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54006275d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:45.644398 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2587689856}
I20260709 13:02:45.645567  8678 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:45.448869  8756 rpcz_store.cc:276] 0709 13:02:44.181918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.524561 (+342643us) service_pool.cc:224] Handling call
0709 13:02:44.586590 (+ 62029us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.587103 (+   513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.587145 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.000377 (+413232us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.441945 (+441568us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:45.442044 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.442207 (+   163us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.442253 (+    46us) raft_consensus.cc:1685] finished
0709 13:02:45.442268 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.442500 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":281746,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":13227,"thread_start_us":440920,"threads_started":1,"wal-append.queue_time_us":372324}
I20260709 13:02:45.548110 10991 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:45.489854  8576 catalog_manager.cc:5697] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 3, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 3 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:45.561057  8751 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:45.562721 11124 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:45.563867 11153 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:45.564769  8794 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:45.565974  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 355) took 2269 ms. Trace:
I20260709 13:02:45.360370 11119 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:45.568831 11007 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:45.353205 11118 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:45.652397 11118 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:45.197175 11123 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:45.569583  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:45.656950  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 325) took 1474 ms. Trace:
I20260709 13:02:45.657130 11123 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:45.657457  8750 rpcz_store.cc:276] 0709 13:02:44.182193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.557165 (+374972us) service_pool.cc:224] Handling call
0709 13:02:44.999653 (+442488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.655636 (+655983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.655684 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.655749 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:45.655788 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.656509 (+   721us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b54002ac454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:45.656554 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.656752 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1507584384}
I20260709 13:02:45.569859  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:45.658735  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 164172us
I20260709 13:02:45.571775  8823 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:45.659884  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1310) took 1316 ms. Trace:
I20260709 13:02:45.572665 11101 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:45.573480 11003 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:45.660465  8647 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.045s (SENT)
I20260709 13:02:45.661017  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:45.660324  8823 rpcz_store.cc:276] 0709 13:02:44.343179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.570296 (+1227117us) service_pool.cc:224] Handling call
0709 13:02:45.571075 (+   779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.659560 (+ 88485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.659820 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.662320 10983 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:45.576059  8672 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 6. Ops: []
I20260709 13:02:45.663002 10983 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:45.509038 11121 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:45.664678 10983 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:45.664702 11121 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:45.667062 11001 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:45.648489  8673 rpcz_store.cc:276] 0709 13:02:43.202177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.428415 (+2226238us) service_pool.cc:224] Handling call
0709 13:02:45.429427 (+  1012us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.430038 (+   611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.430086 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.430195 (+   109us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:45.430266 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.430349 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.430600 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.668540 11001 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:45.669070 11001 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.669633 11001 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:45.670821 11121 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:45.675122  8647 leader_election.cc:336] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.623s (SENT)
I20260709 13:02:45.677896 11119 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:45.370512 11133 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:45.596552  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 76396us
I20260709 13:02:45.580868 10994 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:45.608069  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1306) took 1271 ms. Trace:
I20260709 13:02:45.695807  8830 rpcz_store.cc:276] 0709 13:02:44.333648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.483867 (+1150219us) service_pool.cc:224] Handling call
0709 13:02:45.484557 (+   690us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.601664 (+117107us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.601718 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.601784 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:45.601848 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.601967 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.604575 (+  2608us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102053,"raft.queue_time_us":118494,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":15479}
I20260709 13:02:45.611805  8677 rpcz_store.cc:276] 0709 13:02:43.201845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.431707 (+2229862us) service_pool.cc:224] Handling call
0709 13:02:45.432369 (+   662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.485471 (+ 53102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.485525 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.485603 (+    78us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:45.485646 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.485712 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.485840 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.697305 10920 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:45.698650  8677 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:45.624013 10964 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:45.624779 10871 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:45.625631  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 320) took 1856 ms. Trace:
I20260709 13:02:45.626106 10917 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:45.626714  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 189220us
I20260709 13:02:45.626384  9353 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:45.630112  8826 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:45.646543  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1424) took 2414 ms. Trace:
I20260709 13:02:45.705003  8678 rpcz_store.cc:276] 0709 13:02:43.232562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.644614 (+2412052us) service_pool.cc:224] Handling call
0709 13:02:45.645047 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.646166 (+  1119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.646441 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.649523  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1313) took 1303 ms. Trace:
I20260709 13:02:45.679704  9355 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:45.683887  8755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 316) took 2034 ms (client timeout 2062 ms). Trace:
I20260709 13:02:45.706213  8822 rpcz_store.cc:276] 0709 13:02:44.345729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.631065 (+1285336us) service_pool.cc:224] Handling call
0709 13:02:45.631607 (+   542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.632691 (+  1084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.649460 (+ 16769us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:45.560616  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 112895us
I20260709 13:02:45.691464  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1626) took 1808 ms. Trace:
I20260709 13:02:45.712548  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1425) took 2479 ms. Trace:
W20260709 13:02:45.706554  8755 rpcz_store.cc:269] 0709 13:02:43.649194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.041071 (+391877us) service_pool.cc:224] Handling call
0709 13:02:45.683475 (+1642404us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400157cd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:45.683784 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2931801472}
I20260709 13:02:45.717552  8677 rpcz_store.cc:276] 0709 13:02:43.232799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.697625 (+2464826us) service_pool.cc:224] Handling call
0709 13:02:45.698271 (+   646us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.708963 (+ 10692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.712470 (+  3507us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.668944 11127 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:45.708374  8751 rpcz_store.cc:276] 0709 13:02:43.883519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.112116 (+228597us) service_pool.cc:224] Handling call
0709 13:02:45.686506 (+1574390us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540031b0d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:45.691334 (+  4828us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3083974912}
I20260709 13:02:45.721552  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1312) took 1377 ms. Trace:
I20260709 13:02:45.723001  8826 rpcz_store.cc:276] 0709 13:02:44.343690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.629134 (+1285444us) service_pool.cc:224] Handling call
0709 13:02:45.629695 (+   561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.713008 (+ 83313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.721477 (+  8469us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.704173  8753 rpcz_store.cc:276] 0709 13:02:43.755528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.281029 (+525501us) service_pool.cc:224] Handling call
0709 13:02:44.524693 (+243664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.230933 (+706240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.599013 (+368080us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b540018ced0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:45.611866 (+ 12853us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559530240}
I20260709 13:02:45.699564 11055 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:45.733137  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:02:45.734239  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 108207us
I20260709 13:02:45.737375 11096 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:45.737977 11096 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.738374 11096 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:45.739478  8678 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:45.740298  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1426) took 2507 ms. Trace:
I20260709 13:02:45.740672  8678 rpcz_store.cc:276] 0709 13:02:43.233023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.712844 (+2479821us) service_pool.cc:224] Handling call
0709 13:02:45.739117 (+ 26273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.739976 (+   859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.740216 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.745087  8576 catalog_manager.cc:5697] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 4, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 4 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:45.748965  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1416) took 2518 ms. Trace:
I20260709 13:02:45.749475  8672 rpcz_store.cc:276] 0709 13:02:43.230367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.453458 (+2223091us) service_pool.cc:224] Handling call
0709 13:02:45.454179 (+   721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.662954 (+208775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.748642 (+ 85688us) spinlock_profiling.cc:243] Waited 80.2 ms on lock 0x7b54002409d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:45.748851 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184537856}
I20260709 13:02:45.709344 10917 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:45.751093  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 71684.2us
I20260709 13:02:45.751355 11050 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:45.752031  9355 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:45.752811 10943 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:45.753404 10943 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.753855 10943 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:45.755194 10965 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:45.755790 10965 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.755505 11132 consensus_queue.cc:1059] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:45.756289 10965 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:45.756879  8828 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:45.757143 11132 raft_consensus.cc:3055] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:45.757658 11132 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:45.758970 11132 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:45.760872 11132 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:45.750916 10917 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:45.765146  8677 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:45.766692  8680 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:45.767586  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 365) took 1589 ms. Trace:
I20260709 13:02:45.768019  8680 rpcz_store.cc:276] 0709 13:02:44.178126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.765684 (+1587558us) service_pool.cc:224] Handling call
0709 13:02:45.766168 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.767291 (+  1123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.767518 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.770229  8680 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:45.722503 11127 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:45.773465  8674 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:45.776155  8823 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:45.779378  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 364) took 1607 ms. Trace:
I20260709 13:02:45.752043  8673 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
W20260709 13:02:45.780946  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 47981.4us
I20260709 13:02:45.781828 11055 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:45.766294  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1427) took 2533 ms. Trace:
I20260709 13:02:45.827584  8677 rpcz_store.cc:276] 0709 13:02:43.233266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.764132 (+2530866us) service_pool.cc:224] Handling call
0709 13:02:45.764798 (+   666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.765865 (+  1067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.766191 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.754141  9355 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:45.828423  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1420) took 2596 ms. Trace:
W20260709 13:02:45.793084  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 71832.9us
I20260709 13:02:45.750239 10964 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:45.801893  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1297) took 1544 ms. Trace:
I20260709 13:02:45.803550  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 363) took 1626 ms. Trace:
I20260709 13:02:45.832084  8672 rpcz_store.cc:276] 0709 13:02:44.177571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.750470 (+1572899us) service_pool.cc:224] Handling call
0709 13:02:45.802658 (+ 52188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.803044 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.803079 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.803154 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:45.803213 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.803288 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.803470 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.832645  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1430) took 1654 ms. Trace:
W20260709 13:02:45.742256  8575 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43478 (request call id 254) took 19353 ms (client timeout 15000 ms). Trace:
I20260709 13:02:45.811491  8675 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:45.814530  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.833056  8674 rpcz_store.cc:276] 0709 13:02:44.187069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.772487 (+1585418us) service_pool.cc:224] Handling call
0709 13:02:45.773166 (+   679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.829396 (+ 56230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.832575 (+  3179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.835172 11127 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:45.837399 11127 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:45.840077  8677 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:45.841004  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1434) took 1652 ms. Trace:
I20260709 13:02:45.841384  8677 rpcz_store.cc:276] 0709 13:02:44.188159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.838936 (+1650777us) service_pool.cc:224] Handling call
0709 13:02:45.839533 (+   597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.840603 (+  1070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.840890 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.844684  8672 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:45.845887  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1441) took 1583 ms. Trace:
I20260709 13:02:45.846894  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 364) took 1653 ms. Trace:
I20260709 13:02:45.847450  8673 rpcz_store.cc:276] 0709 13:02:44.177852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.751366 (+1573514us) service_pool.cc:224] Handling call
0709 13:02:45.751787 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.829976 (+ 78189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.831049 (+  1073us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.849807  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:45.851470 11056 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.852361 11056 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:45.849531  8673 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:45.855489  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1442) took 1592 ms. Trace:
I20260709 13:02:45.831317  8828 rpcz_store.cc:276] 0709 13:02:44.240547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.469017 (+1228470us) service_pool.cc:224] Handling call
0709 13:02:45.469839 (+   822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.757373 (+287534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.784340 (+ 26967us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b540009d3d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:45.784594 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":601941248}
I20260709 13:02:45.855947  8673 rpcz_store.cc:276] 0709 13:02:44.263286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.848485 (+1585199us) service_pool.cc:224] Handling call
0709 13:02:45.849022 (+   537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.855064 (+  6042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.855360 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:45.857059  9353 connection.cc:256] RPC call timeout handler was delayed by -1.1458149670000353s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2062s for a response
I20260709 13:02:45.846366  8672 rpcz_store.cc:276] 0709 13:02:44.262918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.843647 (+1580729us) service_pool.cc:224] Handling call
0709 13:02:45.844309 (+   662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.845311 (+  1002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.845800 (+   489us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:45.816768  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 331439us
I20260709 13:02:45.792258  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 362) took 1831 ms. Trace:
W20260709 13:02:45.846786  8793 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.102s (SENT)
I20260709 13:02:45.803844  8576 catalog_manager.cc:5697] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 4, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 4 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:45.816109  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 120062us
I20260709 13:02:45.859153  8678 rpcz_store.cc:276] 0709 13:02:43.960675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.741662 (+1780987us) service_pool.cc:224] Handling call
0709 13:02:45.742149 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.742545 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.742571 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.791688 (+ 49117us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:45.791811 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.791941 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.792153 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.825059  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1429) took 1646 ms. Trace:
I20260709 13:02:45.861260  8680 rpcz_store.cc:276] 0709 13:02:44.178401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.769164 (+1590763us) service_pool.cc:224] Handling call
0709 13:02:45.769719 (+   555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.818879 (+ 49160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.824988 (+  6109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.866202 11138 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:45.867449 11138 consensus_queue.cc:1243] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:02:45.867883 11054 consensus_meta.cc:287] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.639s	user 0.005s	sys 0.018s
I20260709 13:02:45.868075 11138 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:45.868539 11138 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:45.868438 11054 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:45.869803 11138 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:45.826401  8821 rpcz_store.cc:276] 0709 13:02:44.171560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.638856 (+1467296us) service_pool.cc:224] Handling call
0709 13:02:45.777841 (+138985us) spinlock_profiling.cc:243] Waited 48.1 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:02:45.778213 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.778662 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.778701 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.778786 (+    85us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:45.778851 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.778946 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.779142 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110680832}
I20260709 13:02:45.873394  8793 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:45.880489 11056 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:45.881618  8793 leader_election.cc:336] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.120s (SENT)
I20260709 13:02:45.882858 10964 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:45.882246  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:45.885530  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.886281  8677 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:45.886473 10964 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:45.887812  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.890393  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:45.893150 11138 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:45.896931  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1947) took 3507 ms. Trace:
I20260709 13:02:45.897490  8681 rpcz_store.cc:276] 0709 13:02:42.389433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.433639 (+3044206us) service_pool.cc:224] Handling call
0709 13:02:45.434382 (+   743us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.435239 (+   857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.435268 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.436267 (+   999us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.559972 (+123705us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:45.560038 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.560143 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.896463 (+336320us) raft_consensus.cc:1685] finished
0709 13:02:45.896494 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.896687 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":67167,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":298,"thread_start_us":109227,"threads_started":1,"wal-append.queue_time_us":292599}
W20260709 13:02:45.898954  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.900141  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:45.887270  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1438) took 1685 ms. Trace:
I20260709 13:02:45.906209  8677 rpcz_store.cc:276] 0709 13:02:44.201485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.842351 (+1640866us) service_pool.cc:224] Handling call
0709 13:02:45.872574 (+ 30223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.886936 (+ 14362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.887196 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:45.829782 11055 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:45.913374  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 159437us
W20260709 13:02:45.910831  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 61116.5us
I20260709 13:02:45.908324  8576 catalog_manager.cc:5697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 2 to 4, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 4 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:45.829581  8679 rpcz_store.cc:276] 0709 13:02:43.231620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.640073 (+2408453us) service_pool.cc:224] Handling call
0709 13:02:45.640498 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.640943 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:45.641377 (+   434us) log.cc:844] Serialized 14 byte log entry
0709 13:02:45.827860 (+186483us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.827944 (+    84us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:45.828032 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.828135 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.828324 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":775,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":12412,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":135,"thread_start_us":177252,"threads_started":1}
W20260709 13:02:45.923884  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.924157 11116 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:45.924985  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.925602  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:45.926199  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.926746 10964 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.924225  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:45.927395 10964 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:45.924649  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:45.924866 11116 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:45.929152 11116 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:45.930265 10964 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:45.939527  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:45.931629  8576 catalog_manager.cc:5697] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 3, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 3 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:45.940063  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:45.944896  8753 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:45.940629  8719 leader_election.cc:400] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3.
I20260709 13:02:45.940568  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:45.941015  9380 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:45.941174 11099 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:45.833199  8575 rpcz_store.cc:269] 0709 13:02:26.392967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.393168 (+   201us) service_pool.cc:224] Handling call
0709 13:02:45.741897 (+19348729us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:39.720012 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:39.720709 (+   697us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:39.720755 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:02:39.720783 (+    28us) tablet.cc:662] Decoding operations
0709 13:02:39.735819 (+ 15036us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:39.735888 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:39.735915 (+    27us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:02:39.736625 (+   710us) tablet.cc:701] Row locks acquired
0709 13:02:39.736643 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:02:39.736829 (+   186us) write_op.cc:270] Start()
0709 13:02:39.736963 (+   134us) write_op.cc:276] Timestamp: P: 1783602159736768 usec, L: 0
0709 13:02:39.736980 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:02:39.751922 (+ 14942us) log.cc:844] Serialized 3219 byte log entry
0709 13:02:40.680795 (+928873us) op_driver.cc:464] REPLICATION: finished
0709 13:02:40.780749 (+ 99954us) write_op.cc:301] APPLY: starting
0709 13:02:40.780892 (+   143us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:40.781193 (+   301us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:40.781384 (+   191us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:40.782410 (+  1026us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:40.782694 (+   284us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:02:40.782745 (+    51us) write_op.cc:312] APPLY: finished
0709 13:02:40.783067 (+   322us) log.cc:844] Serialized 65 byte log entry
0709 13:02:40.797340 (+ 14273us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:40.797547 (+   207us) write_op.cc:454] Released schema lock
0709 13:02:40.797618 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:40.918533 (+120915us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:40.929321 (+ 10788us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1110349,"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":98163,"apply.run_cpu_time_us":3503,"apply.run_wall_time_us":39831,"num_ops":8,"prepare.queue_time_us":448091,"prepare.run_cpu_time_us":40170,"prepare.run_wall_time_us":354985,"raft.queue_time_us":272941,"raft.run_cpu_time_us":10012,"raft.run_wall_time_us":149765,"replication_time_us":913296,"thread_start_us":940581,"threads_started":4,"wal-append.queue_time_us":389467,"wal-append.run_cpu_time_us":59451,"wal-append.run_wall_time_us":1642738}]]}
I20260709 13:02:45.914256 11055 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:45.948567  8755 raft_consensus.cc:2368] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6.
I20260709 13:02:45.952143 10943 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:45.953477  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:45.975492  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.937227 11054 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:45.941905 10876 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:45.974947  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:45.983115  9387 raft_consensus.cc:2410] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
W20260709 13:02:45.984707  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 84801.3us
I20260709 13:02:45.999634  9352 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:46.003832  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 356) took 2796 ms. Trace:
I20260709 13:02:46.004470  8675 rpcz_store.cc:276] 0709 13:02:43.202492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.194933 (+1992441us) service_pool.cc:224] Handling call
0709 13:02:45.195404 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.875356 (+679952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.990123 (+114767us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54001ff9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:45.998507 (+  8384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1412961024}
I20260709 13:02:46.001369 11003 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.008206 11003 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:45.984440 11124 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.005527  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:46.011440 11008 consensus_meta.cc:287] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.718s	user 0.017s	sys 0.005s
W20260709 13:02:46.003754  8796 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.009s	sys 0.004s
W20260709 13:02:46.015355  8796 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.009s	sys 0.005s
W20260709 13:02:46.015700  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 546155us
I20260709 13:02:46.017526  8645 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:46.014730 11152 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:46.011039 11124 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:46.019167 11003 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:46.014559 11108 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:46.020042 11108 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:46.020519 11108 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:46.020793 11124 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:46.021508 11100 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:46.022207  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 160032us
I20260709 13:02:46.023188  8719 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:46.006784  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:46.027499  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:46.033878  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:46.036700  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:46.037732  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:46.039083  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:46.039988  8828 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:46.043084  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:46.043524  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:46.044210  9388 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:46.044304  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.045301  8829 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:46.044862 11108 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:46.050217  9383 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:46.050868  8755 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:46.043263  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.053592  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:46.054353  8821 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:46.055135  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:46.055938 10969 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.053622 11008 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:02:46.056981 11014 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.057422  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:46.055962  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.037772  8750 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:46.032869  8756 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:02:46.058907 11142 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.059270  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.059894 11108 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:46.060491  8830 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:46.053189  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:46.066942  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:46.067481  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 48273.9us
I20260709 13:02:46.073217  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:46.072546  8827 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:46.069847  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 62520us
I20260709 13:02:46.071806  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:46.096766  9387 raft_consensus.cc:2393] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:46.098658  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.063607  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:46.132920  8826 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:02:46.133870  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 46273.2us
I20260709 13:02:46.146384  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:46.130434  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.132019  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:46.163540  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:46.155695  8751 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:46.094655 11118 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:02:46.171326  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 157368us
W20260709 13:02:46.183319  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.185914 10967 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.186695 10967 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:46.187712 10943 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:46.189278 10967 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:46.189392  9383 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:46.129073 11121 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:46.191874  8753 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:02:46.194187  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 69075.2us
I20260709 13:02:46.195190  8722 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:46.134435 11014 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:46.198046  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.200395  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:46.201371  8748 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:46.201208  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:46.206022  8754 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:46.207651 11174 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:02:46.128476 11119 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:46.209273 11119 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:46.211712 11119 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:46.211824  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:46.212586  8755 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:46.222020  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1943) took 3838 ms. Trace:
I20260709 13:02:46.222565  8676 rpcz_store.cc:276] 0709 13:02:42.383185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.446732 (+3063547us) service_pool.cc:224] Handling call
0709 13:02:45.447409 (+   677us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.588470 (+141061us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.588504 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.616676 (+ 28172us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.963351 (+346675us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:45.963426 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.963544 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.221528 (+257984us) raft_consensus.cc:1685] finished
0709 13:02:46.221554 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.221764 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":60134,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":165,"thread_start_us":380206,"threads_started":2,"wal-append.queue_time_us":499756}
I20260709 13:02:46.146387 11008 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.225481  8756 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:46.226181 11008 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:46.136909 11096 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:46.228775 11153 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:46.164994 11142 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:46.228876 11008 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:46.231117  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:46.231731 11001 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.561s	user 0.013s	sys 0.005s
I20260709 13:02:46.232708  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.232317 11001 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:46.233426  8821 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:46.234925  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 210163us
I20260709 13:02:46.235939  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:46.236728  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:46.237478 11054 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:46.237941 11054 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.238276 11054 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:46.238432  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:46.238837 11003 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:46.239239 11003 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.239601 11003 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:46.235198  8825 raft_consensus.cc:2368] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5.
I20260709 13:02:46.240032 10917 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:46.240059  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.247967  8829 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:46.242856  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 45879.5us
I20260709 13:02:46.266752  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:46.267957  9388 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:46.276921  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:46.277626  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.278561  8826 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:46.134577 10969 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:46.174898 11118 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:02:46.293627  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 92702.2us
I20260709 13:02:46.277987  8828 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:02:46.230448 11096 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:46.285285  8825 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:46.310935  8750 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:02:46.314145  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 78317.5us
W20260709 13:02:46.294826 10965 consensus_meta.cc:287] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.005s
I20260709 13:02:46.284773 11138 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:46.328569  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:46.329252  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.320603  8823 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
W20260709 13:02:46.321611  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 45099.9us
I20260709 13:02:46.343410  9383 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:46.318935 11106 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:02:46.349360  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:46.283483 11119 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
W20260709 13:02:46.360731  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.327322 11138 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:46.391230  9383 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:46.349844  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:46.401093  8824 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:02:46.403172  9381 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:46.386286 11138 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:46.386581 11105 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.748s	user 0.012s	sys 0.009s
I20260709 13:02:46.380125 11121 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:02:46.452602 10969 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:46.461354  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.469719  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 343867us
W20260709 13:02:46.473986  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.489883  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 114620us
W20260709 13:02:46.491946  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.340999 10965 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:46.504303  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:46.505707  8823 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:46.507144  8719 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:46.396633 11014 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:46.522132  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 52253.4us
W20260709 13:02:46.494437 11132 consensus_meta.cc:287] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.733s	user 0.023s	sys 0.007s
I20260709 13:02:46.400040 11055 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:46.549997  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.551594  8823 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:46.550791  8755 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:46.553201  8827 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:46.554746  9380 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:46.556689 11138 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:46.535357 11121 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
W20260709 13:02:46.563292  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 208783us
W20260709 13:02:46.564858  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.567919  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.572657  8829 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
W20260709 13:02:46.563547 11127 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.725s	user 0.016s	sys 0.003s
I20260709 13:02:46.523808 11096 consensus_queue.cc:1059] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:02:46.586750  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:46.657709  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:46.723769  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:46.656028  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:46.639941  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.733985  8829 raft_consensus.cc:2376] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5. Re-sending same reply.
W20260709 13:02:46.719002  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 454432us
I20260709 13:02:46.655018  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:46.728675  8826 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:02:46.663937  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:46.662539 11138 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:46.649442 10965 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:46.635016  8830 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
W20260709 13:02:46.639117  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 132115us
I20260709 13:02:46.489037 11118 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:46.547070 10917 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:46.688788  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 292528us
I20260709 13:02:46.759003  8672 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:46.620893  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:46.656937  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:46.753384  8680 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:46.755244  8827 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:46.717437 11119 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:46.758008  8675 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:46.772146  8679 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:02:46.687651 11109 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.289s
I20260709 13:02:46.774690  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.771358  8719 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:46.780007  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.781427  8828 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:46.785485  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 164684us
I20260709 13:02:46.787983  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.788940  8827 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 4: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:46.789168  8676 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:46.789892  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.791988  8827 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:02:46.793056  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.778230 11118 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:46.799372  8821 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.565s	user 0.016s	sys 0.000s
I20260709 13:02:46.799922  8821 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
W20260709 13:02:46.775835 10969 consensus_queue.cc:1175] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7aff04f53d4845a6a4fa491fd2a06145" responder_term: 7 status { last_received { term: 6 index: 6 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:46.761919 11119 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:46.811656  8673 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:02:46.805858  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.778270 11105 raft_consensus.cc:3055] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:02:46.817219  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.822817  8829 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
W20260709 13:02:46.824065  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 160759us
I20260709 13:02:46.828811 11105 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:46.807219  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:46.820806  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:46.813206 11185 log.cc:927] Time spent T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
W20260709 13:02:46.814836 11054 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.526s	user 0.013s	sys 0.000s
I20260709 13:02:46.830413 11119 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:46.852146  8680 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:46.848718  8823 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:46.850409  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:46.853876  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:46.811995 11055 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:46.856442  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:46.853909  8824 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:46.834242 11187 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
W20260709 13:02:46.858759  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 51664.1us
W20260709 13:02:46.859495  8751 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.022s	sys 0.025s
I20260709 13:02:46.863569  8823 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:46.790513 11180 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
W20260709 13:02:46.867820  8751 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.022s	sys 0.025s
I20260709 13:02:46.871263  8821 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:46.844190 10876 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:46.813308 11109 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:02:46.876900  8679 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:46.884752  8719 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:46.885696  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.887427  8823 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:46.891678  8828 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:46.851089 11054 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:46.872705 11194 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:46.886245 11096 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:02:46.888024 10917 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
W20260709 13:02:46.945413  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 149070us
W20260709 13:02:46.905716 11003 consensus_meta.cc:287] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.666s	user 0.014s	sys 0.003s
I20260709 13:02:46.910694 11109 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:02:46.926218 11193 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.068s	user 0.004s	sys 0.000s
W20260709 13:02:46.917277 11116 consensus_meta.cc:287] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.987s	user 0.008s	sys 0.008s
I20260709 13:02:46.899279 11056 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:46.961882  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.898276 11055 consensus_queue.cc:1059] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:02:46.910327 11054 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:46.906754  9384 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:02:46.970533  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 99553us
I20260709 13:02:46.957333 11109 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:46.972947 11116 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:46.896179 10876 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:46.998754 10876 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:46.999548 10876 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:47.000119 10876 raft_consensus.cc:697] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:46.996567  8645 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:46.961823 11003 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:47.017246  9387 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:46.983511 11195 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:46.948930 11105 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:47.023342  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 175209us
I20260709 13:02:46.993033 11142 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:47.031265  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 155985us
W20260709 13:02:47.040380  8752 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.018s	sys 0.025s
W20260709 13:02:47.040804  8752 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.018s	sys 0.025s
I20260709 13:02:47.043460  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:47.044600  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 159946us
I20260709 13:02:47.047453  9355 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:47.055290  8672 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:47.047909  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:47.028053 11056 raft_consensus.cc:3055] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:02:47.093945  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 74973.1us
I20260709 13:02:47.096093  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:47.086011 11056 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:47.022002 10876 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:47.096089 11109 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:47.109956 11197 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:47.110657 11197 raft_consensus.cc:697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:47.110802 11109 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:47.111678 11197 consensus_queue.cc:237] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:47.109449 11195 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:47.115243 11195 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.115795 11195 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:47.114719  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.120149  8719 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.104933 11187 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:47.130076 11055 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:47.130666 11196 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 13:02:47.134104  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.134941 11056 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:47.135843 11134 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:47.136304 11134 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:47.136417 11056 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:47.137401 11134 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:47.140195  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:47.128476  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:47.142598  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.152725  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 156229us
I20260709 13:02:47.154685 11205 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:47.154958  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:47.155965 11205 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:47.156582  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:47.157960  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:47.154677  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:02:47.162313  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.146855 11134 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.164527  8754 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.165184  8755 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:47.157830 11205 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:47.165477  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:47.166057  8722 leader_election.cc:400] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5.
I20260709 13:02:47.166201  8753 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:47.166309 11116 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:47.166689  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:47.174885  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:47.175849  8750 raft_consensus.cc:2410] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:47.176084  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:47.176824  8748 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:47.167284  8756 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.168804  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.165025  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:47.172363  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 48257.6us
I20260709 13:02:47.173372 11003 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:47.172621  8722 leader_election.cc:403] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:47.185781 11054 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:47.184123 11187 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:47.187563 11187 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:47.188712  8749 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:47.190639  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:47.193064  9353 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.196938  9353 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:47.179302 11205 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:47.201578  8755 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:02:47.194826 11054 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:47.198057 11205 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:47.203436 11204 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:47.211243 11132 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:47.216557  8753 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:47.218181  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:47.220660 11001 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:47.226298 11200 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:47.232558  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 66610.8us
I20260709 13:02:47.230276  8756 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
W20260709 13:02:47.282584  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 47267.3us
I20260709 13:02:47.221311 11119 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:47.287496 11215 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:02:47.288826  8821 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
W20260709 13:02:47.289966 11206 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:47.236615 11138 raft_consensus.cc:3055] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:47.290958 11138 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:47.292081 11138 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:47.294742  8722 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:47.239100 11197 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5.
I20260709 13:02:47.299329  8753 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:47.299429  8755 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:47.225528 11054 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:47.302727 11213 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:47.303841 11200 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:47.304311 11200 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:47.304718 11200 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:47.305192  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.293567 11138 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.307957  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:47.324188  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 106105us
I20260709 13:02:47.234789 11207 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:02:47.325904  8748 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
W20260709 13:02:47.328321  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 131500us
I20260709 13:02:47.332264  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:47.339145  9387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 387) took 1189 ms. Trace:
I20260709 13:02:47.339967  9387 rpcz_store.cc:276] 0709 13:02:46.152874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.163769 (+ 10895us) service_pool.cc:224] Handling call
0709 13:02:46.196195 (+ 32426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.022905 (+826710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.338842 (+315937us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b540061c1d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:47.339078 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1877483904}
I20260709 13:02:47.235273 11003 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.101s
I20260709 13:02:47.361268  8645 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.336534 10876 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:47.364529  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 69930us
I20260709 13:02:47.336356 11210 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.351109 11119 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:47.376266  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:47.357479 10943 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.387023 10943 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:47.389467 10943 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:47.372617 11109 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:02:47.360620 11204 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.405107  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:47.378674  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:47.405718  8754 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:47.406184  8828 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:47.409211  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:47.368384 10876 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:02:47.413619  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 81492.7us
I20260709 13:02:47.414207  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:47.416884  8821 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:47.334190 11213 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:02:47.419273  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:47.409384  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:47.420662  8825 raft_consensus.cc:2376] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4. Re-sending same reply.
I20260709 13:02:47.423629  8823 raft_consensus.cc:2410] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:02:47.438105  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 96782.2us
I20260709 13:02:47.419374 10876 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.455360  8747 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:47.455142  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:47.464529  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:47.472117  8823 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:02:47.478539  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 135193us
I20260709 13:02:47.416175 11204 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:47.495198  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1536) took 1432 ms. Trace:
I20260709 13:02:47.495967  8830 rpcz_store.cc:276] 0709 13:02:46.057646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.236421 (+178775us) service_pool.cc:224] Handling call
0709 13:02:47.482212 (+1245791us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b54002189d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:47.489751 (+  7539us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":914143616}
I20260709 13:02:47.492050  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.497507  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:47.500102  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.464419 11210 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:47.500588  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 81465.6us
I20260709 13:02:47.505460  8647 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:47.473238 11209 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:47.507514 11216 log.cc:927] Time spent T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Append to log took a long time: real 0.248s	user 0.003s	sys 0.000s
I20260709 13:02:47.508358  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2038) took 1485 ms. Trace:
I20260709 13:02:47.507599  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:47.510238  8825 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:47.427271 11221 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.074s
W20260709 13:02:47.510003  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.512682  9352 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:47.471123 11138 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:02:47.520052  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:47.520911  8681 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:47.520249  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:47.501667 11225 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:47.521840  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:47.526410  8830 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:47.528563  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.518535 11109 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:47.530228  8823 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:47.537077 11205 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:47.530870 11096 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.541577 11138 consensus_queue.cc:1243] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:02:47.542232 11226 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:47.542666 11226 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:47.543781 11213 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:47.543792 11226 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:47.545557  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.545708 11226 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:47.534759 11209 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:47.535094 11225 raft_consensus.cc:697] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:47.585723 11225 consensus_queue.cc:237] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:47.587726  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:47.588091 11229 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.588617  8681 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 3: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:02:47.536027 11217 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:02:47.544628 11213 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:47.515250 11124 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:47.555964  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:47.558425  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 45818.6us
I20260709 13:02:47.559865 11195 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:47.508742  8752 rpcz_store.cc:276] 0709 13:02:46.022751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.024670 (+  1919us) service_pool.cc:224] Handling call
0709 13:02:46.025141 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.025471 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:46.281870 (+256399us) log.cc:844] Serialized 14 byte log entry
0709 13:02:47.507738 (+1225868us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b18003660a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e20a14bc 00007f32df3ef48b 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a
0709 13:02:47.507976 (+   238us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:47.508030 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:47.508082 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.508167 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.508296 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":356525,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":240,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":2908544,"thread_start_us":943176,"threads_started":2,"wal-append.queue_time_us":730421}
I20260709 13:02:47.578692 11205 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:47.579054 11142 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.580456 11210 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:47.580899  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:47.582262  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:47.620530  9355 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:47.542685 11224 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:47.584246  8672 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:47.584322 11119 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:47.586190  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:47.551945  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:47.629428  8826 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:47.585723  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:47.591861  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.588724 11229 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:47.625908  8676 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:47.599730  8823 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:02:47.600989  8830 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:02:47.601383  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:47.633248  8679 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.629428  9355 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:02:47.627492  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 127547us
I20260709 13:02:47.602427  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:47.622267  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.623208  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:47.633114  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 127806us
W20260709 13:02:47.625815  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 97377.2us
I20260709 13:02:47.649739  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.648772  8678 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:47.632367  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:47.631444  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:47.653381  8673 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:47.629597  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:47.660842  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:47.655115  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:47.664471  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.664827  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:47.667471  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:47.668825  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:47.630271 11221 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:47.670964  8680 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:47.648679 11210 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:47.679275  8677 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.638408 11106 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:47.672266  8672 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:47.685967  8676 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:47.689073  8675 raft_consensus.cc:2368] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5.
I20260709 13:02:47.681226  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:47.695546  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.659528 10983 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
W20260709 13:02:47.700598  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 52010.5us
W20260709 13:02:47.702164  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 79079.1us
W20260709 13:02:47.702539  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 64144.8us
I20260709 13:02:47.703277  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.673707 11108 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:47.631382 11142 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:47.674691  9380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 598) took 1643 ms (client timeout 1931 ms). Trace:
I20260709 13:02:47.706117  8678 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:47.712560  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:47.659082 11119 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:47.719143  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:47.725085  8680 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:47.699576 11106 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:47.738180  8679 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:47.747537  8826 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:47.751125  8681 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: [4.5-5.6]
I20260709 13:02:47.751891 11205 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:47.753047 11229 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:47.751610 10876 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:47.753693 11208 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:47.754370 11208 raft_consensus.cc:697] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:47.755596 10876 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:47.755434 11208 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:47.756909  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 52993.7us
I20260709 13:02:47.757685  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:47.721781 11221 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.719940 11138 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:47.796692  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 93488.9us
I20260709 13:02:47.797464  8677 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:47.797855  8647 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.728554 11224 raft_consensus.cc:2764] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:02:47.799669  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 53030.4us
W20260709 13:02:47.800789  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.801302  8672 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
W20260709 13:02:47.806177  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 137498us
I20260709 13:02:47.809051  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.804045  9352 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:02:47.811143  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.711403  9380 rpcz_store.cc:269] 0709 13:02:46.017108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.018838 (+  1730us) service_pool.cc:224] Handling call
0709 13:02:46.042877 (+ 24039us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:47.647009 (+1604132us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b5400018bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:47.660165 (+ 13156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1201251840}
I20260709 13:02:47.814282  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:47.815500  8825 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:47.816094 11223 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:47.818107  8647 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.818598 11234 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:47.819474 11234 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:47.820012  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:47.821820  8647 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:47.820235  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:47.823022  9387 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:47.733934 11210 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:47.756260 11119 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:47.831602  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:47.820243 11234 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:47.832540  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:47.833691  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:47.834299  8680 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:47.836364  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:47.833814  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:47.837954  8645 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.729147 11180 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:02:47.731729 11228 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 13:02:47.797596 11204 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:47.723261 11213 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:47.843662  9387 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:02:47.844594  8672 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:47.844967  8794 leader_election.cc:400] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6.
W20260709 13:02:47.835325 11231 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:02:47.852185  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51818.5us
I20260709 13:02:47.855803  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:47.835448 11205 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:47.857969  8719 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:47.859145  8673 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:47.859452  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:47.859948  8674 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:47.860827 11210 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:47.838148 11224 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:47.832595 11003 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:47.848908  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:47.824901 11209 raft_consensus.cc:697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:47.856616  8747 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
W20260709 13:02:47.847510 11134 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.684s	user 0.010s	sys 0.005s
I20260709 13:02:47.875247 11242 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.831722 11221 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:47.933178 11242 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:47.876506  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:47.877717 11205 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:47.934809 11228 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:47.935698 11142 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:47.935559 11205 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:47.936421 11240 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:02:47.927860 11224 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:47.936712 11209 consensus_queue.cc:237] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:47.924479 11229 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:47.924013 11195 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:47.877720 11234 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:47.824431 11138 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:47.864087 11200 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.559s	user 0.015s	sys 0.000s
I20260709 13:02:47.845824 10876 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:47.928977  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:47.863438 11244 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:48.095820  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:48.065375 11001 raft_consensus.cc:3055] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:48.109413  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:47.946566 11106 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:48.129688 10876 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:48.113433  8754 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:48.115101  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:48.120123  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:48.123050  8675 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:48.134769  8678 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:02:47.957108 11096 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:48.011840  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.120625 11200 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:48.143703  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.145334 11200 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:48.146565  9380 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:48.147270  8750 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:48.130484 10876 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:48.148528  8756 raft_consensus.cc:2410] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:47.955189 11207 raft_consensus.cc:3055] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:02:47.956542  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 124187us
W20260709 13:02:48.109941  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 277706us
I20260709 13:02:48.014509  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:48.069919  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 332) took 1901 ms. Trace:
W20260709 13:02:48.102226  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 242942us
I20260709 13:02:48.000527 11224 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:48.104779  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:47.940078 11242 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:48.066434 11054 raft_consensus.cc:3055] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:48.013530  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:48.163972  9383 raft_consensus.cc:2410] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:02:48.023437  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.165232  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 150841us
I20260709 13:02:48.166855 11200 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:48.014065  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:48.112268 11001 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:48.139437 11197 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:48.124104  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 315888us
I20260709 13:02:48.174732  9355 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:48.175704  9353 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:48.180428 11239 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:48.181126 11239 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:48.155493  8751 rpcz_store.cc:276] 0709 13:02:46.168742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.185693 (+ 16951us) service_pool.cc:224] Handling call
0709 13:02:46.186027 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.250188 (+ 64161us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:46.872002 (+621814us) log.cc:844] Serialized 14 byte log entry
0709 13:02:47.048571 (+176569us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:47.048632 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:47.048696 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.067535 (+1018839us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b18003af000. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419
0709 13:02:48.069470 (+  1935us) spinlock_profiling.cc:243] Waited 63.8 ms on lock 0x7b54000901d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:48.069548 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.069805 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7314,"prepare.queue_time_us":352046,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":66,"raft.queue_time_us":8090,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":17256,"spinlock_wait_cycles":159769728,"thread_start_us":741020,"threads_started":2,"wal-append.queue_time_us":241599,"wal-append.run_cpu_time_us":1316,"wal-append.run_wall_time_us":1001348}
I20260709 13:02:48.059482 11234 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.183485 11221 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:48.183789 11234 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:48.184242 11124 consensus_queue.cc:1059] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:48.185161 11124 raft_consensus.cc:3055] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:48.185554 11124 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:02:48.187667  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.120396 11138 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:48.175968  9355 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:48.134364  8676 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
W20260709 13:02:47.973639  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 115753us
I20260709 13:02:48.191126  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:48.112761  8674 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:48.112883 11244 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.154009 11129 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:48.206712  8645 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:48.175094  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:48.215714 11138 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:48.218007  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:48.214792  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:48.219004 11229 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:48.219699 11138 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:48.213138  8680 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:48.154363 11207 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:48.171468 11054 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:48.186522 11124 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:48.213379 11250 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:48.220525 11138 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:48.173731 11096 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:02:48.170331 11001 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:48.264423  9389 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:48.224331 10965 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:48.265270 11001 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.265856 10965 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:48.266321 10965 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:48.266989  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:48.267452 10965 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:48.269131 10965 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.224656  8722 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:48.204038 11197 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:48.190557  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:48.287421  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:48.189508 11242 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:02:48.186515 11234 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:48.248113  8647 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.982s (SENT) [suppressed 42 similar messages]
I20260709 13:02:48.186357 11105 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.254529  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:48.211189 11244 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:48.255786 11213 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:48.209662 11132 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:48.182158 11239 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:48.272794  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 48246.8us
W20260709 13:02:48.278008  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.325580  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 85262.2us
I20260709 13:02:48.326606  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:48.248457 10876 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:48.240003 10983 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.306461  9388 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:48.333969  8827 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:48.306972  8674 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:48.308270  8678 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:48.259487 11250 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.264509 11096 raft_consensus.cc:3055] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:48.238830 11205 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:02:48.233596 11129 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:48.286293 11221 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:48.336812  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:48.346884  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:48.341958  8829 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:48.347551  8825 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.347847  8822 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:48.314834 11132 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.312312  9384 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:48.354331  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:48.357160  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:48.341073 11250 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:48.361647  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.332365 11200 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:48.346508 11142 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.401399  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:48.341226 11096 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:02:48.403164  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 248098us
I20260709 13:02:48.359265 11132 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:48.350664 11119 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.110s
W20260709 13:02:48.473735  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 72380.6us
I20260709 13:02:48.361016 11221 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:48.480585  8719 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:48.397017 11124 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:48.483131  8827 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:48.487792  8827 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:48.489125  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:48.471776 11195 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:48.500751  8719 leader_election.cc:400] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5.
I20260709 13:02:48.501299  8719 leader_election.cc:403] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:02:48.343158 11207 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:48.357980 11054 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:48.407855 11142 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:48.527421 11054 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.542765 11195 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:48.543087 11119 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:48.547832  8751 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:48.470901 11138 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:48.523690 11204 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:02:48.550865 11109 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:48.587427  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 98514.6us
I20260709 13:02:48.589233  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:48.551527 11109 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:48.566504 11142 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:48.611121  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.612532  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 364439us
I20260709 13:02:48.554973 11256 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:02:48.519901 11129 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:48.586788 11197 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:48.638269  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:48.639549  9384 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:48.631237 11210 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:02:48.605597 11221 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:48.638278 11264 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:48.648588 11207 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:48.648767 11264 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:48.649461 11264 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.649979 11264 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:48.658125  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.667973  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 167343us
I20260709 13:02:48.661616  8647 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:48.669857  8719 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:48.672010  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.686136  8645 connection.cc:256] RPC call timeout handler was delayed by -2.4097371560000056s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
W20260709 13:02:48.686846  8645 connection.cc:256] RPC call timeout handler was delayed by -2.1784371560000295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
W20260709 13:02:48.689101  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 99990.4us
I20260709 13:02:48.690276  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:48.625953 11142 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:48.693367  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:48.694998  8793 leader_election.cc:336] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.529s (SENT)
I20260709 13:02:48.624083 11234 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:02:48.697708  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:48.703809  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:48.707715  8825 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:48.690255 11205 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:48.711145  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.714470  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:48.719262  8676 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:48.690549 11210 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.723721  8676 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:48.685384  8751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:02:48.727108  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:48.728152  8677 raft_consensus.cc:2393] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:48.636134 11209 consensus_queue.cc:1059] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:02:48.729414 11209 raft_consensus.cc:3055] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:48.729836 11209 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:48.730599  8675 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:48.735824  8719 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:48.738402  8719 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:48.739233  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68090.5us
I20260709 13:02:48.689920 11242 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:02:48.716094 11205 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.746124  8751 raft_consensus.cc:606] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:48.749951  8676 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:48.717926 11265 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:48.743120  9387 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:48.756269  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:48.758545  8675 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:48.759079  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:48.755698  8676 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:48.730541 11276 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.735914 11138 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:48.762058  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 67115us
I20260709 13:02:48.704623 11142 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:48.765733  8793 leader_election.cc:336] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.639s (SENT)
I20260709 13:02:48.763854  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:48.766184  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.763270  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:48.762518  8719 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:48.769537  8676 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:48.770411  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:48.754431 11258 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.755535 11205 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:48.789371  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:48.791312  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:48.794217  8647 leader_election.cc:336] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.506s (SENT)
I20260709 13:02:48.795066  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:48.763321 11265 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:48.757725 11003 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:48.760926 11200 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:48.743088 11272 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:48.803622  8677 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:48.804762  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:48.735567 11210 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:48.811218  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 45512us
I20260709 13:02:48.760496 11275 raft_consensus.cc:2764] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:02:48.816372  8793 leader_election.cc:336] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.700s (SENT)
I20260709 13:02:48.817247  8679 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:48.817746  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:48.790937 11226 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:48.821214  8672 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:48.821525  8673 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:48.819275  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:48.735957 11209 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:48.774003 11268 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:48.830513  8793 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:48.808033 11197 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:48.859336  8672 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:48.859203  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:48.806478 11138 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:48.860761  8679 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
W20260709 13:02:48.862003 10983 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.007s
W20260709 13:02:48.862505 11273 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:02:48.862910 11109 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:48.863483 11138 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:48.828130 11226 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:48.869834 11226 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:48.876742  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 120002us
I20260709 13:02:48.877701  8676 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
W20260709 13:02:48.866377 11001 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.601s	user 0.015s	sys 0.000s
I20260709 13:02:48.771304 11270 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 13:02:48.855700 11224 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:48.854332 11105 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:48.895244  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 78946.1us
W20260709 13:02:48.896936  8793 leader_election.cc:336] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.844s (SENT)
I20260709 13:02:48.859934 11272 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:48.862552 10983 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:48.771358 11276 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:48.906641  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:48.906522  8677 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
W20260709 13:02:48.909008  8647 leader_election.cc:336] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.812s (SENT)
I20260709 13:02:48.909564  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:48.910794 11234 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:48.911717 11265 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:48.913818 11119 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5.
I20260709 13:02:48.803144 11258 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:48.914959 11132 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:48.917421 11132 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:48.918957 11142 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:48.922533 11142 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:48.923619 11281 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.924208  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:48.924733  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:48.925062  8679 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:48.925347  8750 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:48.925707  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:48.925980  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:48.883100 11197 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:48.927450  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:48.924254 11281 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:48.940244  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:48.905058 11105 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:48.943178 11276 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:48.900288 11272 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:48.945281 11250 consensus_meta.cc:287] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.540s	user 0.012s	sys 0.000s
I20260709 13:02:48.950074 10876 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:48.960421  8677 raft_consensus.cc:2368] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4.
I20260709 13:02:48.965268  8675 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:02:48.964550  8750 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:48.966532  8678 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:48.898994 11209 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:48.973026  8672 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:48.976297  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:48.976213  9353 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:48.977174  8674 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:48.898715 11284 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:48.965866 11244 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.639s	user 0.003s	sys 0.011s
I20260709 13:02:48.980914  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:48.984812  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:48.989428 11096 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:48.996171  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 99280.4us
I20260709 13:02:48.966550 11003 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:48.905378 11001 consensus_queue.cc:1059] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.080s
W20260709 13:02:48.980140 10965 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.711s	user 0.014s	sys 0.003s
I20260709 13:02:49.032338  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:49.034133  8793 leader_election.cc:336] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.874s (SENT)
I20260709 13:02:48.973115 11208 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.039143  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:49.045897  8677 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:49.047087  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.047196  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:49.049679  8679 raft_consensus.cc:2410] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:48.972673 11272 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:49.052773  8793 leader_election.cc:336] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.877s (SENT)
I20260709 13:02:49.053381  8793 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:02:49.059651  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 181357us
I20260709 13:02:49.061101  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:48.979135 11200 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:49.003381 11132 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:49.066196  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:49.058035 11278 log.cc:927] Time spent T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.172s	user 0.003s	sys 0.000s
I20260709 13:02:49.077258  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.077793  8750 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:49.079679  8674 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:49.069831 10983 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:49.084326  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.077314  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.085040  8678 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:49.077958  8752 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:49.053289 11096 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:49.058272 11142 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:49.088415  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:49.090777 11197 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.091355 11275 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:49.092176  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:49.095904 11271 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.481s	user 0.002s	sys 0.000s
I20260709 13:02:49.097936 11054 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.099298 11285 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.099177 11200 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.104583  8677 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:48.975335 11250 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:49.107066  8675 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:49.112645  9387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 488) took 1244 ms. Trace:
I20260709 13:02:49.121515  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:49.121208  9387 rpcz_store.cc:276] 0709 13:02:47.868465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.868645 (+   180us) service_pool.cc:224] Handling call
0709 13:02:49.112430 (+1243785us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b54005b13d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:49.112574 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1859081984}
I20260709 13:02:49.122551  8678 raft_consensus.cc:2368] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6.
W20260709 13:02:49.115588 11195 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.572s	user 0.013s	sys 0.000s
I20260709 13:02:48.986843 11244 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:49.125043  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.125419  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.155406  8754 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:49.080240 11132 raft_consensus.cc:697] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:49.127048 11287 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.126933 11138 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:49.127978  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.128796  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:49.130419  8755 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:49.130463 10871 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:02:49.138970  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.119705 11268 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.140151  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:49.140362  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55022.4us
I20260709 13:02:49.058713 11226 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.173867  9381 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:49.126662  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.153353 11275 raft_consensus.cc:697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:49.066844 11208 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:49.167546  8749 raft_consensus.cc:2410] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:49.167660  9388 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:02:49.163058  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:49.169492  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:49.169296  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.254935  8756 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:49.170465  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:49.179121  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.255999  8674 raft_consensus.cc:2393] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:49.190164  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.207440  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.258229  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.181387 11287 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:49.166867 11210 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:49.177109 11286 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:02:49.123310 11054 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.157253 11195 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:49.250361  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 73413.4us
I20260709 13:02:49.300031  8794 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:49.161132 11003 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.210563  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.303213  8677 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:49.171382 11138 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.140843 11285 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.269713  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:49.269964  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.313127  8755 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:49.270582  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.270761  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.314285  8756 raft_consensus.cc:2410] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:02:49.314419  8679 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:49.258384 11132 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:49.291851  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:49.301945  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.306257  8794 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:49.286327 11205 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:49.308104  8677 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:49.317625  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 154707us
I20260709 13:02:49.319377  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.321318  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.328513  8748 raft_consensus.cc:2410] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:49.323503 11138 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:49.323635 11285 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:49.317464 11195 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.308656 10983 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:02:49.263496 11289 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:02:49.328653  8747 raft_consensus.cc:2410] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:49.322860  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.183219 11268 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.300204 11284 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:49.324540 11281 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:49.283321 11275 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:49.338050 11054 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:49.338687 11195 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:49.335837  8645 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.725s (SENT)
I20260709 13:02:49.344761  8749 raft_consensus.cc:2410] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:49.310807 11142 raft_consensus.cc:2764] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:02:49.348717  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.384044  8756 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:49.349314  8747 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:02:49.350368  8753 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:49.352546  8681 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:02:49.343169 11124 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.855s	user 0.006s	sys 0.009s
I20260709 13:02:49.382239  8754 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:49.387578  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:49.401372  8755 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:49.402871  8677 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:49.342162 11272 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:49.407866  8750 raft_consensus.cc:2410] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:49.409847  8679 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:49.406158 11284 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:49.413717 11205 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:49.415854  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 80074.9us
I20260709 13:02:49.416675  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.383584 11054 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:49.441256  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 135367us
I20260709 13:02:49.419905 11250 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:49.447630  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.428653 11244 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:49.441108 11003 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:49.450688  8755 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:49.454444  8749 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:49.478876  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:49.527022  8679 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:49.478205  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.525995  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.529733  8679 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:02:49.549520  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 86737.9us
I20260709 13:02:49.456976  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:49.530921  8752 raft_consensus.cc:2410] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:02:49.530236  8749 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:49.540949  8677 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:02:49.453833 11264 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.784s	user 0.009s	sys 0.005s
I20260709 13:02:49.449607 11284 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.556586  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.546717  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.455142 11226 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:49.559060  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.557471  8755 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:02:49.464495 11272 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:49.466184 11195 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:49.539058 11124 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.579667  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:49.558910 11284 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:49.556228 11200 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.465013 11276 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:49.594763 11276 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:49.595786 11276 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.596843 11276 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:49.596927  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:49.598222  8719 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:49.589735 11200 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:49.590660 11303 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:02:49.609419 11221 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:49.609977 11221 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.610358 11221 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:49.610700 11200 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:49.611445 11200 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:49.612975 11266 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:49.617246  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:49.586185  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.619521  8755 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:49.621048 11287 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:49.478964 11054 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:49.622507  8755 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:49.624334 11054 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:49.624596 11221 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.558751 11264 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.622861 11276 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:49.627771 11221 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:49.627925 11264 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:49.598831 11207 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:49.628342 11105 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:49.598883 11290 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:49.618419 11284 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.589304 11124 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.630725 11284 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:49.631835 11124 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:49.601264 11265 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:49.601740 11270 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:49.632777 11109 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:49.633042 11265 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.633509 11265 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:49.633405 11109 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:49.635768 11109 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:49.599345 11268 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:49.638424  8752 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:49.638370 11200 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:49.639477 11208 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:49.639612 11200 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:49.640616 11200 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.641942 11200 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:49.644611  9353 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:49.645784  9353 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:49.646236  8752 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:02:49.646974 11224 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.647604 11224 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:49.649041 11205 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.649672 11205 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:49.669114  9381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 490) took 1657 ms. Trace:
I20260709 13:02:49.680910  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.647003  8752 raft_consensus.cc:606] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:49.681614  8674 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:49.680848  9381 rpcz_store.cc:276] 0709 13:02:48.011633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.011834 (+   201us) service_pool.cc:224] Handling call
0709 13:02:48.012376 (+   542us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.176144 (+1163768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.668747 (+492603us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54005ea1d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:49.669023 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2668280576}
I20260709 13:02:49.685247  8719 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:02:49.677238  8793 leader_election.cc:336] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.933s (SENT)
W20260709 13:02:49.679577  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 221437us
I20260709 13:02:49.687103  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:49.688499  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.688604  8722 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:49.689284  8679 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:49.691592  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.692552  8674 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
W20260709 13:02:49.695398  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.697080  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:49.697731  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:49.650544 11224 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:49.699608  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:49.702188  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:49.703552  8674 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:49.705099  8677 raft_consensus.cc:3055] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:49.705798  8677 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:49.699429  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:49.547984 11138 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.709883  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.709772  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:49.654415 11208 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:49.712329  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:49.652185 11205 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:49.684805 11268 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.732034  8679 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
W20260709 13:02:49.729594  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 266649us
I20260709 13:02:49.656263 11211 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:49.728798  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:49.732038  8672 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:02:49.732815  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 55615.6us
I20260709 13:02:49.675515  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:49.735953  8722 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:49.700415 11302 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:49.729894  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:49.700493 11124 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:49.746632  8793 leader_election.cc:336] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.116s (SENT)
I20260709 13:02:49.748099  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:49.743068  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.751397 11301 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
W20260709 13:02:49.759896  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 51512us
I20260709 13:02:49.744807 11224 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:02:49.745090 11211 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.769858  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.746659 11276 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:02:49.774252  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:49.657389 10965 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:02:49.754145  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:49.779418  8747 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:49.734390 11195 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:02:49.707454 11266 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:49.545871 11003 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.760867  8752 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:49.749433  8677 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.711511 11290 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:49.729323 11287 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:49.735096 11208 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.775787  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.752563 11197 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:49.773499  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:49.769017  9380 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:49.738116 11289 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:49.771947 11213 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:49.780165 11211 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:49.788115  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:49.800175  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.763482 11124 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.733562 11109 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:02:49.769296 11207 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:49.812148  8753 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:49.745069 11270 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:02:49.774283 11221 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:49.779141 11272 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.735271 11138 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.820041  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:49.832532  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:49.833395  8677 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:49.833940  8645 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:49.835860  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:49.928864  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:49.832106  9381 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:49.788859 11266 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.932402 11266 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:49.859727  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:49.757807 11268 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:49.861261  8829 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:49.939815  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:49.848974  8647 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.649s (SENT)
I20260709 13:02:49.868386  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:49.942168  8751 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:49.942413  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:49.863373  8824 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:49.801448 11290 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:49.815448 11208 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:49.944380  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:49.945883  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:49.940618  8829 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:49.948097  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:49.948954  8829 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:49.877076  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:49.938392 11268 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:49.951117  8826 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:49.940930  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:49.952610  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:49.946472  8824 raft_consensus.cc:2376] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5. Re-sending same reply.
I20260709 13:02:49.877368  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:49.803177 11284 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:49.954324 11284 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:49.955476  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:49.950129  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:49.957564 11284 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:49.891661  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:49.958601  8830 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:49.842836 11287 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:49.960916 11287 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:49.961764  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:49.962814  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:49.944412 11290 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:49.968300  8824 raft_consensus.cc:2393] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:49.936743  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:49.985517  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:49.953459  8826 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:49.994890  8829 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:02:49.995760  8824 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:02:49.997325  8821 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:50.003084  8821 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:49.859530 11306 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:50.492919  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:50.493892  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:50.494370  8753 raft_consensus.cc:2410] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:50.494810  8821 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:49.867904 11226 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:50.081427  8755 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:50.059532  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:50.508836  8823 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:02:50.064250  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 256385us
I20260709 13:02:50.510838  8645 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:50.499034 11323 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:02:50.309528 11302 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.020s	sys 0.050s
I20260709 13:02:50.117648 11197 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:50.064069  8824 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:02:50.540364  8824 pending_rounds.cc:85] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 5
I20260709 13:02:50.285658  8677 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
W20260709 13:02:50.449692  9380 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.000s
W20260709 13:02:50.079469  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:50.548094  8722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.014s	sys 0.025s
W20260709 13:02:50.438525  8720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.020s	sys 0.008s
W20260709 13:02:50.548715  8720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.020s	sys 0.008s
I20260709 13:02:50.063776 11210 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:50.436756 11276 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.080600  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:49.876415 11124 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:50.551561  9380 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:02:50.539418 11323 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:49.874081 11207 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.568910 11207 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:50.569389 11323 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:50.570595 11197 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:50.104671  8826 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:02:50.483278 11003 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:50.575616  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:50.576387  8826 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
W20260709 13:02:50.516361  8793 leader_election.cc:336] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.897s (SENT)
I20260709 13:02:50.517249  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:49.872047 11213 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:50.577437  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:50.577873  8828 raft_consensus.cc:2410] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:02:50.524076  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:50.360234 11310 log.cc:927] Time spent T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.491s	user 0.001s	sys 0.000s
I20260709 13:02:50.541115  8677 pending_rounds.cc:85] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 2
I20260709 13:02:50.580263  8677 pending_rounds.cc:107] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:02:50.580687  8677 raft_consensus.cc:2889] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.581182  8677 pending_rounds.cc:107] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:02:50.581524  8677 raft_consensus.cc:2889] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.003947  8822 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:50.542593  8821 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:49.901780 11224 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:50.020443  8830 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
W20260709 13:02:50.456142  8644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.028s	sys 0.044s
W20260709 13:02:50.551272  8722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.014s	sys 0.026s
W20260709 13:02:50.585143  8644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.034s	sys 0.044s
W20260709 13:02:50.518488 11265 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.884s	user 0.016s	sys 0.004s
W20260709 13:02:50.496740  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 647838us
I20260709 13:02:50.496693 11109 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:50.512635 11187 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.398062 11317 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.064456 11264 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:50.539645 11302 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.024s	sys 0.061s
I20260709 13:02:50.540804  8824 pending_rounds.cc:107] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:02:50.552347 11316 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.501348  8719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.001s	sys 0.014s
I20260709 13:02:50.552788 11138 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:50.539871 10965 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:02:50.583365  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:50.613502  8821 raft_consensus.cc:1813] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Handling vote request from an unknown peer 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:50.614068  8821 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:50.339282  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:50.117964 11205 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:50.549230 11229 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:50.590890  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:50.636446  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 45652.9us
W20260709 13:02:50.637562  8747 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.854s	user 0.013s	sys 0.000s
I20260709 13:02:50.638149  8747 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:02:50.522922 11221 raft_consensus.cc:697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:50.598444  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:50.599273  8824 raft_consensus.cc:2889] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.512738 11204 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.995517 11272 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:50.578492  8823 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
W20260709 13:02:50.603377  8719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.001s	sys 0.014s
I20260709 13:02:50.605711  8822 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:50.560822 11289 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:50.651430  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:50.592058 11109 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.584606 11268 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.596125 11200 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.954s	user 0.015s	sys 0.000s
I20260709 13:02:50.627825  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:50.606487 11224 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:50.603291 11210 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.586728  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:50.642839  8674 consensus_meta.cc:287] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.935s	user 0.010s	sys 0.003s
I20260709 13:02:50.678282  8674 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:50.599082 11302 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:50.597554 11265 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:50.603610 11187 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:50.650717  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 773428us
I20260709 13:02:50.633945  8677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35e078746b204711a1e15f221c16e31a"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:50.704260  8677 raft_consensus.cc:606] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:50.665573  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:50.582572 11276 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:50.578678  8645 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:50.680363  8722 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:50.614988 11321 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:50.673733 11266 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.741s	user 0.012s	sys 0.004s
I20260709 13:02:50.519277 11306 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:50.692826  8822 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:50.692523  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:50.720116  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:50.639380 11320 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:50.649786 11325 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.704771  8677 raft_consensus.cc:614] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:50.660619  8826 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:50.704401  8719 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:50.686158  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:50.707557 11290 consensus_meta.cc:287] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.740s	user 0.015s	sys 0.000s
I20260709 13:02:50.648599  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:50.606957 11264 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:50.715106  8824 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:50.774240  8821 raft_consensus.cc:2410] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:02:50.676492 11205 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.718293  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 162211us
I20260709 13:02:50.659722 11109 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:50.727816  8752 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:02:50.727510  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 212094us
I20260709 13:02:50.585495 11322 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.144s
I20260709 13:02:50.737236  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:02:50.786134  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 49163.4us
I20260709 13:02:50.787271  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:50.656193 11204 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:50.737799  8719 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:50.600519 11317 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:50.703609 11324 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:02:50.766522  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 250205us
I20260709 13:02:50.778962  8826 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:50.719383 11270 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:50.686939 11268 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:50.610906 11316 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:50.788934 11330 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:02:50.794056  9382 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 13:02:50.799286  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 79278.6us
I20260709 13:02:50.788220 11325 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:50.734913 11323 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.806886  8821 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:51.003311  8829 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:51.004171  8821 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:50.807983  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1487) took 1109 ms. Trace:
I20260709 13:02:50.765395 11332 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:50.719884 11200 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 13:02:50.691973 11210 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:50.816072  9380 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:50.817353 11204 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:50.686431 11289 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:51.031226 11320 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.031963 11320 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:51.033428 11289 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:50.805931 11208 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.861s	user 0.014s	sys 0.000s
I20260709 13:02:50.817775  8828 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:51.034418 11208 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:50.823262  8645 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:50.824385 11334 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:51.037096 11208 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:51.037571 11334 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:50.824693  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:50.825699  8825 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:51.039474  8677 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:51.040899  8825 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:50.724236 11266 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:51.044107 11266 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:50.824494 11213 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:51.045382 11213 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:50.820183 11001 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:51.046765 11001 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:50.721206 11321 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:50.808943 11205 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:50.841543 11327 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:51.063651 11327 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.064352 11327 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:50.830353 11197 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:51.067229 11197 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:51.068758 11197 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:50.847489 11333 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:02:50.848304 11187 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:50.849231 11336 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:02:50.852298 11306 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:51.079164  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:50.852196 11207 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:51.086454  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.853328 11054 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:50.854439 11317 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:50.856992 11124 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:50.858176 11287 consensus_meta.cc:287] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.897s	user 0.012s	sys 0.001s
I20260709 13:02:50.859350 11272 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:50.775750 11290 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:50.865459  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:50.869355 11328 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:50.811187 11324 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:50.871819  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1349) took 1103 ms. Trace:
I20260709 13:02:50.786275 11221 consensus_queue.cc:237] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:50.862979 11339 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:50.882797 11229 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:50.873507 11142 raft_consensus.cc:3055] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:02:50.886732  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 121080us
I20260709 13:02:50.890103  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1490) took 1210 ms. Trace:
I20260709 13:02:50.826912 11302 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:50.896977 11322 consensus_queue.cc:1243] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:02:50.904942  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:50.906301  8750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1640) took 1781 ms (client timeout 1992 ms). Trace:
I20260709 13:02:50.906787 11258 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:50.907645 11342 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.908819 11264 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:50.909924  8748 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1650) took 1776 ms (client timeout 1843 ms). Trace:
W20260709 13:02:50.914004 11211 consensus_meta.cc:287] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.024s	user 0.012s	sys 0.000s
I20260709 13:02:50.895107 11265 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:50.917817  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:50.925128  8822 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:51.254714  8748 rpcz_store.cc:269] 0709 13:02:49.133823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.175098 (+ 41275us) service_pool.cc:224] Handling call
0709 13:02:50.909674 (+1734576us) spinlock_profiling.cc:243] Waited 68 ms on lock 0x7b540038aed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:50.909847 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156351872}
W20260709 13:02:50.927824 11314 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.664s	user 0.002s	sys 0.000s
I20260709 13:02:50.933301  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:02:50.941195  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 203491us
W20260709 13:02:50.951360 11347 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 13:02:50.953938  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 143938us
I20260709 13:02:50.946336  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:50.962671 11312 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.852s	user 0.005s	sys 0.000s
I20260709 13:02:51.280323  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:51.270850 11266 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.281773 11264 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:51.281446 11266 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:51.282274 11264 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.282768 11264 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.126602  8827 rpcz_store.cc:276] 0709 13:02:49.768532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.774478 (+  5946us) service_pool.cc:224] Handling call
0709 13:02:49.788583 (+ 14105us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.884937 (+ 96354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.884975 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:49.903420 (+ 18445us) log.cc:844] Serialized 26 byte log entry
0709 13:02:50.551052 (+647632us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:50.551102 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.551188 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.871508 (+320320us) raft_consensus.cc:1685] finished
0709 13:02:50.871534 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.871734 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":110894,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":82694,"thread_start_us":527326,"threads_started":2,"wal-append.queue_time_us":735388}
I20260709 13:02:51.071848 11289 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.973855 11109 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:51.285022 11289 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:51.285496  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:51.286279  8678 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:50.979077 11275 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:51.286842 11275 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:51.287294 11289 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:51.287650  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1359) took 1281 ms. Trace:
I20260709 13:02:50.983704 11276 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:50.985743  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:50.987718  9388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 395) took 2259 ms. Trace:
W20260709 13:02:50.990590  8793 leader_election.cc:336] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.163s (SENT)
W20260709 13:02:50.994385 11313 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.537s	user 0.001s	sys 0.001s
W20260709 13:02:50.995930  8681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1468) took 2191 ms (client timeout 1968 ms). Trace:
I20260709 13:02:50.999006  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:51.002101  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:51.004920  8680 rpcz_store.cc:276] 0709 13:02:49.678989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.681798 (+  2809us) service_pool.cc:224] Handling call
0709 13:02:49.682166 (+   368us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:49.702811 (+ 20645us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.702852 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:49.711218 (+  8366us) log.cc:844] Serialized 52 byte log entry
0709 13:02:50.081633 (+370415us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:50.081690 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.081782 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.788411 (+706629us) raft_consensus.cc:1685] finished
0709 13:02:50.788436 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.788620 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":211,"raft.queue_time_us":20556,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":26571,"thread_start_us":306074,"threads_started":1,"wal-append.queue_time_us":888171}
W20260709 13:02:51.298516  8681 rpcz_store.cc:269] 0709 13:02:48.804139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.804476 (+   337us) service_pool.cc:224] Handling call
0709 13:02:50.995562 (+2191086us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b5400430dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:50.995792 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1254448896}
I20260709 13:02:51.020324  8679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:51.299868  8679 raft_consensus.cc:606] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:51.025383  8821 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:51.284068 11266 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:51.029147 11210 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:51.305182  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:51.305943  8680 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:51.034377 11320 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:51.075789  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:51.076968  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2067) took 1329 ms. Trace:
I20260709 13:02:51.019291 11256 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:51.079233 11001 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:50.853149 11224 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:51.080302  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:02:51.157928  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 96905us
I20260709 13:02:51.118160  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1488) took 1438 ms. Trace:
I20260709 13:02:51.333036  8756 rpcz_store.cc:276] 0709 13:02:49.748099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.814184 (+ 66085us) service_pool.cc:224] Handling call
0709 13:02:51.076726 (+1262542us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b5400138654. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:51.076896 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2033971584}
I20260709 13:02:51.290017 11287 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:51.272894  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 339685us
I20260709 13:02:51.341259  8675 rpcz_store.cc:276] 0709 13:02:49.679267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.751254 (+ 71987us) service_pool.cc:224] Handling call
0709 13:02:49.751706 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.783917 (+ 32211us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:49.845712 (+ 61795us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.355761 (+510049us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:50.392355 (+ 36594us) log.cc:844] Serialized 26 byte log entry
0709 13:02:50.392896 (+   541us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:50.393026 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.393238 (+   212us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:51.117764 (+724526us) raft_consensus.cc:1685] finished
0709 13:02:51.117802 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.118052 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21488,"prepare.queue_time_us":47744,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":3765,"raft.queue_time_us":173177,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":20146,"thread_start_us":295631,"threads_started":1,"wal-append.queue_time_us":779432}
I20260709 13:02:51.343972  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:51.264566 11054 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:51.284106 11207 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:50.971324 11348 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.348016  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1969) took 2448 ms. Trace:
I20260709 13:02:50.975826  8823 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.109673 11311 raft_consensus.cc:993] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: : Instructing follower e814ed2013244fa6be42623ea5c0c7b9 to start an election
I20260709 13:02:51.292891  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:51.294994  9381 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:51.294927  8793 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:51.297364  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 366) took 1830 ms. Trace:
I20260709 13:02:51.255510 11142 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:51.021777  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2065) took 1280 ms. Trace:
I20260709 13:02:51.386687  8752 rpcz_store.cc:276] 0709 13:02:49.741830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.799767 (+ 57937us) service_pool.cc:224] Handling call
0709 13:02:51.021545 (+1221778us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b540038b8d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:51.021694 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1978364672}
I20260709 13:02:51.023669 11323 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:51.265048 11328 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.294325  9388 rpcz_store.cc:276] 0709 13:02:48.728490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.728680 (+   190us) service_pool.cc:224] Handling call
0709 13:02:48.729104 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.254135 (+525031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.987340 (+1733205us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b5400469954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:50.987619 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1004568832}
I20260709 13:02:51.303210  9352 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:51.309460  8679 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:51.108167 11316 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:51.106693 11205 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:51.306891  9355 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:51.110517 11268 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:51.256711 11343 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:51.333065  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 331052us
I20260709 13:02:51.438855  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:51.112397 11325 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:51.335850  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:51.440501  8672 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:02:51.336995  9383 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:51.441933  9383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 399) took 1766 ms. Trace:
I20260709 13:02:51.442461  9383 rpcz_store.cc:276] 0709 13:02:49.675890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.689921 (+ 14031us) service_pool.cc:224] Handling call
0709 13:02:51.075590 (+1385669us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:51.441840 (+366250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3944192}
I20260709 13:02:51.331102 11213 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:51.332439 11256 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:02:51.269616  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.309883 11204 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 13:02:51.276469  8674 rpcz_store.cc:276] 0709 13:02:49.679732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.698787 (+1019055us) service_pool.cc:224] Handling call
0709 13:02:50.699417 (+   630us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:50.710204 (+ 10787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:50.710261 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:50.741792 (+ 31531us) log.cc:844] Serialized 26 byte log entry
0709 13:02:50.881049 (+139257us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:50.881093 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.882019 (+   926us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.889828 (+  7809us) raft_consensus.cc:1685] finished
0709 13:02:50.889849 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.890029 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10343,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":9133,"thread_start_us":70140,"threads_started":1,"wal-append.queue_time_us":103634}
I20260709 13:02:51.448618  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1494) took 1764 ms. Trace:
W20260709 13:02:51.265461  8750 rpcz_store.cc:269] 0709 13:02:49.125308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.127582 (+  2274us) service_pool.cc:224] Handling call
0709 13:02:50.905945 (+1778363us) spinlock_profiling.cc:243] Waited 79.2 ms on lock 0x7b54002ad354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:50.906201 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182269696}
I20260709 13:02:51.288110  8829 rpcz_store.cc:276] 0709 13:02:50.005899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.021616 (+ 15717us) service_pool.cc:224] Handling call
0709 13:02:51.287453 (+1265837us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b540016bf54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:51.287578 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1017695104}
I20260709 13:02:51.294585 11109 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:51.349476  8676 rpcz_store.cc:276] 0709 13:02:48.887807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.888549 (+   742us) service_pool.cc:224] Handling call
0709 13:02:48.888895 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:48.889131 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:49.257267 (+368136us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.926230 (+1668963us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b18002f0d40. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:02:51.335489 (+409259us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:51.335552 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:51.335626 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.335694 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.335831 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":418203,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":762,"raft.queue_time_us":43125,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":30464256,"thread_start_us":688128,"threads_started":3,"wal-append.queue_time_us":1771993}
I20260709 13:02:51.262641 11330 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:02:51.357053  9386 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:51.381991  8754 rpcz_store.cc:276] 0709 13:02:49.467302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.604940 (+137638us) service_pool.cc:224] Handling call
0709 13:02:49.605354 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.605844 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.605895 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:49.606476 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.763178 (+156702us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:49.763228 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.763768 (+   540us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.958738 (+1194970us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:51.289432 (+330694us) raft_consensus.cc:1685] finished
0709 13:02:51.289456 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.297265 (+  7809us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1060,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":132,"spinlock_wait_cycles":515064576,"thread_start_us":148762,"threads_started":1,"wal-append.queue_time_us":256444}
I20260709 13:02:51.287911 11275 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:51.390475 11348 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:51.346043  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 400) took 1662 ms. Trace:
I20260709 13:02:51.416646 10983 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:51.355686  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:51.291472 11211 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:51.333985 11266 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:51.345444  8751 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:51.449213  8678 rpcz_store.cc:276] 0709 13:02:49.683934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.289778 (+1605844us) service_pool.cc:224] Handling call
0709 13:02:51.307947 (+ 18169us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:51.308254 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:51.331280 (+ 23026us) log.cc:844] Serialized 14 byte log entry
0709 13:02:51.448172 (+116892us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:51.448250 (+    78us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:51.448311 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.448391 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.448537 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8690,"prepare.queue_time_us":23559,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":12981,"thread_start_us":107712,"threads_started":1}
I20260709 13:02:51.459391  8680 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:02:51.461485  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 638403us
I20260709 13:02:51.432215 11207 raft_consensus.cc:1081] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Signalling peer e814ed2013244fa6be42623ea5c0c7b9 to start an election
W20260709 13:02:51.468987  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 165888us
W20260709 13:02:51.467748  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 160980us
I20260709 13:02:51.494153  8747 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:51.457669 10983 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:51.499544  8752 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:51.431118 11328 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.499300  8750 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:51.513506  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:51.494853  8681 raft_consensus.cc:2955] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:51.515733  8823 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:51.446888 10965 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:51.445775 11213 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:51.523940  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:51.528546  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:51.529349  8754 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:51.303898 11342 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:51.453212 11317 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:51.460026 11205 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:51.461443 11138 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:51.498704 11226 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.534102  8755 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:02:51.416129 11142 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:51.524585 11275 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.543547 11187 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:51.545473 11239 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:51.547431  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:51.547179  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:51.535425  8756 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:51.566783  8829 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:51.486991 11109 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.487402 11250 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:51.491376 11287 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:51.547173 11322 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:51.601323 11322 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:51.453972  9384 rpcz_store.cc:276] 0709 13:02:49.676212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.684418 (+  8206us) service_pool.cc:224] Handling call
0709 13:02:50.946160 (+1261742us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:51.338206 (+392046us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45292800}
I20260709 13:02:51.463464 11208 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:02:51.496822 11334 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:51.540441  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:51.614744  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 74423.2us
I20260709 13:02:51.486233 11210 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.080s
I20260709 13:02:51.454517 11343 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:51.529620 11213 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:51.585474 11142 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.532045 10965 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.511545 11054 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:51.556788 11258 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:02:51.591158  9352 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:51.584887  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:51.553773  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:51.503464 11266 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:51.553721 11317 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:51.613699  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 151146us
I20260709 13:02:51.562521  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:51.667730  8747 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.561486 11226 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:51.618561  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:51.672467  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:02:51.675395  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 57018.2us
I20260709 13:02:51.675980  8675 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:51.570966  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 57531.2us
I20260709 13:02:51.556854 11132 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:02:51.683694 11360 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:51.684269 11360 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.684644 11360 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.685874  8672 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:02:51.686648  8672 raft_consensus.cc:606] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:51.687016  8672 raft_consensus.cc:614] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:51.689422  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:51.690205  8676 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.563871  9380 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:51.695355  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 376) took 1917 ms. Trace:
I20260709 13:02:51.695858  8751 rpcz_store.cc:276] 0709 13:02:49.778100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.943016 (+164916us) service_pool.cc:224] Handling call
0709 13:02:49.943346 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:51.486286 (+1542940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.695040 (+208754us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54002fdfd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:51.695285 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3224428416}
I20260709 13:02:51.697760  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 410) took 1084 ms. Trace:
I20260709 13:02:51.698390  8823 rpcz_store.cc:276] 0709 13:02:50.613052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.660470 (+ 47418us) service_pool.cc:224] Handling call
0709 13:02:51.697414 (+1036944us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b5400267fd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:51.697676 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":587124992}
I20260709 13:02:51.538307 11339 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:51.700199 11339 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:51.557762 11138 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:51.688148  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:51.707036  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:51.664517  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 73499us
I20260709 13:02:51.668857  8719 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:51.535843 11224 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:02:51.677413  8645 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:51.626830 10983 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:51.633994 11205 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:51.719607  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50857.4us
I20260709 13:02:51.633580 11334 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:51.682950 11321 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:51.697870  8824 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:02:51.634163  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1472) took 2826 ms. Trace:
I20260709 13:02:51.667528 11250 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:51.712111  8672 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:51.712188  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:51.673996 11287 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.602533 11322 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:51.605149 11109 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:51.721246  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 397) took 1027 ms. Trace:
I20260709 13:02:51.784560  9352 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:51.749892  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 402) took 1053 ms. Trace:
I20260709 13:02:51.787104  8680 rpcz_store.cc:276] 0709 13:02:50.696186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.320254 (+624068us) service_pool.cc:224] Handling call
0709 13:02:51.320806 (+   552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:51.468736 (+147930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.749645 (+280909us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400257bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:51.749820 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":310377728}
I20260709 13:02:51.787678  8675 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:02:51.749941  8719 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:51.788722  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:51.751494 11210 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:51.755903  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:51.789510  8675 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:51.790092  8679 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:51.663970  8755 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:51.638386 10965 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:51.768322 11358 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:02:51.752731  8673 rpcz_store.cc:276] 0709 13:02:48.807483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.822430 (+ 14947us) service_pool.cc:224] Handling call
0709 13:02:48.823181 (+   751us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:49.265291 (+442110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.265347 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:49.269828 (+  4481us) log.cc:844] Serialized 52 byte log entry
0709 13:02:50.907135 (+1637307us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b18003a66c0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:02:50.959991 (+ 52856us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:51.254393 (+294402us) log.cc:844] Serialized 14 byte log entry
0709 13:02:51.254944 (+   551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.633447 (+378503us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b5400176454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:51.633539 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:51.633625 (+    86us) raft_consensus.cc:1685] finished
0709 13:02:51.633647 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.633920 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13388,"prepare.queue_time_us":318518,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":2313,"raft.run_cpu_time_us":444,"raft.run_wall_time_us":43055,"spinlock_wait_cycles":1034596480,"thread_start_us":163713,"threads_started":2,"wal-append.queue_time_us":1780898}
I20260709 13:02:51.769102 11317 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:51.769187 11323 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:51.766577 11290 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:51.770967  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 93655.3us
I20260709 13:02:51.810426  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:51.697921 11332 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.002s
I20260709 13:02:51.778038  8747 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:51.783232  8681 rpcz_store.cc:276] 0709 13:02:50.694169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.331338 (+637169us) service_pool.cc:224] Handling call
0709 13:02:51.338833 (+  7495us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:51.343153 (+  4320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:51.343188 (+    35us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:51.344494 (+  1306us) log.cc:844] Serialized 546 byte log entry
0709 13:02:51.462266 (+117772us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:51.608398 (+146132us) log.cc:844] Serialized 14 byte log entry
0709 13:02:51.608671 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.608763 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:51.720847 (+112084us) raft_consensus.cc:1685] finished
0709 13:02:51.720880 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.721145 (+   265us) inbound_call.cc:177] Queueing success response
0709 13:02:51.773840 (+ 52695us) spinlock_profiling.cc:243] Waited 9.22 ms on lock 0x7b48001002d8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
Metrics: {"raft.queue_time_us":44024,"raft.run_cpu_time_us":2692,"raft.run_wall_time_us":193590,"spinlock_wait_cycles":21197568,"thread_start_us":86898,"threads_started":1,"wal-append.queue_time_us":220057}
I20260709 13:02:51.758685  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:51.796588  8676 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:51.802716  8678 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:51.775348 11342 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:51.751899 11265 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:51.826592  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:51.830896  8680 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:51.820330  9352 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:51.764878 11311 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.827865  8679 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:51.777493 11324 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:51.821051  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:51.829370  8681 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:51.775401 11211 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:51.813889 10983 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:51.832775  8645 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:51.823800  8672 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.837239  8676 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:51.775348 11348 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:51.839053  8675 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.821545 11332 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:51.815116 11229 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:51.837049 11197 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:51.783924 11287 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:51.808190 11250 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:02:51.830673 11265 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:02:51.879634  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 129777us
I20260709 13:02:51.833771 11258 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:02:51.853753 11341 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:51.887398  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 55122.1us
I20260709 13:02:51.841442 11371 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:51.855021  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 178612us
I20260709 13:02:51.858194 11226 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:51.885097  8672 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:51.855535 11323 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:51.866314 11197 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.892704 11197 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:51.892417  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:02:51.881908 11366 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:02:51.871843 11138 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:51.839150 11311 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:51.858528 11210 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:02:51.890192 11371 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.882500 11265 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:51.865643 11224 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:02:51.925217  8647 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:02:51.887933 11341 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.930357  8675 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:51.936662 11229 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:51.910200 11266 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:51.943246  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 66654.8us
I20260709 13:02:51.925999 11323 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.937789 11372 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:51.926574 11371 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:51.966907  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:51.937938 11341 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:51.966921 11264 consensus_meta.cc:287] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.683s	user 0.015s	sys 0.000s
I20260709 13:02:51.980667  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:02:51.993779  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.993835  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1491) took 2313 ms. Trace:
I20260709 13:02:51.963419 11229 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:51.994673  8677 rpcz_store.cc:276] 0709 13:02:49.679980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.040604 (+1360624us) service_pool.cc:224] Handling call
0709 13:02:51.040974 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:51.354883 (+313909us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:02:51.354943 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:51.387735 (+ 32792us) log.cc:844] Serialized 26 byte log entry
0709 13:02:51.632141 (+244406us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:51.632196 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.632281 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:51.993573 (+361292us) raft_consensus.cc:1685] finished
0709 13:02:51.993601 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.993755 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4868,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":90,"thread_start_us":504293,"threads_started":2,"wal-append.queue_time_us":264439}
I20260709 13:02:51.977780 11372 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.976430 11205 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:51.988507 11348 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:51.967998 11328 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.000304 11221 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:51.966718 11369 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:02:51.970952 11323 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.016609 11372 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.998065 11264 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.021205 11334 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:52.000527 11324 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:02:52.029955  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:52.046829  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:52.051415  8677 raft_consensus.cc:2376] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7. Re-sending same reply.
I20260709 13:02:52.047142  8675 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:52.033622 11348 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:52.026144 11205 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.069473 11229 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.062381 11373 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.071065 11373 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.043849 11224 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:52.062039  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:52.056432 11327 consensus_meta.cc:287] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.989s	user 0.010s	sys 0.008s
I20260709 13:02:52.078277  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.066740  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1573) took 1480 ms. Trace:
I20260709 13:02:52.079025  8824 rpcz_store.cc:276] 0709 13:02:50.586830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.776029 (+189199us) service_pool.cc:224] Handling call
0709 13:02:51.697582 (+921553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:51.752691 (+ 55109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.066433 (+313742us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b5400267fd0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:52.066654 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2118566272}
I20260709 13:02:52.052619 11287 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:52.075338  8678 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:52.057111 11382 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.088963 11359 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:52.065079 11351 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:52.096212 11213 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.097348 11224 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.108093  8575 catalog_manager.cc:5697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 2 to 4, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 4 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:52.104941 11264 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:52.064064 11352 raft_consensus.cc:3055] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:02:52.119943  8647 leader_election.cc:336] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.532s (SENT)
I20260709 13:02:52.107903  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:52.135008 11352 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:52.135190  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:52.136605 11357 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:52.136193 11352 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.133630  8575 catalog_manager.cc:5697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 3 to 4, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 4 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:02:52.128525  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.139807  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.140815  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:52.144891  8747 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:52.137995  8575 catalog_manager.cc:5697] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:52.145015  8752 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:52.110122 11351 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:52.143464  8680 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:52.143711  8748 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:52.165022  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:52.147214  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.166064  8751 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:52.155452  8575 catalog_manager.cc:5697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 4 to 6, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 6 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:02:52.150374  8719 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:52.165215  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.151435 11109 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:52.167423  8575 catalog_manager.cc:5697] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:02:52.156778  8576 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40096 (request call id 139) took 19327 ms (client timeout 15000 ms). Trace:
I20260709 13:02:52.168819  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:52.165911  8755 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:52.172235  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:52.173785  8672 raft_consensus.cc:2376] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4. Re-sending same reply.
I20260709 13:02:52.175084  8719 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:52.175930  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:52.175889  8575 catalog_manager.cc:5697] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:02:52.178380  8575 catalog_manager.cc:5697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 4 to 5, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 5 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:52.180701  8575 catalog_manager.cc:5697] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:52.182830  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.183148  8678 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:52.186345  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.140679 11224 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:52.080615 11327 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.189574  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.191423  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.185185  8575 catalog_manager.cc:5697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 6, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 6 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:52.193665  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.195883  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.197191  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.197933  8672 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:52.195339  8575 catalog_manager.cc:5697] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 2, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:02:52.127831 11382 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.201500  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.202832  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.203696 11382 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:52.204097 11352 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:52.183411 11290 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:52.205252 11290 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.205653 11290 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:52.177012  8576 rpcz_store.cc:269] 0709 13:02:32.829745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.845448 (+ 15703us) service_pool.cc:224] Handling call
0709 13:02:34.960036 (+2114588us) spinlock_profiling.cc:243] Waited 9.39 ms on lock 0x7b54002d7dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2463090 00007f32e2501f3a 00007f32e82b81cc 00007f32e83b637e 00007f32e68772a4 00007f32e6877232 00007f32e6877161 00007f32e68770dc 00007f32e6876392 00007f32e083448c 00007f32e08338c6
0709 13:02:52.156671 (+17196635us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:46.903292 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:46.999323 (+ 96031us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:46.999392 (+    69us) write_op.cc:435] Acquired schema lock
0709 13:02:46.999408 (+    16us) tablet.cc:662] Decoding operations
0709 13:02:47.000472 (+  1064us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:47.000566 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:47.000597 (+    31us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:02:47.014349 (+ 13752us) tablet.cc:701] Row locks acquired
0709 13:02:47.030229 (+ 15880us) write_op.cc:260] PREPARE: finished
0709 13:02:47.069165 (+ 38936us) write_op.cc:270] Start()
0709 13:02:47.095881 (+ 26716us) write_op.cc:276] Timestamp: P: 1783602167048429 usec, L: 0
0709 13:02:47.095919 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:02:47.139161 (+ 43242us) log.cc:844] Serialized 2780 byte log entry
0709 13:02:47.756577 (+617416us) op_driver.cc:464] REPLICATION: finished
0709 13:02:48.471070 (+714493us) write_op.cc:301] APPLY: starting
0709 13:02:48.484451 (+ 13381us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:48.484700 (+   249us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:48.484899 (+   199us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:48.580445 (+ 95546us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:48.650814 (+ 70369us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:02:48.650880 (+    66us) write_op.cc:312] APPLY: finished
0709 13:02:48.651341 (+   461us) log.cc:844] Serialized 59 byte log entry
0709 13:02:48.651616 (+   275us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:48.651757 (+   141us) write_op.cc:454] Released schema lock
0709 13:02:48.651863 (+   106us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:48.652443 (+   580us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:48.652480 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":647997,"reported_tablets":17,"spinlock_wait_cycles":21590272,"child_traces":[["op",{"apply.queue_time_us":574363,"apply.run_cpu_time_us":5717,"apply.run_wall_time_us":254881,"num_ops":7,"prepare.queue_time_us":867849,"prepare.run_cpu_time_us":13749,"prepare.run_wall_time_us":287949,"raft.queue_time_us":190264,"raft.run_cpu_time_us":20125,"raft.run_wall_time_us":529979,"replication_time_us":628354,"thread_start_us":726399,"threads_started":4,"wal-append.queue_time_us":67436,"wal-append.run_cpu_time_us":36180,"wal-append.run_wall_time_us":2445464}]]}
I20260709 13:02:52.211880  8575 catalog_manager.cc:5697] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: leader changed from <none> to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 2 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:52.213479  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.186414 11334 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:02:52.190181 11205 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:52.223686 11379 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:02:52.250422  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:52.254273  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.187139 11330 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:02:52.224308  8749 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:52.189517 11109 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:02:52.219460 11324 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:52.089865 11359 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.258177 11324 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:52.201488 11213 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:52.185017 11348 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.224741  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.223580 11327 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:52.255034  8676 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:52.221109 11360 consensus_meta.cc:287] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.536s	user 0.015s	sys 0.000s
I20260709 13:02:52.258731 11351 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.265282 11351 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:52.264971 11360 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.221825 11317 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:52.274590  8677 raft_consensus.cc:2393] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:52.273892  8676 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
W20260709 13:02:52.294333  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 84216.9us
I20260709 13:02:52.270447 11323 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:52.247028  8575 catalog_manager.cc:5697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 3, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:02:52.266873  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:52.268368  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 92490us
I20260709 13:02:52.245554 11224 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:52.269938 10965 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.270637 11360 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:52.271181 11380 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:02:52.271716 11373 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:52.273476 11197 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:52.352240  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 1.00837e+06us
I20260709 13:02:52.353214  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:52.259243 11324 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.311877  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.348717  8674 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:02:52.354411  8672 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:52.265785 11372 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.356009  8575 catalog_manager.cc:5697] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 3 to 4, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 4 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:02:52.360096  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:52.360803  8674 raft_consensus.cc:2376] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6. Re-sending same reply.
I20260709 13:02:52.361570  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:52.363216  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.363770  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.364073  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.364425  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.299154 11317 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:52.366981 11284 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:52.368089  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 1.37758e+06us
I20260709 13:02:52.328240 11359 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:52.371289  8793 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.211s (SENT)
I20260709 13:02:52.313428 11221 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:52.372255  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:52.356547 11276 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:52.373160  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:52.374325  8755 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:52.370580  8674 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:52.376685  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.377772  8673 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:52.310145 11341 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.385784 11320 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:52.387176 11301 consensus_queue.cc:1059] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:52.387368  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:52.388532 11301 raft_consensus.cc:3055] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:52.388931 11301 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:52.389318 11321 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:52.389941 11321 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:52.390117 11372 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:52.391842 11321 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.378978 11272 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:52.395057 11276 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:52.395700 11276 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.397396  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1660) took 3034 ms. Trace:
I20260709 13:02:52.398064  8673 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:52.398197  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:52.399874  8821 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:52.402249  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.401227  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:52.403246  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.367393 11325 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.397838  8749 rpcz_store.cc:276] 0709 13:02:49.344763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.587797 (+243034us) service_pool.cc:224] Handling call
0709 13:02:50.915813 (+1328016us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.256698 (+1340885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.373318 (+116620us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540038aed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:52.379192 (+  5874us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540038aed0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:52.379356 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5860380288}
I20260709 13:02:52.401911  8814 tablet_service.cc:1558] Processing DeleteTablet for tablet 51388ea026cc4d098949a4a0327a35fc with delete_type TABLET_DATA_TOMBSTONED (TS e814ed2013244fa6be42623ea5c0c7b9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60414
I20260709 13:02:52.407128  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.376205 11268 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.411443  8673 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:02:52.273605 11256 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.415550 11256 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:52.354984 11224 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.366860 11389 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.047s
W20260709 13:02:52.427090  8793 connection.cc:256] RPC call timeout handler was delayed by -1.8928920659999449s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20440000000000003s for a response
W20260709 13:02:52.427690  8793 connection.cc:256] RPC call timeout handler was delayed by -1.8397920659999727s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21030000000000004s for a response
I20260709 13:02:52.394158 11272 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:52.428320  8793 leader_election.cc:336] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.956s (SENT)
I20260709 13:02:52.428824  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:52.426659 11359 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:02:52.331609 11371 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.419478  8821 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:52.387176 11341 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:52.439448  9355 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:52.427207 11302 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:52.416498 11343 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:52.461186  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.401005 11221 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.465808  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.450601 11275 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:52.480429  8748 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:52.480286  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:52.481664  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.467880  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.483068  8749 raft_consensus.cc:1813] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Handling vote request from an unknown peer e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:52.469198  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.483402  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:52.484930  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.471491 11398 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.485159  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.475255  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:52.486001  8830 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
W20260709 13:02:52.452389 11394 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:02:52.487283  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.479419  8821 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:52.482795  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:52.452894 11383 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:52.486091  8753 raft_consensus.cc:2410] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:52.476984  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:52.486843  8749 raft_consensus.cc:2368] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3.
I20260709 13:02:52.470281  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.490797  8719 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:52.485703  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:52.491123  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:52.491292  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:52.453222 11284 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.493155  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:52.493326 11284 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.416818 11325 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.497742  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:52.494370 11398 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.499691 11325 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:52.447374 11322 consensus_queue.cc:1059] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:02:52.488561  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.481083 11275 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:52.438134 11352 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:02:52.507951 11275 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.464484 11302 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:52.509002 11302 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:52.489461  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:52.510875  8676 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:52.436561 11265 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.491727  8825 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:52.516664  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:52.491847  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:52.491727  8822 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:52.438781 11323 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:02:52.492416  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.492697  9384 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:02:52.519999 11396 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.075s	user 0.000s	sys 0.002s
I20260709 13:02:52.462597 11359 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:52.521337 11359 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.523734 11109 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:52.524822  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.524931 11359 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:52.526782  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:52.527525  8829 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:52.528479  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:52.528713 11322 raft_consensus.cc:3055] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:52.529268  8829 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:52.529318 11322 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:52.533245 11224 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:52.423720 11268 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.497212 11284 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:52.438931 11264 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:02:52.502115  8830 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:52.537756 11301 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.502379  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.503573 11324 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:52.503940 11197 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:52.504312 11334 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:52.462548 11317 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.505439  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 408) took 1903 ms. Trace:
W20260709 13:02:52.545225  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.547456  8826 rpcz_store.cc:276] 0709 13:02:50.602161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.002918 (+400757us) service_pool.cc:224] Handling call
0709 13:02:51.024846 (+ 21928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:51.256844 (+231998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:51.256885 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:51.256935 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:51.256966 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.505205 (+1248239us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54001946d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:52.505256 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:52.505377 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":533017984}
W20260709 13:02:52.548508  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.506275  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:52.518435  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:52.549371  8822 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:52.552981  8676 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:02:52.497221 11371 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:52.535370  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 43615.3us
I20260709 13:02:52.551399 11334 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:52.550904  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 44708us
I20260709 13:02:52.568348 11334 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:52.554821  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:52.573014 11197 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:52.573742 11197 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:52.547226 11324 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.574923 11324 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:52.574756 11197 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.576092 11265 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:52.545812 11343 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:52.555042  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:52.586601  8676 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:52.589041  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:52.556404  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:52.488656 11341 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:02:52.560163  8751 raft_consensus.cc:2410] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:52.510546 11383 raft_consensus.cc:697] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:52.539939  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:52.601609  8825 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:52.602106  8829 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:02:52.504518 11398 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:52.510185 11275 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:52.583678  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:52.584852  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:52.604708 11208 raft_consensus.cc:3055] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:52.605299 11208 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:52.605345  8752 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:52.606520 11208 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.607103  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.607637 11208 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:52.607919  8751 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:52.610338  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.611359 11301 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:52.612433  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.611547 11383 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.595405 11268 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:52.614152  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.615566  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.600467  8676 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:52.590734  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:52.516076 11330 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:02:52.570271 11322 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.622682 11393 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:52.623049 11322 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:52.623301 11393 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:52.622862 11272 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:52.616920 11375 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.586422 11343 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.578841 11323 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:02:52.618296  8752 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:52.628394  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:52.628723  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.626842  8719 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:52.627475 11210 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:02:52.634198  8752 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:02:52.634718  8752 pending_rounds.cc:85] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 6
I20260709 13:02:52.635039  8752 pending_rounds.cc:107] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:02:52.634809  8645 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:52.635350  8752 raft_consensus.cc:2889] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:52.637531  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:52.625247 11359 consensus_queue.cc:1175] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7aff04f53d4845a6a4fa491fd2a06145" responder_term: 6 status { last_received { term: 4 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: [5.6-5.6]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:52.631539 11343 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:52.625803 11268 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:52.640573  8752 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:02:52.656502  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.690971  8677 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:52.639928  8645 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:02:52.692167  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 52329.9us
W20260709 13:02:52.692508  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 70309.1us
I20260709 13:02:52.630157  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.695537  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:52.696851  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:52.618289  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:52.662385  8793 connection.cc:256] RPC call timeout handler was delayed by -1.2294397409999647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17200000000000001s for a response
I20260709 13:02:52.700769  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:52.701005  8793 connection.cc:256] RPC call timeout handler was delayed by -1.145739740999943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18130000000000002s for a response
W20260709 13:02:52.701700  8793 connection.cc:256] RPC call timeout handler was delayed by -1.1277397409999708s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18330000000000002s for a response
I20260709 13:02:52.624336 11393 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:52.702370  8793 connection.cc:256] RPC call timeout handler was delayed by -1.1016397409999854s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18620000000000003s for a response
I20260709 13:02:52.702824 11393 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:52.702955  8793 connection.cc:256] RPC call timeout handler was delayed by -1.0917397409999694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18730000000000002s for a response
I20260709 13:02:52.639743 11229 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:52.656203 11382 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:52.703529  8793 connection.cc:256] RPC call timeout handler was delayed by -1.0098397409999507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
I20260709 13:02:52.709821  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:52.706007  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:52.711073  8677 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:52.651773 11311 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:52.661638 11205 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.697129 11213 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:52.629993 11224 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 13:02:52.703941 11330 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:52.722359 11392 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:02:52.723021 11205 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.727246 11210 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:52.696636 11324 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:52.634152 11352 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 13:02:52.709438 11272 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.740778 11392 consensus_queue.cc:1243] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:02:52.734040 11268 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:52.736905 11360 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.708473 11229 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.754912 11229 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:52.752876 11330 raft_consensus.cc:697] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:52.765367 11265 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:52.766081 11265 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.749966  8676 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:02:52.834569  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 124895us
I20260709 13:02:52.835772  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:52.848940  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:52.865458  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:52.655990 11375 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:52.869153 11375 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:52.875700  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.876616  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.919788  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.930810 11370 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.838s	user 0.004s	sys 0.001s
I20260709 13:02:52.760107 11272 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:52.940709  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.096076  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.096551  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.986045  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:52.834105 11360 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:53.033272  8575 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.030s	sys 0.057s
W20260709 13:02:53.102221  8575 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.031s	sys 0.058s
W20260709 13:02:53.047286  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.080466  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 210551us
I20260709 13:02:52.863023 11275 raft_consensus.cc:2764] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:02:52.863698 11357 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:52.864924 11316 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.864742 11398 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:53.128831  8830 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.010s	sys 0.001s
I20260709 13:02:53.033278 11372 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:02:53.110221  8822 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.003s	sys 0.016s
I20260709 13:02:53.125206  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:53.126083  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:53.126734  8824 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.016s	sys 0.009s
I20260709 13:02:53.082758  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:53.141053  8822 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.004s	sys 0.017s
I20260709 13:02:53.141307  8825 raft_consensus.cc:2376] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6. Re-sending same reply.
I20260709 13:02:53.143147  9355 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:02:53.143440  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:53.141060  8824 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.017s	sys 0.009s
W20260709 13:02:53.108255 11290 consensus_meta.cc:287] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.902s	user 0.010s	sys 0.017s
W20260709 13:02:53.104568 11334 consensus_meta.cc:287] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.533s	user 0.014s	sys 0.000s
W20260709 13:02:53.129273  8830 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.010s	sys 0.001s
I20260709 13:02:53.145624  8755 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:53.144790  8825 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:53.157688  9355 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:53.157832  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:53.143522  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:53.151964  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 453538us
I20260709 13:02:53.159554  8752 raft_consensus.cc:2368] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8.
I20260709 13:02:53.158003  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:53.160444  8825 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:02:53.127377  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:53.162433  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:53.163357  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:53.164737  8677 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:02:53.166057  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.169872  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:53.171442  9353 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.001s	sys 0.022s
I20260709 13:02:53.025234 11330 consensus_queue.cc:237] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:53.129129 11398 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:53.177069  9353 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.001s	sys 0.022s
I20260709 13:02:53.181713  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:53.183223  8677 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:53.147687 11325 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:53.151466 11371 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:53.127332 11367 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.004s	sys 0.021s
I20260709 13:02:53.191118  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:53.129387 11357 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:53.196187  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.186718 11256 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.771s	user 0.010s	sys 0.010s
W20260709 13:02:53.198724  8826 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.009s	sys 0.021s
I20260709 13:02:53.200078  8825 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:53.210106  8826 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.010s	sys 0.021s
I20260709 13:02:53.184854  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:53.215837  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:53.176867 11213 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:53.147393 11372 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:53.134572 11316 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:53.187347 11398 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:53.207119  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:53.228546  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:53.196267 11367 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.006s	sys 0.021s
I20260709 13:02:53.168038 11352 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:02:53.186452 11369 raft_consensus.cc:2764] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:02:53.192956 11205 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:53.228497 11211 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:53.220959 11371 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:53.228263  9380 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:53.237993  8677 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:53.195524 11339 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:53.166996 11334 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:53.227974 11397 tablet_replica.cc:333] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:02:53.158595 11290 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:53.236115 11213 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:53.228916 11375 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:53.264292  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.248809  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:53.251274  8825 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:53.252004  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 416) took 1036 ms. Trace:
I20260709 13:02:53.236464 11256 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.230495 11265 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:53.272094  8753 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:53.272316  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 386) took 1699 ms. Trace:
I20260709 13:02:53.269745  8677 rpcz_store.cc:276] 0709 13:02:52.215596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.235463 (+1019867us) service_pool.cc:224] Handling call
0709 13:02:53.236860 (+  1397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.251546 (+ 14686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.251902 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:53.269363  8647 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:53.262840  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 157436us
I20260709 13:02:53.272780  8756 rpcz_store.cc:276] 0709 13:02:51.573104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.573304 (+   200us) service_pool.cc:224] Handling call
0709 13:02:51.604017 (+ 30713us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:51.702544 (+ 98527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:51.702580 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:51.703038 (+   458us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:51.703097 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.272046 (+1568949us) spinlock_profiling.cc:243] Waited 98.3 ms on lock 0x7b54002ac454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:53.272108 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.272251 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225988992}
I20260709 13:02:53.274765  8677 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:53.275063  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:53.231289 11197 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.001s	sys 0.017s
I20260709 13:02:53.250190 11211 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.262765 11205 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:53.292035  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:53.286000  8793 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:53.285044  8752 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:53.260841 11339 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:53.270431 11204 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.013s	sys 0.029s
I20260709 13:02:53.279906 11369 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:53.295604  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:53.261065 11411 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:53.279940 11197 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.002s	sys 0.017s
I20260709 13:02:53.266088 11213 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:53.301993  8823 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:53.304813  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:53.296591  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.302843 11204 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.013s	sys 0.029s
I20260709 13:02:53.275437 11375 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:53.292434 11211 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:53.296239 11205 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:53.299011 11275 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:53.330009  8719 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:53.300401 11339 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:53.297729 11352 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:53.286002 11208 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:02:53.303202 11284 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:53.303695 11369 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:53.346261  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:53.347216  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:53.281013 11256 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:53.348001  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:53.326560 11360 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:53.331722 11317 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.023s	sys 0.092s
I20260709 13:02:53.348423  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:53.328867 11272 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:53.365669 11272 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:53.366480 11272 consensus_queue.cc:1243] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:02:53.367035 11272 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:53.367574 11272 raft_consensus.cc:697] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:53.368553 11272 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:53.344635 11205 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:53.366848  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:53.360400 11317 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.024s	sys 0.093s
I20260709 13:02:53.364809 11408 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:53.375474 11408 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:53.376200 11357 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:53.376619 11408 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:53.360024  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:53.351637 11352 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.358057 11369 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:53.404461  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:53.410482  9386 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:53.353315 11221 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:53.414108 11405 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:02:53.353652 11413 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:53.385651 11360 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:53.395534 11357 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:53.393862 11367 consensus_queue.cc:1175] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e814ed2013244fa6be42623ea5c0c7b9" responder_term: 8 status { last_received { term: 6 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: [7.7-7.7]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:02:53.452279  8647 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.132:41749 timed out after 1.553s (SENT) [suppressed 14 similar messages]
I20260709 13:02:53.431496 11417 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:02:53.432791 11407 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
W20260709 13:02:53.433765 11302 consensus_meta.cc:287] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.924s	user 0.021s	sys 0.002s
I20260709 13:02:53.378002 11416 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:53.443936 11322 consensus_meta.cc:287] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.820s	user 0.015s	sys 0.009s
W20260709 13:02:53.445896 11412 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:02:53.447137 11334 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:53.325240 11371 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:53.455308 11385 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:53.447820 11316 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:53.409659 11408 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:53.458081 11301 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.846s	user 0.011s	sys 0.011s
I20260709 13:02:53.569463  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 380) took 2042 ms. Trace:
I20260709 13:02:53.431504 11221 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:53.932242 11425 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:53.878763 11418 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.933441 11302 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:53.744122 11213 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:53.825259 11405 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 13:02:53.531821  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.935055  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1674) took 1559 ms. Trace:
I20260709 13:02:53.950802  8754 rpcz_store.cc:276] 0709 13:02:52.376489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.376671 (+   182us) service_pool.cc:224] Handling call
0709 13:02:52.404600 (+ 27929us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:52.449023 (+ 44423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:52.449077 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:52.449688 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 13:02:52.655515 (+205827us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b18002c4ee0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:02:53.147685 (+492170us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:53.147733 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.147817 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.934600 (+786783us) raft_consensus.cc:1685] finished
0709 13:02:53.934664 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.934981 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43875,"raft.queue_time_us":46370,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":24894,"spinlock_wait_cycles":47089920,"thread_start_us":491949,"threads_started":1,"wal-append.queue_time_us":812661}
I20260709 13:02:53.459003  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:53.951138  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.954747 11418 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:53.943871 11417 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:02:53.930258  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 625548us
I20260709 13:02:53.958791 11417 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:53.787957  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.943480  9388 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:53.961014 11417 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:53.931479  8750 rpcz_store.cc:276] 0709 13:02:51.527403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.527588 (+   185us) service_pool.cc:224] Handling call
0709 13:02:51.533518 (+  5930us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:51.712416 (+178898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:51.712449 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:51.712511 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:51.712561 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.569043 (+1856482us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b540034b554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:53.569120 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.569366 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410953472}
W20260709 13:02:53.512449  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 60236.8us
I20260709 13:02:53.713747 11124 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:53.944800  9380 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:02:53.740598 11419 log.cc:927] Time spent T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:02:53.900303 11420 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.964527 11420 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:53.965516  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1582) took 1488 ms. Trace:
I20260709 13:02:53.943674 11322 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.967998 11420 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:53.967893 11322 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:53.693166  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.969266 11420 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:53.949833  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1590) took 1470 ms. Trace:
I20260709 13:02:53.971705  8822 rpcz_store.cc:276] 0709 13:02:52.478930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.550315 (+ 71385us) service_pool.cc:224] Handling call
0709 13:02:52.550946 (+   631us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:52.552897 (+  1951us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:52.552944 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:52.553623 (+   679us) log.cc:844] Serialized 26 byte log entry
0709 13:02:53.141920 (+588297us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:53.142285 (+   365us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.318364 (+176079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.318507 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.949581 (+631074us) raft_consensus.cc:1685] finished
0709 13:02:53.949606 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.949755 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1345,"prepare.queue_time_us":92849,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":6517,"raft.queue_time_us":31985,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":13124,"thread_start_us":729783,"threads_started":2,"wal-append.queue_time_us":626623}
I20260709 13:02:53.949651 11208 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:53.460264 11360 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:53.403981 11352 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:53.965958  8830 rpcz_store.cc:276] 0709 13:02:52.476589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.547741 (+ 71152us) service_pool.cc:224] Handling call
0709 13:02:52.557859 (+ 10118us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:52.558282 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:53.155908 (+597626us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.500678 (+344770us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:53.522110 (+ 21432us) log.cc:844] Serialized 26 byte log entry
0709 13:02:53.522332 (+   222us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:53.522396 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.522506 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.965249 (+442743us) raft_consensus.cc:1685] finished
0709 13:02:53.965270 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.965444 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25866,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":40,"raft.queue_time_us":98914,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":250,"thread_start_us":900484,"threads_started":3,"wal-append.queue_time_us":323082}
I20260709 13:02:53.980650  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:53.981835  8750 raft_consensus.cc:2410] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:53.985059  9386 raft_consensus.cc:2393] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 8: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:53.986770  8829 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:53.988615 11302 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:53.932893 11221 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:53.932961 11425 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.049418 11425 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:54.004483  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.052769  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:02:53.962252  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.961437  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:54.056237  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 94900.1us
I20260709 13:02:54.057067 11208 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.058115  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:54.057776 11208 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.059116 11272 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:54.059917 11272 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:54.061197 11208 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:54.061359 11272 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.069191  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:54.040861 11322 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:54.043781 11420 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:54.044871  8646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.025s	sys 0.000s
W20260709 13:02:54.100210  8646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.026s	sys 0.000s
I20260709 13:02:53.954223  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:54.030979 11204 consensus_queue.cc:1059] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:54.062266 11208 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.669552 11211 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:54.108131  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50132.6us
W20260709 13:02:54.109522  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 82891.6us
I20260709 13:02:53.501479 11357 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.130357  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:53.946177 11213 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:54.126741  9353 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:54.137284  8793 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:53.678982 11275 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:54.083001 11290 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:54.141830  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:54.111045 11124 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.120695 11208 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.170378  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:54.175937  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:54.177117  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:54.186873  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:54.121884 11322 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.191689  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:54.192893  9388 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:54.044720 11429 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.193840  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:54.109854 11385 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:54.139848 11301 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:53.960083 11418 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:54.132946 11416 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:54.215315 11301 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:54.184511 11316 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:54.233867 11316 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:54.233603  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:54.235527  9386 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:54.167909 11256 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:54.174819 11268 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:54.191217 11393 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.481s	user 0.008s	sys 0.012s
I20260709 13:02:54.236685 11268 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.237210 11268 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.209653  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:54.197340 11204 raft_consensus.cc:3055] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:54.190230 11213 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.234875 11316 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:54.193504 11124 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.234522  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:54.222417  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:54.229467  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:54.243912 11229 consensus_meta.cc:287] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.488s	user 0.019s	sys 0.007s
I20260709 13:02:54.278352 11328 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:54.274085  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:54.327834  8675 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:54.276258 11402 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 1.048s	user 0.010s	sys 0.012s
W20260709 13:02:54.248466 11205 consensus_meta.cc:287] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.869s	user 0.010s	sys 0.003s
I20260709 13:02:54.336107  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2090) took 1209 ms. Trace:
I20260709 13:02:54.338682  8752 rpcz_store.cc:276] 0709 13:02:53.126934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.160562 (+ 33628us) service_pool.cc:224] Handling call
0709 13:02:54.335764 (+1175202us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540031bd54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:54.336005 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":269980928}
I20260709 13:02:54.249305 11418 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:54.238402 11411 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.910s	user 0.004s	sys 0.005s
I20260709 13:02:54.325059  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:54.300958  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:54.261329 11204 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:54.288424 11302 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:54.379551  8825 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:02:54.357235 11343 consensus_meta.cc:287] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.659s	user 0.020s	sys 0.006s
I20260709 13:02:54.209822 11322 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.238456  9380 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:54.291929 11425 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.365823  9387 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:54.412137  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:54.400195  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1593) took 1274 ms. Trace:
I20260709 13:02:54.412873  8679 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:54.403482  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1594) took 1277 ms. Trace:
I20260709 13:02:54.415175  8825 rpcz_store.cc:276] 0709 13:02:53.125805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.337577 (+1211772us) service_pool.cc:224] Handling call
0709 13:02:54.379244 (+ 41667us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:54.403042 (+ 23798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.403397 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:54.369079 11411 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:54.290580 11213 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.340608 11211 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:54.336053 11385 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:54.404767 11438 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:02:54.412950  8830 rpcz_store.cc:276] 0709 13:02:53.125524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.979189 (+853665us) service_pool.cc:224] Handling call
0709 13:02:53.979696 (+   507us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:54.008059 (+ 28363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.008101 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:54.009029 (+   928us) log.cc:844] Serialized 52 byte log entry
0709 13:02:54.074052 (+ 65023us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:54.128071 (+ 54019us) log.cc:844] Serialized 14 byte log entry
0709 13:02:54.128358 (+   287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.128451 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.399856 (+271405us) raft_consensus.cc:1685] finished
0709 13:02:54.399890 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.400108 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27801,"prepare.queue_time_us":58538,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":6977,"raft.queue_time_us":29319,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":7429,"spinlock_wait_cycles":26386688,"thread_start_us":64493,"threads_started":1,"wal-append.queue_time_us":232830}
W20260709 13:02:54.424697 11422 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:02:54.426311 11208 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:54.426497 11406 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.882s	user 0.004s	sys 0.018s
I20260709 13:02:54.374239 11418 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:54.339778 11229 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:54.448879  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.227977 11429 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.355160 11205 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.478850  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:54.470520  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 434) took 1234 ms. Trace:
I20260709 13:02:54.480916  8830 rpcz_store.cc:276] 0709 13:02:53.236591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.430858 (+1194267us) service_pool.cc:224] Handling call
0709 13:02:54.469776 (+ 38918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.470030 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.470068 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:54.470129 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:54.470188 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.470271 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.470435 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:54.431519 11369 consensus_meta.cc:287] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.017s	user 0.007s	sys 0.007s
I20260709 13:02:54.472283  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1592) took 1992 ms. Trace:
I20260709 13:02:54.476550  8825 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:54.482934 11317 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.487170  8679 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:54.493017  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1595) took 1366 ms. Trace:
I20260709 13:02:54.493063  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1541) took 1087 ms. Trace:
I20260709 13:02:54.493953  8679 rpcz_store.cc:276] 0709 13:02:53.405781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.411830 (+1006049us) service_pool.cc:224] Handling call
0709 13:02:54.492967 (+ 81137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:54.482934 11290 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:54.489691  8826 rpcz_store.cc:276] 0709 13:02:52.479420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.578306 (+ 98886us) service_pool.cc:224] Handling call
0709 13:02:52.584552 (+  6246us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:52.585023 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:52.585055 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:52.585705 (+   650us) log.cc:844] Serialized 52 byte log entry
0709 13:02:53.210681 (+624976us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:53.210825 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.210937 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.216468 (+1005531us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:54.456791 (+240323us) raft_consensus.cc:1685] finished
0709 13:02:54.456814 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.472168 (+ 15354us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3937,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":6131,"spinlock_wait_cycles":16037760,"thread_start_us":624518,"threads_started":1,"wal-append.queue_time_us":698808}
I20260709 13:02:54.496860  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:54.497536  8830 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:54.405120 11428 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.493793  8825 rpcz_store.cc:276] 0709 13:02:53.126113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.416294 (+1290181us) service_pool.cc:224] Handling call
0709 13:02:54.476071 (+ 59777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.492565 (+ 16494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.492922 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:54.402021 11302 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.407191 11343 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.512845  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:54.513811  8826 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:02:54.458330 11268 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:54.492112 11328 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.506114 11317 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.507989  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 419) took 2102 ms. Trace:
I20260709 13:02:54.508396  8677 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:54.509895  8676 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:54.516746 11429 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:54.508527 11290 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.520015 11322 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:54.429168 11425 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.495447 11357 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:54.524266  8675 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:02:54.489228 11385 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:54.532195  8824 rpcz_store.cc:276] 0709 13:02:52.395521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.397146 (+  1625us) service_pool.cc:224] Handling call
0709 13:02:52.397508 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:52.426895 (+ 29387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:52.426936 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:52.441879 (+ 14943us) log.cc:844] Serialized 26 byte log entry
0709 13:02:53.145657 (+703778us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:53.145716 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.145808 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.275160 (+1129352us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:54.485094 (+209934us) raft_consensus.cc:1685] finished
0709 13:02:54.485118 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.497820 (+ 12702us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24190,"raft.queue_time_us":20791,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":8896,"spinlock_wait_cycles":6720768,"thread_start_us":664029,"threads_started":1,"wal-append.queue_time_us":714237}
I20260709 13:02:54.568212  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1539) took 1284 ms. Trace:
W20260709 13:02:54.537964  8644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.016s	user 0.007s	sys 0.041s
W20260709 13:02:54.536144 11339 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.144s	user 0.003s	sys 0.010s
W20260709 13:02:54.551577 11413 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.093s	user 0.013s	sys 0.038s
I20260709 13:02:54.556675  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 432) took 1452 ms. Trace:
I20260709 13:02:54.538472  8830 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
W20260709 13:02:54.559942  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.025s	sys 0.031s
I20260709 13:02:54.546633  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:54.564324  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 415) took 2349 ms. Trace:
W20260709 13:02:54.568881  8644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.047s	user 0.008s	sys 0.041s
W20260709 13:02:54.576558  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.026s	sys 0.031s
W20260709 13:02:54.577061  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.573994  8822 rpcz_store.cc:276] 0709 13:02:53.104031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.974417 (+870386us) service_pool.cc:224] Handling call
0709 13:02:53.976808 (+  2391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.977104 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:54.028775 (+ 51671us) log.cc:844] Serialized 14 byte log entry
0709 13:02:54.554359 (+525584us) log.cc:844] Serialized 14 byte log entry
0709 13:02:54.556097 (+  1738us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:54.556181 (+    84us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:54.556265 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.556373 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.556576 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21227,"prepare.queue_time_us":141732,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":4607,"raft.queue_time_us":32293,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":15396,"thread_start_us":525157,"threads_started":3,"wal-append.queue_time_us":152551}
I20260709 13:02:54.568624  8675 rpcz_store.cc:276] 0709 13:02:53.283955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.273659 (+989704us) service_pool.cc:224] Handling call
0709 13:02:54.568137 (+294478us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:54.580330  8675 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:54.581025  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1526) took 2101 ms. Trace:
W20260709 13:02:54.577109 11413 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.118s	user 0.013s	sys 0.039s
W20260709 13:02:54.570494  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 425027us
I20260709 13:02:54.572458 11421 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:54.581393  8675 rpcz_store.cc:276] 0709 13:02:52.479558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.579739 (+2100181us) service_pool.cc:224] Handling call
0709 13:02:54.580134 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.580759 (+   625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.580965 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:54.391813 11275 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.578156  8719 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:54.563621 11429 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.588109  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.488956 11204 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.586334  9355 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:54.571236 11322 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.589076  8722 connection.cc:256] RPC call timeout handler was delayed by -1.2172868049999579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1966s for a response
I20260709 13:02:54.575320 11124 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:54.579937 11357 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.602173  8722 connection.cc:256] RPC call timeout handler was delayed by -1.2058864379999932s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
W20260709 13:02:54.608881  8722 connection.cc:256] RPC call timeout handler was delayed by -1.1654572190000181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 13:02:54.609304  8722 connection.cc:256] RPC call timeout handler was delayed by -1.1227868049999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20710000000000003s for a response
W20260709 13:02:54.609688  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0617795929999829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
I20260709 13:02:54.544488 11328 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.603456  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:54.621830  8675 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:54.623528  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:54.584792 11413 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:54.546798 11418 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.528193 11428 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.500037 11369 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:54.642025 11398 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.386s	user 0.014s	sys 0.002s
I20260709 13:02:54.657421 11441 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:54.620151 11258 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:54.600749 11429 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.575980 11339 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.607342 11322 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.608446 11421 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.729646 11213 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:54.726003 11258 raft_consensus.cc:697] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:54.641563 11357 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.713752 11442 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.740207 11258 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.745381 11443 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:54.752457  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 625863us
I20260709 13:02:54.722429 11441 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:54.762075  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 150822us
I20260709 13:02:54.759707 11213 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.765205 11213 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.766343 11205 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:54.767274 11205 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:54.768182  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:54.768955  8675 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:54.771189  8675 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:02:54.772192  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1527) took 2292 ms. Trace:
I20260709 13:02:54.753986 11229 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:54.699988 11398 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:54.754154  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 560507us
I20260709 13:02:54.737524 11442 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.778466  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:54.780090  8793 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:54.781803  8793 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:54.785053  8793 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:54.783360  8675 rpcz_store.cc:276] 0709 13:02:52.479777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.770353 (+2290576us) service_pool.cc:224] Handling call
0709 13:02:54.770793 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:54.771832 (+  1039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.772114 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:54.764693  8722 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:54.759020 11441 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:54.789901 11264 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:54.790469 11213 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:54.790742 11415 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:54.791925 11204 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:54.824404 11399 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:54.792724  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1687) took 1435 ms. Trace:
I20260709 13:02:54.794433 11357 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:54.776904 11352 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.797s	user 0.015s	sys 0.000s
I20260709 13:02:54.773233 11205 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.859014 11339 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:54.860227 11205 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:54.758889 11413 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:54.870937 11316 consensus_meta.cc:287] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.553s	user 0.013s	sys 0.001s
I20260709 13:02:54.783151 11272 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:54.817844  8722 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:54.786213 11268 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:54.825696  8750 rpcz_store.cc:276] 0709 13:02:53.357825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.982787 (+624962us) service_pool.cc:224] Handling call
0709 13:02:53.983304 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:54.111192 (+127888us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.111233 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:54.112077 (+   844us) log.cc:844] Serialized 26 byte log entry
0709 13:02:54.564291 (+452214us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:54.564331 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.564403 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.792485 (+228082us) raft_consensus.cc:1685] finished
0709 13:02:54.792505 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.792650 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127345,"raft.queue_time_us":128346,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":15276,"spinlock_wait_cycles":321408,"thread_start_us":377325,"threads_started":1,"wal-append.queue_time_us":572155}
I20260709 13:02:54.828958  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:54.793733 11229 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.774648 11324 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:54.746239 11443 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:54.881572 11316 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:54.807101 11429 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:54.820446 11290 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:54.820900 11328 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:54.886654 11442 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:54.887725  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 58889.3us
I20260709 13:02:54.921048  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:54.841198 11343 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:54.899907  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 407) took 3288 ms. Trace:
I20260709 13:02:54.803674 11431 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:54.922958  8680 rpcz_store.cc:276] 0709 13:02:51.611215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.857699 (+246484us) service_pool.cc:224] Handling call
0709 13:02:51.873413 (+ 15714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.157072 (+283659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.899514 (+2742442us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54003a25d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:54.899807 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620663552}
I20260709 13:02:54.925559  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:54.931087 11371 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.017s	sys 0.038s
W20260709 13:02:54.932329 11371 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.017s	sys 0.038s
I20260709 13:02:54.936604  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1684) took 1580 ms. Trace:
I20260709 13:02:54.937255  8756 rpcz_store.cc:276] 0709 13:02:53.356659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.356832 (+   173us) service_pool.cc:224] Handling call
0709 13:02:53.364088 (+  7256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:53.399886 (+ 35798us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:53.399948 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:54.117086 (+717138us) log.cc:844] Serialized 26 byte log entry
0709 13:02:54.575851 (+458765us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:54.575896 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.575974 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.936266 (+360292us) raft_consensus.cc:1685] finished
0709 13:02:54.936295 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.936512 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35298,"raft.queue_time_us":35458,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":13164,"thread_start_us":378900,"threads_started":1,"wal-append.queue_time_us":760954}
W20260709 13:02:54.922932 11426 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:02:54.889942 11264 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:54.873732 11352 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:54.960752  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.962967  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:54.974179  8822 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:54.976116 11433 log.cc:927] Time spent T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:02:54.924546 11290 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:02:54.981344  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.983685 11445 log.cc:927] Time spent T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
W20260709 13:02:54.983867 11417 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.022s	user 0.011s	sys 0.008s
I20260709 13:02:54.895601 11229 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:54.948925 11442 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:55.041208  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:55.056082  8679 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 13:02:55.075062  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.930775 11268 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:54.987426  8826 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
W20260709 13:02:55.061057 11446 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:02:54.968832 11322 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:55.093686 11371 consensus_queue.cc:1059] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:02:55.102527  8680 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:02:55.103701  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1531) took 2614 ms. Trace:
I20260709 13:02:55.104265  8680 rpcz_store.cc:276] 0709 13:02:52.488948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.074687 (+2585739us) service_pool.cc:224] Handling call
0709 13:02:55.102206 (+ 27519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.103343 (+  1137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.103627 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:55.107753  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:55.108997  8826 raft_consensus.cc:2368] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6.
I20260709 13:02:55.111465 11352 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:55.113623 11371 raft_consensus.cc:3055] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:55.115166 11371 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:55.115738  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 422) took 2719 ms. Trace:
I20260709 13:02:55.116150 11371 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:55.116330  8827 rpcz_store.cc:276] 0709 13:02:52.396451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.407731 (+ 11280us) service_pool.cc:224] Handling call
0709 13:02:52.408079 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.287866 (+1879787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.287919 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:54.287981 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:54.288033 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.115329 (+827296us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b540009c9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:55.115405 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.115642 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4322746752}
I20260709 13:02:55.091470 11398 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:55.133684  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:55.106523 11397 raft_consensus.cc:2243] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:55.181275 11397 raft_consensus.cc:2272] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:55.141546 11375 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:55.182865 11385 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.609s	user 0.013s	sys 0.004s
W20260709 13:02:55.143126  8674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.032s	sys 0.042s
W20260709 13:02:55.185870  8674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.032s	sys 0.042s
I20260709 13:02:55.188659 11441 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:55.189792 11397 ts_tablet_manager.cc:1916] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:02:55.169649 11389 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.013s	sys 0.026s
W20260709 13:02:55.175571  8719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.031s	sys 0.074s
W20260709 13:02:55.166991 11138 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.047s	sys 0.036s
I20260709 13:02:55.198154  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 414) took 2983 ms. Trace:
W20260709 13:02:55.206449  8719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.031s	sys 0.074s
W20260709 13:02:55.206954  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 628913us
I20260709 13:02:54.964238 11431 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:55.214521  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:55.215663  8830 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:02:55.216962  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:55.217356  9355 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:55.217665  8830 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:55.206823  8673 rpcz_store.cc:276] 0709 13:02:52.214734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.412862 (+198128us) service_pool.cc:224] Handling call
0709 13:02:52.413224 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:52.413670 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:52.413718 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:52.414441 (+   723us) log.cc:844] Serialized 26 byte log entry
0709 13:02:54.751935 (+2337494us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b1800175300. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:02:54.934407 (+182472us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:54.934477 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.934600 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:55.197603 (+263003us) raft_consensus.cc:1685] finished
0709 13:02:55.197634 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.198049 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1229,"raft.queue_time_us":112,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":9265,"spinlock_wait_cycles":43191040,"thread_start_us":182221,"threads_started":1,"wal-append.queue_time_us":2778424}
W20260709 13:02:55.218683  8755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.037s	sys 0.053s
W20260709 13:02:55.229514  8755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.038s	sys 0.053s
I20260709 13:02:55.229667  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:55.223706  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:55.206565 11138 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.048s	sys 0.037s
I20260709 13:02:55.232587  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:55.228643  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:55.223469  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:55.181334  8825 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:55.233596  8673 raft_consensus.cc:2410] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:55.236909  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:55.239331  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:55.239992  8749 raft_consensus.cc:2410] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:02:55.241381  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:55.244931  8825 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:55.247839  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:55.251019  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:55.220255 11415 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:55.258744  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:55.266790 11452 log.cc:927] Time spent T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:02:55.228056 11258 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:55.225404  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:55.277262  8825 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:55.249497 11441 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:55.285895  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 468178us
I20260709 13:02:55.292709  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:02:55.293282  9353 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.017s	sys 0.029s
W20260709 13:02:55.293643  9353 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.017s	sys 0.029s
W20260709 13:02:55.293910  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 533111us
I20260709 13:02:55.297806  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:55.302132  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:55.303819  8830 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:55.307603  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:55.311141  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:55.311419  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:55.312144  8754 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:55.287091 11258 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.315984  8722 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:55.316540  8825 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:02:55.318034  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:55.318753  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1685) took 1961 ms. Trace:
I20260709 13:02:55.319339  8755 rpcz_store.cc:276] 0709 13:02:53.357087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.357236 (+   149us) service_pool.cc:224] Handling call
0709 13:02:53.357666 (+   430us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:53.406202 (+ 48536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:53.406253 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:54.071527 (+665274us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b2c000f0558. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e0df2a11 00007f32e0e02372 00007f32e2534a01 00007f32e24f943a 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2
0709 13:02:54.118207 (+ 46680us) log.cc:844] Serialized 52 byte log entry
0709 13:02:54.523232 (+405025us) spinlock_profiling.cc:243] Waited 6.35 ms on lock 0x7b48001e03d8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:02:54.570913 (+ 47681us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:54.641050 (+ 70137us) log.cc:844] Serialized 14 byte log entry
0709 13:02:55.229872 (+588822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.229955 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:55.318453 (+ 88498us) raft_consensus.cc:1685] finished
0709 13:02:55.318479 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.318669 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25105,"prepare.queue_time_us":548596,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":57,"raft.queue_time_us":297463,"raft.run_cpu_time_us":12131,"raft.run_wall_time_us":1111668,"spinlock_wait_cycles":1538975232,"thread_start_us":1024435,"threads_started":3,"wal-append.queue_time_us":785405}
I20260709 13:02:55.326766  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:55.313979  8830 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:02:55.314432  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 89196.9us
I20260709 13:02:55.327850  8755 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:02:55.333005  8830 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:02:55.335757  8830 pending_rounds.cc:85] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 3
I20260709 13:02:55.336161  8830 pending_rounds.cc:107] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:02:55.336476  8830 raft_consensus.cc:2889] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:55.337774  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:55.343570  8822 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:55.338272  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:55.347577  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:55.317551 11258 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:55.352242  8722 connection.cc:256] RPC call timeout handler was delayed by -1.7482675180000342s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.255s for a response
I20260709 13:02:55.347765 11467 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:55.354197  8719 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:55.339468 11375 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:55.339062  8830 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:60438
W20260709 13:02:55.365461  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.008s	sys 0.046s
I20260709 13:02:55.323052 11389 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:55.346230 11229 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:55.316237 11443 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:55.356213  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:55.366657  8830 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:02:55.366879  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.008s	sys 0.047s
I20260709 13:02:55.363286 11138 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:55.370119  8830 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:55.370517  8822 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:55.360592 11467 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.359305 11266 consensus_queue.cc:1059] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:02:55.380673 11389 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:55.372246 11458 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
W20260709 13:02:55.380605 11408 consensus_meta.cc:287] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.301s	user 0.008s	sys 0.012s
I20260709 13:02:55.370203 11375 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:55.436932 11375 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:55.438241 11276 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:55.380702 11138 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:02:55.387885 11467 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:55.386606  8755 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:02:55.386510  8830 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:55.367157 11390 consensus_queue.cc:1059] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:02:55.481223  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:55.482504  8755 raft_consensus.cc:2410] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:02:55.489930  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.522388  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1532) took 3033 ms. Trace:
I20260709 13:02:55.448269 11138 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:55.522997  8680 rpcz_store.cc:276] 0709 13:02:52.490675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.112699 (+2622024us) service_pool.cc:224] Handling call
0709 13:02:55.174509 (+ 61810us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.179267 (+  4758us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:55.357942 (+178675us) log.cc:844] Serialized 14 byte log entry
0709 13:02:55.521895 (+163953us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:55.521968 (+    73us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:55.522034 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.522121 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.522308 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53347,"prepare.queue_time_us":279294,"thread_start_us":286383,"threads_started":2}
W20260709 13:02:55.458434 11399 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.623s	user 0.015s	sys 0.006s
I20260709 13:02:55.571214  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:55.363915 11415 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:55.468750 11431 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:55.854533  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.862779  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1529) took 3374 ms. Trace:
I20260709 13:02:55.863293  8681 rpcz_store.cc:276] 0709 13:02:52.488348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.853293 (+2364945us) service_pool.cc:224] Handling call
0709 13:02:54.865993 (+ 12700us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:54.866478 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.866512 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:54.867230 (+   718us) log.cc:844] Serialized 52 byte log entry
0709 13:02:55.274776 (+407546us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:55.274842 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.274970 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:55.862458 (+587488us) raft_consensus.cc:1685] finished
0709 13:02:55.862481 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.862685 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5115,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":133,"thread_start_us":406989,"threads_started":1,"wal-append.queue_time_us":454882}
W20260709 13:02:55.879448  8750 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.037s	sys 0.100s
W20260709 13:02:55.880115  8750 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.037s	sys 0.100s
I20260709 13:02:55.565131 11266 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:55.571086 11458 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.626196 11399 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:55.908901 11322 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.987476 11322 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:56.014839 11322 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:56.059455  9352 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.005s	sys 0.005s
W20260709 13:02:56.060125  9352 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.005s	sys 0.005s
W20260709 13:02:56.095929 11476 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:02:55.912760 11266 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:55.980460 11458 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:56.116480 11458 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:56.007534 11229 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.003s	sys 0.015s
W20260709 13:02:56.060495  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 532543us
I20260709 13:02:56.336853  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1528) took 3849 ms. Trace:
I20260709 13:02:56.468714  8675 rpcz_store.cc:276] 0709 13:02:52.488026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.820703 (+2332677us) service_pool.cc:224] Handling call
0709 13:02:54.821304 (+   601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:54.821782 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.821823 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:54.822361 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 13:02:55.283982 (+461621us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:55.285487 (+  1505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.285775 (+   288us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.336479 (+1050704us) raft_consensus.cc:1685] finished
0709 13:02:56.336512 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.336751 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":948,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":23316,"thread_start_us":356446,"threads_started":1,"wal-append.queue_time_us":491988}
I20260709 13:02:56.310050 11399 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:56.122713 11054 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:56.357605 11204 consensus_meta.cc:287] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.531s	user 0.015s	sys 0.001s
I20260709 13:02:56.471298 11399 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:56.440713 11266 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:56.475878  8675 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:56.477234  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1536) took 3895 ms. Trace:
I20260709 13:02:56.477833  8675 rpcz_store.cc:276] 0709 13:02:52.581594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.475120 (+3893526us) service_pool.cc:224] Handling call
0709 13:02:56.475604 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.476809 (+  1205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.477142 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:56.479099 11467 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.011s	user 0.013s	sys 0.000s
I20260709 13:02:56.479749 11399 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:56.480587 11399 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:56.481328 11399 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:56.484597 11399 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:56.491866 11467 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:56.495443 11229 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.056s	user 0.012s	sys 0.016s
I20260709 13:02:55.859740 11389 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:56.461604  8680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.005s	sys 0.017s
W20260709 13:02:56.529510  8680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.005s	sys 0.017s
I20260709 13:02:56.123260  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1374) took 1151 ms. Trace:
W20260709 13:02:56.426496 11462 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 13:02:56.501461  8722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.990s	user 0.005s	sys 0.029s
W20260709 13:02:56.531149  8722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.005s	sys 0.029s
W20260709 13:02:56.534216  8722 connection.cc:256] RPC call timeout handler was delayed by -1.1848330150000947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19720000000000001s for a response
I20260709 13:02:56.531159  8825 rpcz_store.cc:276] 0709 13:02:54.971608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.317440 (+345832us) service_pool.cc:224] Handling call
0709 13:02:55.318074 (+   634us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.329413 (+ 11339us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:55.329470 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:55.330074 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 13:02:55.367625 (+ 37551us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:55.367680 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.367752 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.122966 (+755214us) raft_consensus.cc:1685] finished
0709 13:02:56.122996 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.123180 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":22686,"thread_start_us":19993,"threads_started":1,"wal-append.queue_time_us":559166}
W20260709 13:02:56.612337  8722 connection.cc:256] RPC call timeout handler was delayed by -1.1353330150000147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20270000000000002s for a response
W20260709 13:02:56.517612  9353 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.015s	sys 0.019s
W20260709 13:02:56.613161  9353 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.015s	sys 0.020s
I20260709 13:02:56.572849  8830 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:56.571878 11258 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.207s	user 0.016s	sys 0.000s
W20260709 13:02:56.612771  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0453330150000966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
W20260709 13:02:56.624650  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0453330150000966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
I20260709 13:02:56.667990  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:60438 (request call id 1379) took 1550 ms. Trace:
I20260709 13:02:56.668640  8830 rpcz_store.cc:276] 0709 13:02:55.077312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.338887 (+261575us) service_pool.cc:224] Handling call
0709 13:02:56.614814 (+1275927us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:56.699288 11390 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.096s	user 0.017s	sys 0.005s
W20260709 13:02:56.719534 11390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.145s	user 0.017s	sys 0.006s
I20260709 13:02:56.740170 11467 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:56.777011  8646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.004s	sys 0.018s
W20260709 13:02:56.778046  8646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.004s	sys 0.018s
I20260709 13:02:56.780643  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1535) took 4231 ms. Trace:
W20260709 13:02:56.781662  8678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2000) took 4712 ms (client timeout 1964 ms). Trace:
W20260709 13:02:56.782349  8678 rpcz_store.cc:269] 0709 13:02:52.069006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.084074 (+ 15068us) service_pool.cc:224] Handling call
0709 13:02:56.780895 (+4696821us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x7b54002a99d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:56.781274 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49566976}
I20260709 13:02:56.784077  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1523) took 4326 ms. Trace:
I20260709 13:02:56.784631  8679 rpcz_store.cc:276] 0709 13:02:52.457983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.503198 (+2045215us) service_pool.cc:224] Handling call
0709 13:02:54.507423 (+  4225us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.075607 (+568184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.783230 (+1707623us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b54003a20d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:56.783803 (+   573us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1261009664}
I20260709 13:02:56.787812  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1544) took 2196 ms. Trace:
W20260709 13:02:56.788945  8676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 417) took 4332 ms (client timeout 3139 ms). Trace:
W20260709 13:02:56.795274  8676 rpcz_store.cc:269] 0709 13:02:52.456925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.865220 (+408295us) service_pool.cc:224] Handling call
0709 13:02:56.788369 (+3923149us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400175054. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:56.788659 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3246897408}
I20260709 13:02:56.798342  8676 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:56.806389  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1546) took 1598 ms. Trace:
I20260709 13:02:56.807104  8676 rpcz_store.cc:276] 0709 13:02:55.208374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.797003 (+1588629us) service_pool.cc:224] Handling call
0709 13:02:56.797782 (+   779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.805917 (+  8135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.806277 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:54.582719  8672 rpcz_store.cc:276] 0709 13:02:52.215225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.432305 (+217080us) service_pool.cc:224] Handling call
0709 13:02:52.432642 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.432868 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:52.441441 (+  8573us) log.cc:844] Serialized 14 byte log entry
0709 13:02:52.542791 (+101350us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b180008c1a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:02:54.055873 (+1513082us) spinlock_profiling.cc:243] Waited 2.84 ms on lock 0x7b180008c1a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e5ef13c7 00007f32e25039a7 00007f32e25130bf 00007f32e2513038
0709 13:02:54.128435 (+ 72562us) log.cc:844] Serialized 14 byte log entry
0709 13:02:54.563891 (+435456us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:54.563956 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:54.564019 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.564099 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.564260 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1526224,"prepare.queue_time_us":1630993,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":82,"raft.queue_time_us":27600,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":2399,"spinlock_wait_cycles":12578432,"thread_start_us":430016,"threads_started":1}
W20260709 13:02:56.804656  8753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 389) took 4221 ms (client timeout 3139 ms). Trace:
I20260709 13:02:56.817901  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 419) took 4360 ms. Trace:
I20260709 13:02:56.781562  8681 rpcz_store.cc:276] 0709 13:02:52.549019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.889956 (+3340937us) service_pool.cc:224] Handling call
0709 13:02:55.909166 (+ 19210us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:55.909621 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.909675 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:56.121067 (+211392us) log.cc:844] Serialized 52 byte log entry
0709 13:02:56.779349 (+658282us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:56.779549 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.779725 (+   176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.779842 (+   117us) raft_consensus.cc:1685] finished
0709 13:02:56.779873 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.780296 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3571,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":37183,"spinlock_wait_cycles":800128,"thread_start_us":440890,"threads_started":1,"wal-append.queue_time_us":606841}
I20260709 13:02:56.871520  8677 rpcz_store.cc:276] 0709 13:02:52.457638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.276665 (+819027us) service_pool.cc:224] Handling call
0709 13:02:53.276959 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.532347 (+1255388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.817144 (+2284797us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400353c54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:56.817589 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2831741824}
I20260709 13:02:56.874125  8677 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:02:56.818967  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.875267  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.875767  8672 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:56.831590  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:56.876459  8748 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:02:56.835927  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2086) took 3713 ms. Trace:
I20260709 13:02:56.841045  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:56.883229  8824 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:56.786266  8678 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:56.844859  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 502) took 4858 ms. Trace:
I20260709 13:02:56.845278  9380 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:56.846458  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:56.875222  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1551) took 1451 ms. Trace:
I20260709 13:02:56.876955  8827 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:56.833240  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 415) took 6148 ms. Trace:
I20260709 13:02:56.819907  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1517) took 4649 ms. Trace:
I20260709 13:02:56.886590  8677 rpcz_store.cc:276] 0709 13:02:55.423701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.872981 (+1449280us) service_pool.cc:224] Handling call
0709 13:02:56.873529 (+   548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.874857 (+  1328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.875141 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.888446  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:56.667178 11258 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:56.889968  9383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 515) took 1553 ms. Trace:
W20260709 13:02:56.877379  8753 rpcz_store.cc:269] 0709 13:02:52.583133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.583259 (+   126us) service_pool.cc:224] Handling call
0709 13:02:56.803802 (+4220543us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54001572d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:56.804232 (+   430us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1533830144}
I20260709 13:02:56.876587  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1549) took 1665 ms. Trace:
I20260709 13:02:56.881222  8679 rpcz_store.cc:276] 0709 13:02:54.591727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.786084 (+2194357us) service_pool.cc:224] Handling call
0709 13:02:56.786775 (+   691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.787306 (+   531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.787363 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:56.787466 (+   103us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:56.787513 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.787578 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.787710 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:56.700312 11375 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.003s	sys 0.015s
I20260709 13:02:56.915907  8672 rpcz_store.cc:276] 0709 13:02:55.211255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.869125 (+1657870us) service_pool.cc:224] Handling call
0709 13:02:56.869897 (+   772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.876243 (+  6346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.876495 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.882803  8751 rpcz_store.cc:276] 0709 13:02:53.122824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.124364 (+  1540us) service_pool.cc:224] Handling call
0709 13:02:53.124737 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.260511 (+135774us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:53.260561 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:53.260626 (+    65us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:53.260688 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.835244 (+3574556us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b540018d3d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:56.835334 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.835640 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308153728}
I20260709 13:02:56.914762  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1540) took 3491 ms. Trace:
I20260709 13:02:56.976059  8678 rpcz_store.cc:276] 0709 13:02:53.393669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.784470 (+3390801us) service_pool.cc:224] Handling call
0709 13:02:56.785373 (+   903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.884083 (+ 98710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.884734 (+   651us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:56.913853  8824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1367) took 1956 ms (client timeout 2071 ms). Trace:
I20260709 13:02:56.890596  9383 rpcz_store.cc:276] 0709 13:02:55.335995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.336161 (+   166us) service_pool.cc:224] Handling call
0709 13:02:56.884658 (+1548497us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:56.889821 (+  5163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6930176}
I20260709 13:02:56.993155 11258 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:56.989997 11399 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.998019 11399 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:57.002303 11399 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:57.003491 11399 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:57.004798 10965 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:57.004415 11399 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:57.005964 11210 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.007282 11197 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:57.006992 11375 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.008s	sys 0.015s
W20260709 13:02:56.980628  8824 rpcz_store.cc:269] 0709 13:02:54.927149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.927286 (+   137us) service_pool.cc:224] Handling call
0709 13:02:56.840357 (+1913071us) spinlock_profiling.cc:243] Waited 982 us on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:02:56.883843 (+ 43486us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2277632}
I20260709 13:02:56.884871  9384 rpcz_store.cc:276] 0709 13:02:51.986599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.987612 (+  1013us) service_pool.cc:224] Handling call
0709 13:02:52.023136 (+ 35524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:52.519852 (+496716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.844010 (+4324158us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b54004b64d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:56.844561 (+   551us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1017009920}
I20260709 13:02:57.065904  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1550) took 1692 ms. Trace:
I20260709 13:02:57.071132  8681 rpcz_store.cc:276] 0709 13:02:55.368501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.871920 (+1503419us) service_pool.cc:224] Handling call
0709 13:02:56.872537 (+   617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.873172 (+   635us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:56.873764 (+   592us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.060006 (+186242us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:57.060096 (+    90us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:57.060182 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.060299 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.060527 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4125,"prepare.queue_time_us":176911,"thread_start_us":174537,"threads_started":1}
I20260709 13:02:57.054720 11197 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:57.119889  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1534) took 4580 ms. Trace:
I20260709 13:02:57.121155 11332 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.121424 11484 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:57.122246 11332 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:57.124205 11443 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.007s	sys 0.016s
W20260709 13:02:57.124841 11443 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.007s	sys 0.016s
I20260709 13:02:57.125681 11332 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:57.126091 11413 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.007s	sys 0.027s
W20260709 13:02:57.126758 11413 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.008s	sys 0.027s
W20260709 13:02:57.126888 11138 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.389s	user 0.014s	sys 0.005s
I20260709 13:02:57.128000  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:57.119692  8794 connection.cc:256] RPC call timeout handler was delayed by -2.838070369000036s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
I20260709 13:02:57.129164  8825 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
W20260709 13:02:57.129416  8794 connection.cc:256] RPC call timeout handler was delayed by -2.7741703690000463s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
W20260709 13:02:57.129964  8794 connection.cc:256] RPC call timeout handler was delayed by -2.6126888590000021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
I20260709 13:02:57.127784 11138 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:57.132728 11138 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:57.134475  8647 connection.cc:256] RPC call timeout handler was delayed by -2.397422388999928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2042s for a response
I20260709 13:02:57.099130 11210 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:57.136530  8647 connection.cc:256] RPC call timeout handler was delayed by -2.0485196209999117s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2556s for a response
W20260709 13:02:57.137126  8647 connection.cc:256] RPC call timeout handler was delayed by -1.9083210749999466s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2882s for a response
W20260709 13:02:57.137440  8647 connection.cc:256] RPC call timeout handler was delayed by -1.7877733429999125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
W20260709 13:02:57.137934  8647 connection.cc:256] RPC call timeout handler was delayed by -1.0307625489999737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
I20260709 13:02:57.122538 11484 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:56.886305  8674 rpcz_store.cc:276] 0709 13:02:52.170140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.384023 (+213883us) service_pool.cc:224] Handling call
0709 13:02:52.384340 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.444344 (+ 60004us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:54.535137 (+2090793us) log.cc:844] Serialized 14 byte log entry
0709 13:02:55.309822 (+774685us) log.cc:844] Serialized 14 byte log entry
0709 13:02:55.310069 (+   247us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:55.310121 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:55.310187 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.819263 (+1509076us) spinlock_profiling.cc:243] Waited 59.7 ms on lock 0x7b54003cfd54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:56.819360 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.819670 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1666807,"prepare.queue_time_us":2309666,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":101,"raft.queue_time_us":47033,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":18245,"spinlock_wait_cycles":137370880,"thread_start_us":1244533,"threads_started":3,"wal-append.queue_time_us":660823,"wal-append.run_cpu_time_us":2883,"wal-append.run_wall_time_us":1172455}
W20260709 13:02:57.173803  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.138561 11210 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:57.128000  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 502325us
W20260709 13:02:57.187232  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0354330150000806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2138s for a response
W20260709 13:02:57.187844  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0354330150000806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2138s for a response
I20260709 13:02:57.144939 11502 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.147068 11372 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:56.886039  8828 rpcz_store.cc:276] 0709 13:02:50.691850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.035257 (+343407us) service_pool.cc:224] Handling call
0709 13:02:51.035763 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:51.048258 (+ 12495us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:51.048674 (+   416us) log.cc:844] Serialized 14 byte log entry
0709 13:02:51.305562 (+256888us) log.cc:844] Serialized 14 byte log entry
0709 13:02:52.349809 (+1044247us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b18003cc580. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:02:52.349968 (+   159us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:52.350024 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:52.350084 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.468476 (+118392us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b18003cc580. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240
0709 13:02:52.468536 (+    60us) spinlock_profiling.cc:243] Waited 5.44 ms on lock 0x7b18003cc580. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:02:52.583711 (+115175us) spinlock_profiling.cc:243] Waited 7.21 ms on lock 0x7b18003cc580. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1
0709 13:02:52.584148 (+   437us) spinlock_profiling.cc:243] Waited 8.1 ms on lock 0x7b18003cc580. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:02:56.832662 (+4248514us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b54002292d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:56.832746 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.833009 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1025141,"prepare.queue_time_us":5230,"prepare.run_cpu_time_us":156,"prepare.run_wall_time_us":10179,"raft.queue_time_us":1049506,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":10936,"spinlock_wait_cycles":202737792,"thread_start_us":256254,"threads_started":1,"wal-append.queue_time_us":293046,"wal-append.run_cpu_time_us":1659,"wal-append.run_wall_time_us":1028409}
I20260709 13:02:57.199051 11284 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:57.200388 11284 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:57.201241  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:57.120549  8680 rpcz_store.cc:276] 0709 13:02:52.512174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.607694 (+3095520us) service_pool.cc:224] Handling call
0709 13:02:55.608204 (+   510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.608521 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:55.915761 (+307240us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.775816 (+860055us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.777641 (+  1825us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.050728 (+273087us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:57.056119 (+  5391us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:57.070414 (+ 14295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.073778 (+  3364us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.092889 (+ 19111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":281060,"prepare.queue_time_us":491065,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":458975,"raft.run_cpu_time_us":589,"raft.run_wall_time_us":22346,"spinlock_wait_cycles":21094144,"thread_start_us":871289,"threads_started":2,"wal-append.queue_time_us":665813}
I20260709 13:02:57.225566  8827 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
W20260709 13:02:57.203284  8750 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.083s	user 0.014s	sys 0.017s
I20260709 13:02:57.203070  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:57.207118  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 256596us
W20260709 13:02:57.319203  8794 connection.cc:256] RPC call timeout handler was delayed by -2.5307888590000402s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
I20260709 13:02:57.326417  8822 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:57.336555  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:57.209067  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:57.338157  8794 connection.cc:256] RPC call timeout handler was delayed by -2.3750888590000159s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19910000000000003s for a response
I20260709 13:02:57.204483  8751 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:57.338830  8794 connection.cc:256] RPC call timeout handler was delayed by -2.3192888590000393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
I20260709 13:02:57.204044  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:57.339989  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:57.340375  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:57.340903  8828 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:57.341910  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:57.342548  8752 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:02:57.343015  8752 pending_rounds.cc:85] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 2
I20260709 13:02:57.343070 11503 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.343436  8752 pending_rounds.cc:107] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:02:57.343900  8752 raft_consensus.cc:2889] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:57.344017 11503 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:57.206888 11321 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.347487  8825 raft_consensus.cc:2410] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:02:57.206772  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 54256.9us
I20260709 13:02:57.349933  8828 raft_consensus.cc:2410] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:02:57.350478  8794 connection.cc:256] RPC call timeout handler was delayed by -1.515070369s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3189s for a response
I20260709 13:02:57.203131  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:57.351146  8794 connection.cc:256] RPC call timeout handler was delayed by -1.238388858999997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3254s for a response
I20260709 13:02:57.209697  8673 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
W20260709 13:02:57.324551 11495 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.089s	user 0.002s	sys 0.000s
I20260709 13:02:57.210353  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:57.203089 11210 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.152s
I20260709 13:02:57.354276 11321 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:57.207895  8645 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.971s	user 0.008s	sys 0.008s
W20260709 13:02:57.202932 11415 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.017s	sys 0.008s
I20260709 13:02:57.203519 11502 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:57.574775  8794 leader_election.cc:400] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 2. Current term is 3.
W20260709 13:02:57.352811  8827 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1381) took 2233 ms (client timeout 2226 ms). Trace:
I20260709 13:02:57.439723  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:57.502437  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.436084  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:57.446775  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:57.501869  8824 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
W20260709 13:02:57.397338  8750 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.377s	user 0.014s	sys 0.024s
W20260709 13:02:57.589354  8827 rpcz_store.cc:269] 0709 13:02:55.101499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.830766 (+1729267us) service_pool.cc:224] Handling call
0709 13:02:57.334685 (+503919us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:57.587070  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 383331us
I20260709 13:02:57.219509 11493 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.218529 11317 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:57.219290 11382 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:57.547861  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:57.607753  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:02:57.598733 11415 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.110s	user 0.021s	sys 0.016s
I20260709 13:02:57.369078  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:57.415709 11484 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:57.580826  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:57.598296  8645 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.477s	user 0.013s	sys 0.015s
W20260709 13:02:57.574487 11404 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 4.157s	user 0.021s	sys 0.039s
I20260709 13:02:57.365797  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:57.626371  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:02:57.619517  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 399643us
I20260709 13:02:57.633194  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:57.586014 11369 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:57.545182 11487 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:57.372741 11428 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:57.636837  8748 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1680) took 5026 ms (client timeout 2042 ms). Trace:
W20260709 13:02:57.438822 11513 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.068s	user 0.002s	sys 0.000s
W20260709 13:02:57.637286  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 225018us
I20260709 13:02:57.208307 11399 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:57.362576 11372 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.276s
I20260709 13:02:57.638936  8747 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:57.365185 11138 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.277s
I20260709 13:02:57.658320  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:57.372998 11504 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.435149 10965 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:57.667064 11504 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:57.667173 10965 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:57.429807 11503 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:57.202765 11443 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.433s
I20260709 13:02:57.673827  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:57.366979 11425 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:57.631691 11382 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:57.489688 11499 consensus_queue.cc:1059] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.148s
I20260709 13:02:57.649488  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:57.373390 11416 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:57.650971  8828 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:57.653280 11372 raft_consensus.cc:3055] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:57.654541 11502 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:57.654605  8753 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:57.655404  8749 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:02:57.656400  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 178882us
I20260709 13:02:57.656090 11342 consensus_queue.cc:1059] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:57.362805 11330 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.279s
W20260709 13:02:57.643567  8748 rpcz_store.cc:269] 0709 13:02:52.609915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.610042 (+   127us) service_pool.cc:224] Handling call
0709 13:02:57.620490 (+5010448us) spinlock_profiling.cc:243] Waited 4.27 s on lock 0x7b54002fd5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:57.636743 (+ 16253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9809732864}
I20260709 13:02:57.657466 11221 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:57.660218  8822 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:57.660241  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:57.663030  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:57.663371  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:57.594203 11397 ts_tablet_manager.cc:1929] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
W20260709 13:02:57.663957  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 1.54328e+06us
I20260709 13:02:57.664162  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:57.666088 11321 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:57.666273 11276 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:57.683229  8825 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:57.685408  8747 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:02:57.686061  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:57.801986 11372 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:57.820382 11372 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:57.821921 11372 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:57.822885 11466 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:57.796880  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 394) took 2499 ms. Trace:
I20260709 13:02:57.823647 11428 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.824535 11428 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:57.825088 11276 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:57.823943  8750 rpcz_store.cc:276] 0709 13:02:55.297685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.298547 (+   862us) service_pool.cc:224] Handling call
0709 13:02:55.299795 (+  1248us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:55.300342 (+   547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.300384 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:55.307458 (+  7074us) log.cc:844] Serialized 52 byte log entry
0709 13:02:55.880736 (+573278us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:57.623750 (+1743014us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.795893 (+172143us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.796268 (+   375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.796415 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:57.796497 (+    82us) raft_consensus.cc:1685] finished
0709 13:02:57.796512 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.796738 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1605887,"prepare.run_cpu_time_us":165,"prepare.run_wall_time_us":13081,"raft.queue_time_us":14625,"raft.run_cpu_time_us":443,"raft.run_wall_time_us":10529,"thread_start_us":2180567,"threads_started":3,"wal-append.queue_time_us":1035835,"wal-append.run_cpu_time_us":3084,"wal-append.run_wall_time_us":1112611}
I20260709 13:02:57.819046  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:02:57.718279  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 49037.7us
W20260709 13:02:57.747961  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 320446us
I20260709 13:02:57.689139  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:57.725889  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:57.797340  8749 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:57.834018  8752 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 3: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:57.818601  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:57.839295  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:57.841348  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:57.842815  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:57.835855 11428 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:57.847052  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:57.818011 11330 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:57.850554  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:57.703769 11284 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:57.851579  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:57.824549 11418 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.856055  8824 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:57.685333 11138 consensus_queue.cc:1243] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:02:57.863626  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.865401  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:57.647262 11493 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:57.869289  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:57.852901 11397 log.cc:1199] T 51388ea026cc4d098949a4a0327a35fc P e814ed2013244fa6be42623ea5c0c7b9: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root/wals/51388ea026cc4d098949a4a0327a35fc
W20260709 13:02:57.869534  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 91067.2us
I20260709 13:02:57.851497 11504 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:57.850726 11330 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:57.871279  9388 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:57.907578  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:57.898779 11515 raft_consensus.cc:3055] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:57.899726  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:57.903846  8749 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:57.900915  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:57.898972  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:57.872263  9384 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:57.876920  8674 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:57.873813  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:57.904927 11523 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.907125 11197 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:57.916672  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:57.898195 11483 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:02:57.938082  8680 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:57.915918 10965 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:57.774327 11382 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:57.943573  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:57.927062 11515 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:57.996016  8828 raft_consensus.cc:2376] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6. Re-sending same reply.
W20260709 13:02:58.023048  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 78419.3us
I20260709 13:02:58.025817  8824 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:58.026103  8677 raft_consensus.cc:2410] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:58.029071  8749 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:58.033879 11506 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:58.035871 11397 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60414 (request call id 78) took 5640 ms. Trace:
I20260709 13:02:58.036598 11397 rpcz_store.cc:276] 0709 13:02:52.395105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.396955 (+  1850us) service_pool.cc:224] Handling call
0709 13:02:53.159012 (+762057us) ts_tablet_manager.cc:1151] Deleting tablet 51388ea026cc4d098949a4a0327a35fc
0709 13:02:53.192179 (+ 33167us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:02:56.445502 (+3253323us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:57.566265 (+1120763us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:58.035692 (+469427us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1533710,"tablet-delete.queue_time_us":705690,"thread_start_us":218807,"threads_started":1}
I20260709 13:02:58.038877  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1364) took 2994 ms. Trace:
I20260709 13:02:58.039498  8821 rpcz_store.cc:276] 0709 13:02:54.924532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.924685 (+   153us) service_pool.cc:224] Handling call
0709 13:02:54.925072 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:57.210484 (+2285412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.210571 (+    87us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:57.211533 (+   962us) log.cc:844] Serialized 26 byte log entry
0709 13:02:57.336546 (+125013us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:57.595151 (+258605us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.658340 (+ 63189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.898989 (+240649us) spinlock_profiling.cc:243] Waited 2.21 s on lock 0x7b5400126954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:57.899048 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:57.899099 (+    51us) raft_consensus.cc:1685] finished
0709 13:02:57.918740 (+ 19641us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.919253 (+   513us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98146,"prepare.queue_time_us":498467,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":298918,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":346,"spinlock_wait_cycles":5081959808,"thread_start_us":348595,"threads_started":2,"wal-append.queue_time_us":417435}
W20260709 13:02:58.046741  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 196706us
I20260709 13:02:58.047101 11330 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:58.048388  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:58.048890 11330 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:58.049813 11001 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:58.029713  8676 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:02:58.050552 11001 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:58.057859  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 45650us
I20260709 13:02:57.927570 11416 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:58.088282  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.088608  8830 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:02:57.926306 11418 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:58.011811 11197 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:58.123571  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:58.123411  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:58.125623  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:58.029153 11504 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:58.130154  8828 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
W20260709 13:02:58.112141  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 63981us
I20260709 13:02:58.124054 11506 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:58.144213  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:58.146907  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 287371us
I20260709 13:02:58.149619 11197 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:58.127341 11382 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:58.152925  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:58.161441  8828 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:58.121758  8827 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:58.126020 11124 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:02:58.159634  8822 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:02:58.225843  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:58.241439  8749 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:58.248353 11197 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:58.251086  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 85157.2us
I20260709 13:02:58.269833  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:58.270146  8677 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:02:58.273255 11504 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:58.273937 11418 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:58.275789 11504 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:58.271697  8677 pending_rounds.cc:85] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 5
I20260709 13:02:58.276955  8677 pending_rounds.cc:107] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:02:58.277254  8677 raft_consensus.cc:2889] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:58.276782  8676 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:58.289234  8680 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:02:58.055650 11523 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:02:58.283846  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 122646us
W20260709 13:02:58.292133  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 59613.1us
I20260709 13:02:58.296414  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:58.299329  8824 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:02:58.240617  9380 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:58.311003  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:58.314749  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:58.316043  8794 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:58.324476  8677 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:02:58.330976  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:58.336249  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:58.337402  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:58.341171  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:58.340410  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:58.346544  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:58.348006  8680 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:58.349006  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:58.349620  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:58.350523  8676 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 2.
I20260709 13:02:58.348539  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:58.347000  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:58.352517  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:58.341898  8750 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:02:58.361361  8677 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:58.369872  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:58.372372  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:58.383057  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:58.383766  8827 raft_consensus.cc:2376] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6. Re-sending same reply.
I20260709 13:02:58.385677  8794 leader_election.cc:400] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8.
I20260709 13:02:58.386088  8794 leader_election.cc:403] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:58.372117  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:58.391557  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:58.365135 11372 consensus_meta.cc:287] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.009s
I20260709 13:02:58.393468  8823 raft_consensus.cc:2410] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:58.395092  8748 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:02:58.361824 11416 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:58.400990  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:58.403252  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:58.392446  8824 raft_consensus.cc:2376] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5. Re-sending same reply.
I20260709 13:02:58.404732  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:58.405522  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:58.409065  8680 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:58.449558  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:58.450217  8681 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:02:58.451269  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:58.382592 11493 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:58.452476  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 63673.9us
I20260709 13:02:58.460139  8678 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:58.461850  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:58.390482 11504 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:58.454711  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54740.1us
I20260709 13:02:58.472146  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:58.476512  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 145551us
I20260709 13:02:58.480105  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:58.464246  8793 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.964s (SENT) [suppressed 44 similar messages]
I20260709 13:02:58.454478 11506 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:58.490411  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:58.467195 11504 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:58.493680  8679 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:58.491865  8756 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:02:58.497308  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:58.500211  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:58.500756  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:58.501453  8679 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:02:58.501616  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:58.502347  8822 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:02:58.503618  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:58.469633 11416 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.509294  8824 raft_consensus.cc:2368] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6.
I20260709 13:02:58.499217 11504 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:58.514935 11416 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:58.497184 11466 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:02:58.517400 11466 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:58.517807 11466 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:58.518513 11416 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:02:58.527447  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.528967  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.494270 11420 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:58.532160 11420 consensus_queue.cc:1059] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:58.531085  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 425) took 5450 ms. Trace:
I20260709 13:02:58.530761 11504 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:58.514437 11534 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:02:58.557174  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:58.538498  8829 rpcz_store.cc:276] 0709 13:02:53.081104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.081238 (+   134us) service_pool.cc:224] Handling call
0709 13:02:53.081605 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.987321 (+905716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.498762 (+4511441us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b540032ee54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:58.530878 (+ 32116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2081151360}
I20260709 13:02:58.576701  9383 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:58.577206  8827 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:58.531245 11360 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:02:58.562719  8674 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.623s	user 0.006s	sys 0.005s
I20260709 13:02:58.736723  8677 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:02:58.714581  8722 leader_election.cc:336] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.052s (SENT)
W20260709 13:02:58.594280  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 53780us
I20260709 13:02:58.616724  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:02:58.751736  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 344420us
I20260709 13:02:58.594210  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:58.897186  8674 raft_consensus.cc:2410] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:02:58.904543  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 411868us
I20260709 13:02:58.904774  8748 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:58.691167  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:58.663072  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.884948  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:02:58.640470  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:58.910193  8829 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:58.691169  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:02:58.915879  8793 connection.cc:256] RPC call timeout handler was delayed by -4.4370353980000345s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:02:58.916514  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1395) took 1406 ms. Trace:
I20260709 13:02:58.918015  9388 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:02:58.918134  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 421) took 1073 ms. Trace:
I20260709 13:02:58.918792  8674 rpcz_store.cc:276] 0709 13:02:57.835781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.843122 (+  7341us) service_pool.cc:224] Handling call
0709 13:02:58.909383 (+1066261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.916926  8822 rpcz_store.cc:276] 0709 13:02:57.509713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.548142 (+1038429us) service_pool.cc:224] Handling call
0709 13:02:58.916442 (+368300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:58.925324  8672 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:58.617803 11536 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 13:02:58.929226  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.932305  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 402423us
W20260709 13:02:58.937863  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.697542 11523 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:58.825582  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:58.930331  8681 pending_rounds.cc:85] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 1
I20260709 13:02:58.948661  8681 pending_rounds.cc:107] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 13:02:58.948997  8681 raft_consensus.cc:2889] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:58.945935  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:58.824404 11504 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:58.963387  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:58.618206 11442 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:58.871479  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:58.871358  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:58.951159  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:58.961539  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:58.979350  8677 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 2.
W20260709 13:02:58.930575 11330 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.879s	user 0.015s	sys 0.002s
I20260709 13:02:58.966049  8824 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:02:58.968451  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:58.982492  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:58.983475  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1406) took 1163 ms. Trace:
W20260709 13:02:58.983495  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.986320  8824 rpcz_store.cc:276] 0709 13:02:57.819923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.539242 (+719319us) service_pool.cc:224] Handling call
0709 13:02:58.983400 (+444158us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:58.992529  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 326022us
I20260709 13:02:59.004951  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:58.594954 11466 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:59.015856  8756 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:59.017294  8822 raft_consensus.cc:2410] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:58.971728 11442 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:02:59.019941  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 52578.3us
I20260709 13:02:59.020990  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:59.024410  8748 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:02:59.023234  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1402) took 1204 ms. Trace:
I20260709 13:02:59.026062  8822 rpcz_store.cc:276] 0709 13:02:57.818446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.934504 (+1116058us) service_pool.cc:224] Handling call
0709 13:02:59.023131 (+ 88627us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:59.015569  9383 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
W20260709 13:02:59.028071  8747 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.028s	sys 0.059s
W20260709 13:02:59.028628  8747 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.028s	sys 0.059s
I20260709 13:02:59.008592 11504 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:59.018782  9384 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.996s	user 0.016s	sys 0.000s
I20260709 13:02:59.032317  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:59.038257  8824 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:02:59.041549  8647 connection.cc:256] RPC call timeout handler was delayed by -1.2730127800000446s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.48510000000000003s for a response
I20260709 13:02:59.045435  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:58.962954 11493 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.059s
W20260709 13:02:59.039531  8722 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.096s (SENT)
W20260709 13:02:59.043037 11506 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.003s
I20260709 13:02:59.054117  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1396) took 1537 ms. Trace:
W20260709 13:02:59.057919 11324 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:02:59.058902  9385 raft_consensus.cc:2410] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:59.059768  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:59.062458  8824 rpcz_store.cc:276] 0709 13:02:57.510018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.006186 (+1496168us) service_pool.cc:224] Handling call
0709 13:02:59.030985 (+ 24799us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.046427 (+ 15442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.046905 (+   478us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:59.071899  8828 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.906s	user 0.011s	sys 0.002s
I20260709 13:02:59.080235  8828 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:02:59.055105  9388 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:02:59.051239  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:59.082122  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1421) took 1178 ms. Trace:
I20260709 13:02:59.081914  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:59.083320  8828 rpcz_store.cc:276] 0709 13:02:57.903778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.152503 (+248725us) service_pool.cc:224] Handling call
0709 13:02:59.082053 (+929550us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:59.084190  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1538) took 5806 ms. Trace:
I20260709 13:02:59.086431  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:59.087556  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.088297  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:59.090168  8678 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.587s	user 0.011s	sys 0.001s
I20260709 13:02:59.090632  8678 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:59.088955  8824 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:59.048934  9384 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:02:59.084647  8675 rpcz_store.cc:276] 0709 13:02:53.271175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.479027 (+3207852us) service_pool.cc:224] Handling call
0709 13:02:56.789572 (+310545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.032449 (+242877us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.032494 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:57.035730 (+  3236us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:57.035801 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.035912 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.077738 (+2041826us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54002a99d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:59.077937 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":215269,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":157,"spinlock_wait_cycles":712805632,"thread_start_us":240108,"threads_started":1}
W20260709 13:02:59.096468 11382 consensus_meta.cc:287] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.931s	user 0.013s	sys 0.001s
I20260709 13:02:59.094739 11324 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:59.099094  8750 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:59.099099  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:59.100133  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.100826  8824 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:59.093681  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:59.088577  8647 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:59.078485 11506 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8.
I20260709 13:02:59.107436  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:59.049767 11543 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:02:59.108886  9389 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:59.110282  8822 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:59.122962  8822 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:02:59.101943  8828 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:59.085868 11504 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:59.102984  9384 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 517) took 1538 ms (client timeout 1584 ms). Trace:
I20260709 13:02:59.147774  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.148778  8828 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 2.
I20260709 13:02:59.135236  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.154433  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:59.164542  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:59.112138 11442 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:59.170086  8677 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:02:59.170532  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 140437us
I20260709 13:02:59.167727  9382 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:59.197546  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:59.173048  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 225996us
I20260709 13:02:59.209215  8829 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:02:59.210227  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1399) took 1561 ms. Trace:
I20260709 13:02:59.210681  8829 rpcz_store.cc:276] 0709 13:02:57.651301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.617596 (+966295us) service_pool.cc:224] Handling call
0709 13:02:59.210139 (+592543us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:59.154542 11506 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:59.222591  8828 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
W20260709 13:02:59.146557  9384 rpcz_store.cc:269] 0709 13:02:57.588956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.621034 (+ 32078us) service_pool.cc:224] Handling call
0709 13:02:59.100534 (+1479500us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:59.234959  8722 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.117s (SENT)
I20260709 13:02:59.238534  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.239722  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:59.241499  8824 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:02:59.232479  8822 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:02:59.244609  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 156135us
I20260709 13:02:59.233287 11442 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:59.248478 11547 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:02:59.222195 11544 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:59.246352 11549 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:59.257454  8822 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:59.253509  8823 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:02:59.254073  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:59.249260 11324 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:02:59.273264 11371 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:59.275727  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:59.278779  8647 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:59.280526  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:59.280802  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.258809 11493 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:02:59.263080  8645 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:59.281890  8821 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:59.281981 11493 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:59.283027  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:59.284050 11224 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:59.248942 11506 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:59.285607 11544 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:59.286101 11544 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:02:59.267405  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 44902.8us
W20260709 13:02:59.281345  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.298808  8828 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:59.283232 11556 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:59.286921 11557 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:59.257055 11549 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.287385 11506 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:59.258898  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.306560 11549 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:59.306926 11506 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.307210  8824 raft_consensus.cc:2410] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:59.307605 11506 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:59.308283  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.309628  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.286490 11544 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:59.284986 11224 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:59.278992 11302 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:59.290483 11290 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:59.311913 11290 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:02:59.313138  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:59.313649 11506 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:59.289317  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.292126  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.315201  8824 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:59.313177 11290 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:58.085340  8566 catalog_manager.cc:5028] TS e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): tablet 51388ea026cc4d098949a4a0327a35fc (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:02:59.316013  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 440) took 1729 ms. Trace:
I20260709 13:02:59.316179 11290 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:59.316560  8566 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 1.23149e+06us
I20260709 13:02:59.316617  8821 rpcz_store.cc:276] 0709 13:02:57.586885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.314955 (+1728070us) service_pool.cc:224] Handling call
0709 13:02:59.315364 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.315591 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:59.315615 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:59.315689 (+    74us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:59.315722 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.315784 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.315909 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:59.317513  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 441) took 1682 ms. Trace:
I20260709 13:02:59.294864  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:59.322384 11550 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.322592  9355 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:59.318039  8824 rpcz_store.cc:276] 0709 13:02:57.634805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.316226 (+1681421us) service_pool.cc:224] Handling call
0709 13:02:59.316649 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.317050 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:59.317082 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:59.317153 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:59.317221 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.317288 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.317421 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:59.324958 11504 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:59.295074  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:59.295857  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:59.296368  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.284925 11493 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:59.303653  8647 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:59.293883  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.323311  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.334147  8680 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:59.337172  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.338351  8677 raft_consensus.cc:2368] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8.
I20260709 13:02:59.344720  8679 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:02:59.344758  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 102198us
I20260709 13:02:59.346652  8822 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:02:59.347272  8822 pending_rounds.cc:85] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 4
I20260709 13:02:59.347592  8822 pending_rounds.cc:107] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:02:59.347919  8822 raft_consensus.cc:2889] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:59.351899  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:59.323292 11550 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:59.363981  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.305290 11557 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.365156  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:02:59.368953  8677 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:59.371464  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.374256  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.377539  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:59.378568  8680 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:59.378576  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.384315  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1397) took 1867 ms. Trace:
I20260709 13:02:59.385084  8822 rpcz_store.cc:276] 0709 13:02:57.510467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.026901 (+1516434us) service_pool.cc:224] Handling call
0709 13:02:59.049491 (+ 22590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.377893 (+328402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.378227 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:59.366494 11556 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:59.387096  8675 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:59.391732  8680 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:59.364040 11001 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:02:59.399466  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 117856us
I20260709 13:02:59.403343  8680 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:59.404390  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.371143 11384 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:02:59.405972  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.406587 11327 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:59.407152 11327 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:59.407506 11551 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:02:59.408283  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.408528 11441 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:59.415213 11441 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:02:59.415516 11520 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.685s	user 0.005s	sys 0.009s
I20260709 13:02:59.416540 11441 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:59.377058 11504 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:59.418561 11504 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:59.419086  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:59.374115 11493 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.410792 11544 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:59.373466 11557 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:59.498293  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.372645 11506 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:59.398217 11408 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:02:59.511454  8677 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:59.512645  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.409195 11550 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:59.422941  8828 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:59.423578  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.491659 11003 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:02:59.493477 11421 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:59.407537 11327 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:59.499607  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 66779.7us
I20260709 13:02:59.500712  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.509796  8675 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:59.510317  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:59.512666  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 168303us
I20260709 13:02:59.516963  8821 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:59.516450  8680 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:02:59.571549  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:59.580277  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 215217us
I20260709 13:02:59.577764  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.583307  8678 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:59.521100  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:59.585299  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2098) took 2118 ms. Trace:
W20260709 13:02:59.520857  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 103298us
I20260709 13:02:59.557749  8677 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:59.588968  8677 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:59.589998  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 69025.6us
I20260709 13:02:59.582517  8822 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:02:59.584333  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.509879 11504 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:59.599532  8647 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:59.601951  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.585784  8755 rpcz_store.cc:276] 0709 13:02:57.365436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.366082 (+   646us) service_pool.cc:224] Handling call
0709 13:02:57.366541 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:57.367175 (+   634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.367208 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:57.367833 (+   625us) log.cc:844] Serialized 26 byte log entry
0709 13:02:57.588217 (+220384us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:57.588266 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.588343 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:59.414836 (+1826493us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:02:59.421426 (+  6590us) raft_consensus.cc:1685] finished
0709 13:02:59.421450 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.483359 (+ 61909us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2296,"spinlock_wait_cycles":106836736,"thread_start_us":219870,"threads_started":1,"wal-append.queue_time_us":362479}
I20260709 13:02:59.603974  9355 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:59.601154 11003 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:02:59.613252 11224 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:59.616245 11564 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:59.616845 11564 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:59.617269 11564 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:59.618150  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.619166  8678 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:02:59.620455  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:59.622238  8647 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:59.623471 11224 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:59.624024  8647 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:02:59.631138 11558 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:59.518226 11506 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:59.633213  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.596271 11421 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:59.639281  8822 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:59.647423  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:02:59.649204  8826 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1375) took 4649 ms (client timeout 2550 ms). Trace:
W20260709 13:02:59.649732  8826 rpcz_store.cc:269] 0709 13:02:54.982914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.631644 (+4648730us) service_pool.cc:211] Skipping call since client already timed out
0709 13:02:59.631838 (+   194us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:02:59.593273 11493 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:59.657677  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1563) took 1307 ms. Trace:
I20260709 13:02:59.658367  8676 rpcz_store.cc:276] 0709 13:02:58.350052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.351305 (+  1253us) service_pool.cc:224] Handling call
0709 13:02:58.401984 (+ 50679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.487271 (+ 85287us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:58.506467 (+ 19196us) log.cc:844] Serialized 14 byte log entry
0709 13:02:59.326312 (+819845us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b18003510a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:02:59.656774 (+330462us) log.cc:844] Serialized 14 byte log entry
0709 13:02:59.657227 (+   453us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:59.657300 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:59.657366 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.657458 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.657602 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":117931,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":107,"raft.queue_time_us":9822,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":26007,"spinlock_wait_cycles":46838144,"thread_start_us":320706,"threads_started":2,"wal-append.queue_time_us":1101871}
W20260709 13:02:59.663610  8647 leader_election.cc:336] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Timed out: RequestConsensusVote RPC to 127.7.89.132:41749 timed out after 1.645s (SENT)
I20260709 13:02:59.668046  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:02:59.669831  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:02:59.671573  8828 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
W20260709 13:02:59.675925  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 93663.8us
I20260709 13:02:59.677701  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.638652 11549 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:59.679869  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.685628  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.688344  8676 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:02:59.680284 11384 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:59.693099  8675 raft_consensus.cc:2393] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:59.693101  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.693604  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.689903 11556 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:59.695508 11515 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:59.697048 11515 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:59.702456 11506 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:59.703002 11557 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:59.704514  8674 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:59.705148  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.707213  8678 raft_consensus.cc:2393] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:59.707159 11360 raft_consensus.cc:3055] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:59.715548 11360 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:59.716688  8822 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:59.716502 11360 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:59.719879 11327 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:59.740037 11211 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.740859 11211 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:59.751178  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.752750  8676 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:59.756137  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1562) took 1406 ms. Trace:
I20260709 13:02:59.756630  8672 rpcz_store.cc:276] 0709 13:02:58.349621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.350421 (+   800us) service_pool.cc:224] Handling call
0709 13:02:58.909358 (+558937us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.926039 (+ 16681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.755887 (+829848us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b540033aed0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:59.756059 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1284586880}
I20260709 13:02:59.760070  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:02:59.762796  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 99227.3us
I20260709 13:02:59.764711  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:59.765777 11506 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:59.769645  9381 pending_rounds.cc:85] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting all ops after (but not including) 6
I20260709 13:02:59.752810 11224 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:59.769999  9381 pending_rounds.cc:107] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:02:59.770445  9381 raft_consensus.cc:2889] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:59.771090  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:02:59.772120  8672 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:02:59.773520  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:59.775249 11506 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:59.762502 11360 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:59.779410  9389 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
W20260709 13:02:59.781746  8827 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1420) took 1878 ms (client timeout 2054 ms). Trace:
I20260709 13:02:59.782675  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1392) took 2274 ms. Trace:
W20260709 13:02:59.782243  8827 rpcz_store.cc:269] 0709 13:02:57.903436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.384538 (+481102us) service_pool.cc:224] Handling call
0709 13:02:59.781538 (+1397000us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54002283d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:59.781678 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":394763648}
I20260709 13:02:59.777565  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.785540  8829 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:59.783103  8825 rpcz_store.cc:276] 0709 13:02:57.508637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.510766 (+  2129us) service_pool.cc:224] Handling call
0709 13:02:57.511293 (+   527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.832351 (+321058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.782416 (+1950065us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b540032f854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:02:59.782588 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361437952}
I20260709 13:02:59.787854  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.792485  8755 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:59.767685 11493 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:59.787338  8796 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:59.796594  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:59.798051  8645 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:02:59.816434  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.802300  8825 raft_consensus.cc:2376] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5. Re-sending same reply.
I20260709 13:02:59.819958  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:59.820604  8747 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:59.820848  8829 raft_consensus.cc:2376] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8. Re-sending same reply.
I20260709 13:02:59.821976  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:59.822942  8827 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:59.822310  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.771090  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:02:59.825356  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.851302  8829 raft_consensus.cc:2368] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7.
I20260709 13:02:59.852715  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.861796  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.862550  8825 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
W20260709 13:02:59.863830  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 81061us
I20260709 13:02:59.864850  8793 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:59.868444  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.869447  8829 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:59.869567  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:02:59.870258  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 177993us
I20260709 13:02:59.871773  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.872519  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:59.872023  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1713) took 1564 ms. Trace:
I20260709 13:02:59.817539 11483 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:59.860101 11290 consensus_meta.cc:287] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.543s	user 0.012s	sys 0.002s
I20260709 13:02:59.827191 11506 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:59.871845  8751 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:59.857937 11556 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:59.888499  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:59.891605  8755 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:59.893949 11375 raft_consensus.cc:3055] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:59.889777 11506 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:59.899765  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.901212  9355 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:02:59.874388  8753 rpcz_store.cc:276] 0709 13:02:58.314468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.328213 (+ 13745us) service_pool.cc:224] Handling call
0709 13:02:58.345731 (+ 17518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.374035 (+ 28304us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:58.402882 (+ 28847us) log.cc:844] Serialized 14 byte log entry
0709 13:02:59.775187 (+1372305us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b18000d6120. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:02:59.861046 (+ 85859us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:59.861106 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:59.861157 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.861229 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.866118 (+  4889us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7981,"prepare.queue_time_us":69420,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":102,"spinlock_wait_cycles":5829888,"thread_start_us":85564,"threads_started":1}
I20260709 13:02:59.892647  8793 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:59.903777  9387 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:59.904302  8747 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:59.908499  8829 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:02:59.909313  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.909912  8829 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:02:59.910815  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:59.911461  8825 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:59.910554  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:59.914580  8829 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:59.915647  8755 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:59.917924  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.918561  8825 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:02:59.920115  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:59.905587 11556 raft_consensus.cc:697] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:02:59.921689  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.922293  8828 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:02:59.905781 11506 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:59.902866 11375 raft_consensus.cc:740] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:02:59.905282 11416 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:59.928836  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:59.929770  8828 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:59.919220 11567 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.917565 11371 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:59.928177  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:59.932372  8827 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:02:59.934939  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:59.935961  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.936985  8827 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:59.937201  8793 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:59.937502  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:59.933508 11342 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:59.934859 11371 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:02:59.941959  8825 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:59.942757 11493 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:59.917069 11483 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:59.945006  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.945457  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:59.948176  8829 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:59.949803  8753 raft_consensus.cc:2410] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:59.950138  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:02:59.950356  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:02:59.951054  8829 raft_consensus.cc:2376] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6. Re-sending same reply.
I20260709 13:02:59.936642 11567 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:59.944540 11556 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:02:59.954532 11567 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:02:59.955597  8747 raft_consensus.cc:2410] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:59.945832 11371 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:59.960310  8825 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:59.961488  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.962095  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:59.952272  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.962348  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:02:59.966125  8751 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:59.963949  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1404) took 2144 ms (client timeout 2179 ms). Trace:
I20260709 13:02:59.965566  8754 raft_consensus.cc:2368] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9.
I20260709 13:02:59.968111  8829 raft_consensus.cc:2410] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:02:59.966785  8823 rpcz_store.cc:269] 0709 13:02:57.819202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.404303 (+585101us) service_pool.cc:224] Handling call
0709 13:02:59.963795 (+1559492us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b5400127354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:59.963894 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1531519744}
I20260709 13:02:59.971434 11493 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:59.973593 11493 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:02:59.974007 11493 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:59.975405 11535 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.509s	user 0.004s	sys 0.015s
W20260709 13:02:59.969527  8673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1548) took 4759 ms (client timeout 2673 ms). Trace:
I20260709 13:02:59.971011  8827 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:02:59.970853  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:02:59.976768  8822 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:02:59.977783 11276 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:02:59.973161  8823 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:02:59.976466  8673 rpcz_store.cc:269] 0709 13:02:55.210242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.246015 (+ 35773us) service_pool.cc:224] Handling call
0709 13:02:59.965548 (+4719533us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b5400298e54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:02:59.969443 (+  3895us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4431467904}
I20260709 13:02:59.979365  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1401) took 2161 ms. Trace:
I20260709 13:02:59.968910  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:02:59.979712  8823 rpcz_store.cc:276] 0709 13:02:57.817907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.972485 (+2154578us) service_pool.cc:224] Handling call
0709 13:02:59.972801 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.979052 (+  6251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.979278 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:59.981113 11483 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:02:59.980500  8822 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:59.981947  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1407) took 2161 ms. Trace:
I20260709 13:02:59.981907 11375 consensus_queue.cc:260] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:02:59.980095  8749 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:02:59.982501  8822 rpcz_store.cc:276] 0709 13:02:57.820253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.977540 (+2157287us) service_pool.cc:224] Handling call
0709 13:02:59.977858 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.981677 (+  3819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.981887 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:59.983127 11375 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:59.984110  8823 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:59.986886  8822 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:59.987819  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1411) took 2166 ms. Trace:
I20260709 13:02:59.988550  8822 rpcz_store.cc:276] 0709 13:02:57.821397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.983652 (+2162255us) service_pool.cc:224] Handling call
0709 13:02:59.985210 (+  1558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.987502 (+  2292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.987754 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:59.990420  8822 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:59.991205  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1418) took 2088 ms. Trace:
I20260709 13:02:59.992571 11466 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:00.000556  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:00.001395  8750 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:00.001861  8822 rpcz_store.cc:276] 0709 13:02:57.902638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.989586 (+2086948us) service_pool.cc:224] Handling call
0709 13:02:59.989998 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.990918 (+   920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.991143 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.005302  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1423) took 2100 ms. Trace:
I20260709 13:03:00.005120  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:00.007812  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:00.008369  8750 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:03:00.008554  8756 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:02:59.992704  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 446) took 2094 ms. Trace:
I20260709 13:03:00.009068  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:00.006094  8828 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:00.007118  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:00.009701  8749 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:00.010138  8748 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:00.009351  8827 rpcz_store.cc:276] 0709 13:02:57.897993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.991599 (+2093606us) service_pool.cc:224] Handling call
0709 13:02:59.991985 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.992297 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:59.992327 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:59.992410 (+    83us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:59.992457 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.992521 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.992640 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:59.971709  8825 raft_consensus.cc:2410] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 9 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:00.013101  8827 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:00.014457  8828 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:03:00.016752  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 447) took 2066 ms. Trace:
I20260709 13:03:00.014907  8822 rpcz_store.cc:276] 0709 13:02:57.904603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.004250 (+2099647us) service_pool.cc:224] Handling call
0709 13:03:00.004577 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.004933 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.004963 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:00.005006 (+    43us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:00.005049 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.005109 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.005239 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.018036  8827 rpcz_store.cc:276] 0709 13:02:57.950856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.012492 (+2061636us) service_pool.cc:224] Handling call
0709 13:03:00.012766 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.016447 (+  3681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.016688 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:00.019268  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 69014.1us
I20260709 13:03:00.020292  8829 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:03:00.023893  8751 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:00.026386  8827 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:03:00.027083  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 60927.3us
I20260709 13:03:00.030049  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:00.031023  8753 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:03:00.033519  8793 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:00.028864 11208 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
W20260709 13:03:00.035555  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.036015  8827 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:00.036538  8645 leader_election.cc:400] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9.
I20260709 13:03:00.039223  8645 leader_election.cc:403] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 13:03:00.037586  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 43921.3us
I20260709 13:03:00.041028 11204 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:03:00.043540 11416 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:00.043845  8829 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:03:00.044943 11544 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:00.045708  8829 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:03:00.046047 11418 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:00.047467 11208 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:00.047668  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:00.033043 11541 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:00.048813 11541 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:00.049793  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:00.050135  9380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 518) took 2450 ms (client timeout 2688 ms). Trace:
I20260709 13:03:00.051903  8827 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:00.052948  8828 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:03:00.056839  8829 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:00.060674  8749 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:00.053076  8827 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:00.053580  8828 pending_rounds.cc:85] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 5
I20260709 13:03:00.075114  8828 pending_rounds.cc:107] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:03:00.075623  8828 raft_consensus.cc:2889] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:00.073375  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:00.078248  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:00.079608  8749 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 3: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:00.080124  8829 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:00.072193 11413 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.081753  8825 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 13:03:00.081974  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 45625.4us
I20260709 13:03:00.088645  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:00.091029  8679 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:00.094749  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:00.095708  8679 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:00.099152  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.072635 11342 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:03:00.050571  9380 rpcz_store.cc:269] 0709 13:02:57.599696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.626479 (+ 26783us) service_pool.cc:224] Handling call
0709 13:03:00.049924 (+2423445us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b54001a4854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:00.050047 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":768540416}
I20260709 13:03:00.082659 11276 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:00.110788  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:00.104565  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:00.113263  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:00.087878 11544 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:00.115051  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:00.110728  8830 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1383) took 3111 ms (client timeout 2182 ms). Trace:
I20260709 13:03:00.129144  8672 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:03:00.117400  8749 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:00.124907  8676 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:00.116170  8673 raft_consensus.cc:2376] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7. Re-sending same reply.
I20260709 13:03:00.102905 11556 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:00.080994 11504 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:03:00.064355 11416 consensus_queue.cc:1059] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
W20260709 13:03:00.129076  8830 rpcz_store.cc:269] 0709 13:02:57.068309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.097719 (+ 29410us) service_pool.cc:224] Handling call
0709 13:03:00.106994 (+3009275us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b54001946d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:00.107131 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1601103616}
I20260709 13:03:00.135238  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:00.135660  8751 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:03:00.137799  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 98239.9us
I20260709 13:03:00.101191 11413 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:00.144737  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:00.148115  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:00.150146  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:00.115803 11276 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.169126  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:00.169919  8747 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:00.146742 11342 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:03:00.163049 11564 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.542s	user 0.005s	sys 0.008s
I20260709 13:03:00.147403 11556 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:00.151165 11413 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:00.162773 11276 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:00.139601 11587 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:00.139075 11442 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:03:00.162386 11541 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:00.191457 11290 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:03:00.254050 11586 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:03:00.252466  8830 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:00.253401 11556 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:00.177564 11342 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:00.251721  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 97660us
W20260709 13:03:00.276667  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 126615us
I20260709 13:03:00.280659  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:00.193668 11587 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:00.267983  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 157401us
I20260709 13:03:00.284155  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:00.282684  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 131593us
I20260709 13:03:00.177928 11371 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:00.294081  8755 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:00.267383 11290 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.297277  9353 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:00.283368 11342 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:00.302165  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:00.297883  9352 connection.cc:256] RPC call timeout handler was delayed by -1.7229821610000045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
I20260709 13:03:00.285756 11360 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:00.303503  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1569) took 1763 ms. Trace:
I20260709 13:03:00.309237  8681 rpcz_store.cc:276] 0709 13:02:58.540274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.566683 (+ 26409us) service_pool.cc:224] Handling call
0709 13:02:58.920532 (+353849us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:59.990121 (+1069589us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b2400195e18. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df2929c4 00007f32df290acb 00007f32df44bdc9 00007f32df44ba55 00007f32df44adb5 00007f32e20c12a5 00007f32e20c1f12 00007f32e20c94c4 00007f32e24b5a86 00007f32e2470a75
0709 13:03:00.041560 (+ 51439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.041593 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:00.042298 (+   705us) log.cc:844] Serialized 52 byte log entry
0709 13:03:00.103057 (+ 60759us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:03:00.103115 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.103217 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:00.303143 (+199926us) raft_consensus.cc:1685] finished
0709 13:03:00.303164 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.303434 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30943,"raft.queue_time_us":18896,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":7740,"spinlock_wait_cycles":73782144,"thread_start_us":60290,"threads_started":1,"wal-append.queue_time_us":98467}
I20260709 13:03:00.313048  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:00.314316  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:00.283365 11587 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:00.306985  8793 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:00.315434  8681 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:00.190155 11564 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:00.305138 11290 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:00.302358 11003 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:00.315989 11204 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:00.310773 11360 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.347702  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:00.306412 11418 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:03:00.331310 11442 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:00.349763  8681 raft_consensus.cc:2376] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4. Re-sending same reply.
W20260709 13:03:00.342288 11581 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.066s	user 0.001s	sys 0.001s
I20260709 13:03:00.336742 11567 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:00.363727 11528 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 2.613s	user 0.004s	sys 0.030s
I20260709 13:03:00.349229 11204 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.361613 11360 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:00.372429  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:00.336490 11541 consensus_queue.cc:1059] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:03:00.387532  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:00.379797  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 72885.5us
I20260709 13:03:00.391296  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:00.392544 11200 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:03:00.392582  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1717) took 2069 ms. Trace:
W20260709 13:03:00.396615  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 60825.5us
I20260709 13:03:00.394692  8752 rpcz_store.cc:276] 0709 13:02:58.322967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.387171 (+2064204us) service_pool.cc:224] Handling call
0709 13:03:00.392503 (+  5332us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.397275  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:00.398355  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 91318.9us
I20260709 13:03:00.368407 11416 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:00.386713 11204 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:00.407056  8679 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:03:00.410590  8679 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:00.412401  8722 leader_election.cc:400] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6.
I20260709 13:03:00.412812  8722 leader_election.cc:403] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:03:00.383567 11442 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:00.432098 11592 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:03:00.434808 11250 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:03:00.383684 11567 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:00.437803 11567 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:00.405176 11590 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:03:00.469063  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.438449 11200 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6.
W20260709 13:03:00.530344  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 133177us
I20260709 13:03:00.438546 11564 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:00.536654  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 124437us
W20260709 13:03:00.537431  9352 connection.cc:256] RPC call timeout handler was delayed by -1.6275821609999639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18020000000000003s for a response
W20260709 13:03:00.537992  9352 connection.cc:256] RPC call timeout handler was delayed by -1.6014821609999217s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
W20260709 13:03:00.538407  9352 connection.cc:256] RPC call timeout handler was delayed by -1.5006821609999861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
W20260709 13:03:00.539916  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 134544us
I20260709 13:03:00.541841  8722 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:00.426682 11593 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.110s
W20260709 13:03:00.546155  9352 connection.cc:256] RPC call timeout handler was delayed by -1.0866821609999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2403s for a response
W20260709 13:03:00.546769  9352 connection.cc:256] RPC call timeout handler was delayed by -1.0445418689999997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 13:03:00.545166  9353 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:00.450933 11493 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:00.451634 11375 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
W20260709 13:03:00.564846  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:00.566293 11506 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.635s	user 0.010s	sys 0.001s
W20260709 13:03:00.579385  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.575475 11393 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:00.588220 11393 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.588797 11393 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:00.594254  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 443) took 2810 ms. Trace:
I20260709 13:03:00.594810  8824 rpcz_store.cc:276] 0709 13:02:57.798663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.324422 (+1525759us) service_pool.cc:224] Handling call
0709 13:03:00.107531 (+783109us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.107741 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.107766 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:00.107860 (+    94us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:00.107904 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.107968 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.590498 (+482530us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b54001946d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:00.594127 (+  3629us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1800236928}
I20260709 13:03:00.596334 11211 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:00.597462  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 442) took 2928 ms. Trace:
I20260709 13:03:00.597851  8821 rpcz_store.cc:276] 0709 13:02:57.670415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.318030 (+1647615us) service_pool.cc:224] Handling call
0709 13:02:59.516704 (+198674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.583881 (+ 67177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.597256 (+1013375us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b54003ac5d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:00.597400 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456117504}
I20260709 13:03:00.579180 11506 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:00.553834 11371 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:00.585687 11360 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:00.607067  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 53071.6us
I20260709 13:03:00.446910 11598 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.109s
W20260709 13:03:00.608008  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 62942.8us
I20260709 13:03:00.612396  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2023) took 1292 ms. Trace:
I20260709 13:03:00.623288  8674 rpcz_store.cc:276] 0709 13:02:59.320161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.380997 (+ 60836us) service_pool.cc:224] Handling call
0709 13:02:59.418023 (+ 37026us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.704999 (+286976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.611983 (+906984us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5400299d54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:00.612295 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":658351488}
I20260709 13:03:00.632460  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:00.633505  9380 raft_consensus.cc:2393] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 8: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:03:00.636348  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:00.608143  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.427591 11442 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:00.620251 11506 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:00.620858  8829 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:00.614019 11418 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:00.657289  9381 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:00.642755 11290 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:00.671181  8647 leader_election.cc:336] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.964s (SENT)
I20260709 13:03:00.672114  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:00.679601  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:00.678733  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:00.682415 11544 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.549s	user 0.013s	sys 0.001s
I20260709 13:03:00.682665 11200 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:00.689251 11375 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:00.689303 11556 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:00.682787 11324 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:03:00.695436 11544 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9.
I20260709 13:03:00.704996  9381 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:03:00.705533  9381 pending_rounds.cc:85] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting all ops after (but not including) 6
I20260709 13:03:00.705821  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:00.709389  9381 pending_rounds.cc:107] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:03:00.709759  9381 raft_consensus.cc:2889] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:00.716150 11493 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:00.717200 11557 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:00.717289 11360 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:00.728117 11204 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:00.735654 11276 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:00.736207 11204 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:00.738094  8824 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:03:00.745565  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1408) took 2925 ms. Trace:
I20260709 13:03:00.746126  8823 rpcz_store.cc:276] 0709 13:02:57.820544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.981130 (+2160586us) service_pool.cc:224] Handling call
0709 13:02:59.983921 (+  2791us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.984553 (+   632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.745321 (+760768us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b54002189d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:00.745460 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5617536}
I20260709 13:03:00.749435  9352 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:00.750658 11204 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:00.751147  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:00.752266 11204 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:00.752796 11204 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.753252 11204 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:00.753582  9387 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 415) took 2410 ms (client timeout 1645 ms). Trace:
I20260709 13:03:00.755947 11226 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:00.756466 11226 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:00.757611 11226 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:00.783493 11226 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:00.733798 11342 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:00.760289 11290 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:00.762959  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:00.791412  8749 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:00.764505 11393 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:00.765868 11382 raft_consensus.cc:3055] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:03:00.770321 11588 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.429s	user 0.001s	sys 0.000s
I20260709 13:03:00.770788 11442 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:00.738444 11276 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:00.807646 11564 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:00.809486 11549 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:00.799836 11382 raft_consensus.cc:740] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:00.812796 11200 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:00.813301 11382 consensus_queue.cc:260] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:00.774176  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:03:00.777825  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68117.9us
W20260709 13:03:00.754096  9387 rpcz_store.cc:269] 0709 13:02:58.343151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.343299 (+   148us) service_pool.cc:224] Handling call
0709 13:03:00.753366 (+2410067us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54005b0754. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:00.753483 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3151483136}
I20260709 13:03:00.780784  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:00.706075 11587 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:00.800458  8672 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:00.801281  9352 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:00.821853  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 122997us
W20260709 13:03:00.827600  8722 connection.cc:256] RPC call timeout handler was delayed by -1.9011361599999645s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
W20260709 13:03:00.828218  8722 connection.cc:256] RPC call timeout handler was delayed by -1.8975361599999587s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
I20260709 13:03:00.828634  9381 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:00.814587 11382 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:00.814848 11393 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:00.839002  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:00.839715 11342 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:00.840296 11327 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:00.842329  8827 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:00.842808 11413 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.866326 11413 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:00.843832 11587 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:00.868990 11413 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:00.836557 11557 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.869904 11557 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:00.848757 11367 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:00.870888 11367 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:00.872347 11557 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:00.847477 11515 raft_consensus.cc:3055] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:00.844496  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:00.871950 11367 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:00.856653  8722 connection.cc:256] RPC call timeout handler was delayed by -1.6671361600000409s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
I20260709 13:03:00.880239  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:00.881560  8722 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:00.866314 11224 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:00.887480  8827 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:00.895540  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1435) took 1529 ms. Trace:
I20260709 13:03:00.896169  8827 rpcz_store.cc:276] 0709 13:02:59.365935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.030794 (+664859us) service_pool.cc:224] Handling call
0709 13:03:00.031098 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.841818 (+810720us) spinlock_profiling.cc:243] Waited 97.1 ms on lock 0x7b54001d2520. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e246ea1d 00007f32e24551a4 00007f32e24d7447 00007f32e24effe7 00007f32e24f17de 00007f32e24fbb49 00007f32e24fc416 00007f32e24f8d8f 00007f32e24f883e 00007f32e74bd419
0709 13:03:00.895105 (+ 53287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.895453 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223265024}
W20260709 13:03:00.898871  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 97739.7us
W20260709 13:03:00.906567  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.878144 11515 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:00.887775 11367 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:00.888123 11290 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:00.886080 11224 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:00.888293 11208 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:00.888525 11416 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.918855  9352 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:00.925787  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.927660  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:00.905841 11327 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:00.931433  9387 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:00.911626 11342 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:00.937413  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 55920.2us
W20260709 13:03:00.938271  8794 connection.cc:256] RPC call timeout handler was delayed by -1.5481186869999419s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 13:03:00.938719  8794 connection.cc:256] RPC call timeout handler was delayed by -1.521118687000012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1605s for a response
W20260709 13:03:00.939023  8794 connection.cc:256] RPC call timeout handler was delayed by -1.4554186869999057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
W20260709 13:03:00.939378  8794 connection.cc:256] RPC call timeout handler was delayed by -1.3708186869999963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
I20260709 13:03:00.939610  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1636) took 1169 ms. Trace:
I20260709 13:03:00.898787 11557 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:03:00.924759 11224 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:00.945134  8754 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:00.947301  8722 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:00.940007  8825 rpcz_store.cc:276] 0709 13:02:59.769858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.087860 (+318002us) service_pool.cc:224] Handling call
0709 13:03:00.101704 (+ 13844us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.114556 (+ 12852us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:00.181115 (+ 66559us) log.cc:844] Serialized 14 byte log entry
0709 13:03:00.419353 (+238238us) log.cc:844] Serialized 14 byte log entry
0709 13:03:00.875901 (+456548us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b18003adaa0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:03:00.939190 (+ 63289us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:00.939251 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:00.939308 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.939389 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.939531 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43793,"prepare.queue_time_us":68766,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":104,"raft.queue_time_us":495275,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":15742,"spinlock_wait_cycles":28497920,"thread_start_us":322628,"threads_started":3,"wal-append.queue_time_us":339433}
I20260709 13:03:00.950222  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1637) took 1180 ms. Trace:
I20260709 13:03:00.908557 11258 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:03:00.952481  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:00.908222 11541 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:03:00.955828  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:00.963397  8675 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
W20260709 13:03:00.957696  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.955731  8828 rpcz_store.cc:276] 0709 13:02:59.770240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.109096 (+338856us) service_pool.cc:224] Handling call
0709 13:03:00.127375 (+ 18279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.137142 (+  9767us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:00.309554 (+172412us) log.cc:844] Serialized 14 byte log entry
0709 13:03:00.949713 (+640159us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:00.949790 (+    77us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:00.949844 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.950007 (+   163us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.950151 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":440533,"prepare.queue_time_us":32934,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":454757,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3227,"spinlock_wait_cycles":72304512,"thread_start_us":305770,"threads_started":3,"wal-append.queue_time_us":209982}
I20260709 13:03:00.931033 11590 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.968400  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:00.969750  8827 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:00.970345  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:00.972837  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:00.936060  8647 leader_election.cc:336] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.814s (SENT)
I20260709 13:03:00.944758 11211 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:00.978473  8825 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:00.984442  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:00.945276 11555 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.985906  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:00.994233  8676 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 4: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:00.988749  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:00.986421  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:00.929823 11204 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:00.997524  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:01.004294  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:00.945001 11408 consensus_queue.cc:1059] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:03:01.005826  8677 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:01.004096  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:01.011962  8677 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:01.012223  8826 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:00.970491 11587 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.004320  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.008006  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.007313  8828 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:01.005458  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:00.973737 11327 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:03:01.016842  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 78029.7us
I20260709 13:03:00.947999 11342 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:01.013782  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.020303 11621 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.028569  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:01.023656  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:01.024027  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:01.022861  8823 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
W20260709 13:03:01.009395 11615 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 13:03:01.027275  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.028496  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.025303 11204 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:01.031030  8828 raft_consensus.cc:2376] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3. Re-sending same reply.
I20260709 13:03:01.031538  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:01.031970  8722 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:01.032585  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:01.035611  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.035857  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.033425  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.036532  8828 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:03:01.036829  8825 raft_consensus.cc:2393] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:01.036825  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:01.037442  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:01.034934  8829 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:01.038173  8828 raft_consensus.cc:2376] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6. Re-sending same reply.
I20260709 13:03:01.040747  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.042006  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.042709  8828 raft_consensus.cc:2376] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8. Re-sending same reply.
I20260709 13:03:01.046275  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:01.047014  8827 raft_consensus.cc:2376] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3. Re-sending same reply.
I20260709 13:03:01.047153  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:01.047648  8722 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:01.047917  8680 raft_consensus.cc:2376] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6. Re-sending same reply.
I20260709 13:03:01.037653  8830 raft_consensus.cc:2376] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6. Re-sending same reply.
I20260709 13:03:01.049959  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:01.050787  8673 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
W20260709 13:03:01.052500  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 116485us
I20260709 13:03:00.973523 11590 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:01.053231  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.054159  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:01.060257  8825 raft_consensus.cc:2376] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6. Re-sending same reply.
I20260709 13:03:01.056740  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:01.060812  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:01.056924  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:01.057518  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:00.965066 11515 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:01.029635 11360 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:01.063918  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:01.070567  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:01.065992  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.061371  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.065717  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:01.069691  9387 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:01.061645  8680 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:01.053237  8677 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:01.059042 11327 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:01.071254  8830 raft_consensus.cc:2376] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8. Re-sending same reply.
I20260709 13:03:01.078775  8828 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:01.078923  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.074803  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.079785  8830 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:01.071925  8827 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:01.081609  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:01.002678 11555 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:01.080626  8825 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:01.083165  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:01.085525  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.086355  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:01.095779  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:01.022110 11587 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:01.112124  8719 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:01.116250 11596 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:01.117115  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:01.145273  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.148802  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:01.117892 11208 raft_consensus.cc:3055] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:01.119927  8675 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:01.062067 11504 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:03:01.066128 11290 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:01.124315 11342 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:01.125134 11555 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:01.125989 11557 raft_consensus.cc:3055] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:01.131292  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:01.194864 11557 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:01.195202  9387 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:01.168116 11208 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:01.195989 11557 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:01.197075 11557 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:01.196830 11208 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:01.132880 11587 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:01.189868  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 77842.6us
I20260709 13:03:01.115113 11457 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.200119 11457 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:01.072597 11204 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:03:01.135524  8829 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:01.200120  8719 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:01.204319  8719 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:01.204516  8829 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:01.198299 11208 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:01.136950  8823 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:01.206826 11483 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:01.138813  8824 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:01.139878  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:01.206473 11393 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:01.208355  8673 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:01.209456  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:01.210242  8673 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:03:01.142294  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55959.7us
I20260709 13:03:01.210995 11457 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:01.213029  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:01.213268  9352 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:01.212988  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:01.143525  8677 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:01.144075  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:01.178815  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 61792.9us
I20260709 13:03:01.215022  8827 raft_consensus.cc:1813] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Handling vote request from an unknown peer 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:01.215588  8827 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:01.216476 11548 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:01.216933 11548 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.217273 11548 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:03:01.180681  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 133126us
I20260709 13:03:01.180819  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:01.219923  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:01.081825 11413 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:01.090066 11603 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.518s	user 0.000s	sys 0.003s
I20260709 13:03:01.185544  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:01.086772 11623 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:01.233676  8722 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:01.235823  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:01.191244  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.166633 11549 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.040292 11621 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:01.192543 11226 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:01.179993 11544 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.190702 11504 consensus_queue.cc:1243] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:03:01.137724 11224 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:01.207417 11483 raft_consensus.cc:697] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:01.208317 11393 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.075245 11327 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:01.214227 11572 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:01.214275  8794 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:01.312686  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 98471.4us
I20260709 13:03:01.313596  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:01.082708 11408 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 13:03:01.224457  8826 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:01.225893  8677 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:03:01.315549  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:01.315828  8677 pending_rounds.cc:85] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 8
I20260709 13:03:01.316231  8677 pending_rounds.cc:107] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:03:01.316501  8677 raft_consensus.cc:2889] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:01.189612  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:01.312908  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:03:01.317811  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 128485us
I20260709 13:03:01.319202  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:01.236620  8678 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:01.321033  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:01.231526 11200 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.231397 11276 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:01.324486  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:01.324556  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:01.240573 11590 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:01.233911  8830 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:01.247897  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:01.260136  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:01.263521  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:01.326157  8681 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:01.326287  8676 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:01.267349  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:01.327025  9387 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
W20260709 13:03:01.274457  8722 leader_election.cc:336] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.972s (SENT)
I20260709 13:03:01.278990  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1586) took 1186 ms. Trace:
I20260709 13:03:01.328047  8672 rpcz_store.cc:276] 0709 13:03:00.093072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.144371 (+ 51299us) service_pool.cc:224] Handling call
0709 13:03:01.278711 (+1134340us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b54003cf854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:01.278899 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1297862528}
I20260709 13:03:01.329717  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:01.281673  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.281417 11416 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:01.292634  8579 catalog_manager.cc:5697] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 3 to 5, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 5 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:01.297899  8575 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43462 (request call id 266) took 15114 ms (client timeout 15000 ms). Trace:
I20260709 13:03:00.707621  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:01.305614  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 69890.1us
I20260709 13:03:01.253198 11204 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:01.308120  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:01.309609  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.232388 11619 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:03:01.238798 11624 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:03:01.281137 11565 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:01.232450 11413 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:01.332628  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:01.333917  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:01.363590  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:01.310436 11375 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:01.285691  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:01.337692  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:01.346844  8794 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:01.307554 11393 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:01.311457 11327 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:01.349046  8674 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:01.334774  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:01.379779  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 105387us
I20260709 13:03:01.366044  8673 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:01.326303  8675 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:01.363533  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:01.380913  8722 leader_election.cc:336] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.027s (SENT)
I20260709 13:03:01.381531  8826 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 3.
I20260709 13:03:01.382586  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.335531 11360 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:01.392866  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:01.370695 11541 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:01.280928 11587 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:03:01.372393  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:01.373716  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.363955 11504 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:01.369920  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:01.283748 11549 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:01.383466  8826 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:01.383401  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 49585.4us
I20260709 13:03:01.419070  8793 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:01.278398 11563 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:03:01.283531 11544 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:01.385223  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:01.386385  8579 catalog_manager.cc:5697] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 1 to 2, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:01.365022 11556 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:03:01.331943 11506 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:01.336939 11555 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:01.389413 11416 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:01.405112  9381 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 4: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:03:01.376580  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46167.4us
I20260709 13:03:01.478129  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:01.348095 11607 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:01.348600 11593 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.351354 11619 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.436664  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:01.389881 11360 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.391026 11342 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:01.468346  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 96064us
I20260709 13:03:01.517484  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:01.518893 11420 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:01.519228 11564 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:01.519546  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:01.520740 11420 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:01.521212 11420 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:01.521651  8647 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:01.398049 11541 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:01.522441 11541 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:01.529809  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 462) took 1444 ms. Trace:
I20260709 13:03:01.530522  8821 rpcz_store.cc:276] 0709 13:03:00.085433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.603196 (+517763us) service_pool.cc:224] Handling call
0709 13:03:00.603723 (+   527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.910010 (+306287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.910043 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:00.913593 (+  3550us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.014909 (+101316us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:01.014959 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.015154 (+   195us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.529471 (+514317us) raft_consensus.cc:1685] finished
0709 13:03:01.529501 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.529712 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":312817,"raft.queue_time_us":294120,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":6976,"thread_start_us":62740,"threads_started":1,"wal-append.queue_time_us":111884}
I20260709 13:03:01.535717 11360 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:01.537060 11504 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:01.537870 11504 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:01.476150 11483 consensus_queue.cc:237] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:01.540084 11623 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:01.540980 11623 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:01.336637 11200 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:01.545626 11200 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:01.488149  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:01.500280 11593 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:01.348524 11625 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.134s
W20260709 13:03:01.498927  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.433298  9382 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:01.480774  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 99918us
I20260709 13:03:01.507529 11622 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:01.499552 11382 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:01.499812 11619 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:01.513183 11576 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:01.438062 11596 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:01.482602 11499 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:01.521682 11420 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:01.614818  9355 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:01.474347 11276 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:03:01.554447  8722 leader_election.cc:336] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.127s (SENT)
I20260709 13:03:01.638306  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:01.587072  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.569897  8579 catalog_manager.cc:5697] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
W20260709 13:03:01.456933  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.641814  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.383060 11211 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:01.442377  8674 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:03:01.509141 11626 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
W20260709 13:03:01.652766  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 98378.8us
W20260709 13:03:01.653540  8722 leader_election.cc:336] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.138s (SENT)
I20260709 13:03:01.653967  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:01.552884 11506 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:03:01.659344  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 405994us
I20260709 13:03:01.583876 11556 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:01.662572  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 141058us
I20260709 13:03:01.663712  8647 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:01.584189 11382 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:01.670037  8579 catalog_manager.cc:5697] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 2 to 4, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 4 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.674773  8673 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
W20260709 13:03:01.690737  8722 leader_election.cc:336] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.182s (SENT)
I20260709 13:03:01.649950 11200 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:01.693567  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.650463 11211 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:01.691267  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:01.657191 11258 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:01.688570  8579 catalog_manager.cc:5697] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 3 to 4, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 4 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:03:01.649052 11276 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:01.657359 11506 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:01.657943 11572 raft_consensus.cc:697] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:01.675055 11382 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:01.682689 11576 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:01.724870 11556 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:01.727007 11621 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:01.728164 11631 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:03:01.728518 11544 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:01.730924  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 460) took 1685 ms. Trace:
I20260709 13:03:01.731361  8822 rpcz_store.cc:276] 0709 13:03:00.045986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.450556 (+404570us) service_pool.cc:224] Handling call
0709 13:03:00.450989 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.020241 (+569252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.020287 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.021840 (+  1553us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.293092 (+271252us) spinlock_profiling.cc:243] Waited 8.97 ms on lock 0x7b48000cd158. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:03:01.345572 (+ 52480us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:01.345652 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.345765 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.730579 (+384814us) raft_consensus.cc:1685] finished
0709 13:03:01.730602 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.730856 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":424777,"raft.queue_time_us":454939,"raft.run_cpu_time_us":3434,"raft.run_wall_time_us":384584,"spinlock_wait_cycles":20641024,"thread_start_us":382450,"threads_started":2,"wal-append.queue_time_us":588031}
I20260709 13:03:01.734151  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:01.735369 11549 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:01.735105 11596 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:01.736028 11413 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:01.742849 11623 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:01.704710 11506 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:01.745941 11515 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:01.701821 11276 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:01.750996  8676 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:03:01.702235 11557 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.505s	user 0.015s	sys 0.000s
I20260709 13:03:01.752220  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:01.752751 11258 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:03:01.742909  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 52205.7us
W20260709 13:03:01.756911  8722 leader_election.cc:336] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.194s (SENT)
I20260709 13:03:01.757339  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:01.757628  8793 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:01.758761  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:01.752964  8676 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:03:01.760628  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:01.762507  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 451803us
I20260709 13:03:01.766948  8681 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
W20260709 13:03:01.767791  8827 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.009s	sys 0.006s
W20260709 13:03:01.768144  8827 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.009s	sys 0.006s
W20260709 13:03:01.771907  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 66555.3us
I20260709 13:03:01.772825  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:01.776130  8676 raft_consensus.cc:2410] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:03:01.771624  8579 catalog_manager.cc:5697] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 4 to 7, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 7 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:01.753539 11200 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:01.774142  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:01.782248  8681 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:01.764991 11557 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.766897 11622 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:01.778023 11590 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:01.794723  8676 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:01.796502 11636 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:03:01.802683  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.798452  8579 catalog_manager.cc:5697] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 2 to 5, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 5 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.694720  8719 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:01.816051  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 430964us
I20260709 13:03:01.817303  8794 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:01.782974 11607 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:01.784224 11634 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:01.824501  8676 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:01.772823 11641 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:01.827366  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 132770us
I20260709 13:03:01.828118  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:01.831595  8825 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 3.
I20260709 13:03:01.831347  8579 catalog_manager.cc:5697] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 3 to 5, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 5 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:03:01.833371  9355 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:01.767534 11226 raft_consensus.cc:697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:01.851550 11226 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:01.853415 11642 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:03:01.834833  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:01.788762 11623 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:03:01.806180 11549 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:03:01.802839 11622 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.857131 11549 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:01.857136 11622 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:01.808599 11557 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:01.806901 11211 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:01.819109 11442 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:03:01.861496 11637 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:01.862102 11637 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.862471 11637 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:01.806478 11576 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:03:01.818724 11556 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:01.865525 11556 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:01.339311  8575 rpcz_store.cc:269] 0709 13:02:46.183674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.184552 (+   878us) service_pool.cc:224] Handling call
0709 13:03:01.297596 (+15113044us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:53.193148 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:53.329883 (+136735us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:53.331190 (+  1307us) write_op.cc:435] Acquired schema lock
0709 13:02:53.337451 (+  6261us) tablet.cc:662] Decoding operations
0709 13:02:53.368160 (+ 30709us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:53.368284 (+   124us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:53.368332 (+    48us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:02:53.370086 (+  1754us) tablet.cc:701] Row locks acquired
0709 13:02:53.370147 (+    61us) write_op.cc:260] PREPARE: finished
0709 13:02:53.370420 (+   273us) write_op.cc:270] Start()
0709 13:02:53.392425 (+ 22005us) write_op.cc:276] Timestamp: P: 1783602173370337 usec, L: 0
0709 13:02:53.392783 (+   358us) op_driver.cc:348] REPLICATION: starting
0709 13:02:53.680834 (+288051us) log.cc:844] Serialized 5124 byte log entry
0709 13:02:54.041052 (+360218us) op_driver.cc:464] REPLICATION: finished
0709 13:02:54.455876 (+414824us) spinlock_profiling.cc:243] Waited 4.87 ms on lock 0x7b54002d7dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f4e28 00007f32e5f2f581 00007f32e5f2ed58 00007f32e5f339f1 00007f32e5f339a9 00007f32e5f33939 00007f32e5f33901 00007f32e5f32b2d 000000000038f354 000000000038f199
0709 13:02:54.603756 (+147880us) write_op.cc:301] APPLY: starting
0709 13:02:54.621069 (+ 17313us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:54.737378 (+116309us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:54.737411 (+    33us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:54.741347 (+  3936us) spinlock_profiling.cc:243] Waited 57.4 ms on lock 0x7b180033b440. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:02:54.892903 (+151556us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:54.893330 (+   427us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:02:54.893403 (+    73us) write_op.cc:312] APPLY: finished
0709 13:02:54.898324 (+  4921us) log.cc:844] Serialized 95 byte log entry
0709 13:02:54.898572 (+   248us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:54.898797 (+   225us) write_op.cc:454] Released schema lock
0709 13:02:54.898857 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:54.967896 (+ 69039us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:54.969291 (+  1395us) write_op.cc:454] Released schema lock
0709 13:02:59.707144 (+4737853us) spinlock_profiling.cc:243] Waited 8.1 ms on lock 0x7b180033b440. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c557 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
Metrics: {"mutex_wait_us":752078,"reported_tablets":30,"spinlock_wait_cycles":5120,"child_traces":[["op",{"apply.queue_time_us":537011,"apply.run_cpu_time_us":7252,"apply.run_wall_time_us":334394,"num_ops":13,"prepare.queue_time_us":701361,"prepare.run_cpu_time_us":20408,"prepare.run_wall_time_us":1281646,"raft.queue_time_us":63356,"raft.run_cpu_time_us":21752,"raft.run_wall_time_us":462632,"replication_time_us":645850,"spinlock_wait_cycles":161762304,"thread_start_us":1022568,"threads_started":4,"wal-append.queue_time_us":149138,"wal-append.run_cpu_time_us":6158,"wal-append.run_wall_time_us":2004959}]]}
I20260709 13:03:01.845227  9382 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:01.796525 11258 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:01.875348  8579 catalog_manager.cc:5697] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 1 to 4, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 4 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.847962 11646 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:01.848814 11393 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:01.849833  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:01.823168 11548 consensus_meta.cc:287] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.605s	user 0.011s	sys 0.005s
I20260709 13:03:01.810173 11544 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:01.857777 11549 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:01.996238  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:01.879932  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:01.996896  8753 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:01.952754  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.991923  8752 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:01.989156  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:02.000751  8749 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:01.842425  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:01.968616 11646 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.875749 11644 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:02.012462  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:02.024689  8755 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:02.018693  8749 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:02.022967  8579 catalog_manager.cc:5697] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 2 to 4, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 4 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.041442  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:02.049139  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:02.050765  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.055192 11607 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:02.056058 11607 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:02.056576 11607 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:02.060177  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 217846us
I20260709 13:03:01.993793 11541 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:02.061764  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:02.064479  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1451) took 1020 ms. Trace:
I20260709 13:03:02.021378 11646 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:02.003342 11565 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:03:02.069729  8755 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:02.064951  8829 rpcz_store.cc:276] 0709 13:03:01.043996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.205420 (+161424us) service_pool.cc:224] Handling call
0709 13:03:01.205711 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.206024 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:01.414262 (+208238us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.063856 (+649594us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.064081 (+   225us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.064135 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:02.064194 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.064265 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.064412 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75815,"prepare.queue_time_us":55714,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":121550,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":11040,"thread_start_us":556085,"threads_started":2}
I20260709 13:03:02.078181  8681 raft_consensus.cc:1217] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Deduplicated request from leader. Original: 4.3->[5.4-8.5]   Dedup: 5.4->[8.5-8.5]
W20260709 13:03:02.079267  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 89413.7us
I20260709 13:03:02.240877  8681 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: [5.4-8.5]
I20260709 13:03:02.234098  8753 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
W20260709 13:03:02.150635  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.226559  8647 leader_election.cc:336] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.598s (SENT)
I20260709 13:03:02.242813  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:02.215143 11556 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:02.189320  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 445) took 1232 ms. Trace:
I20260709 13:03:02.083499 11413 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:03:02.068579 11224 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:02.000275 11548 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.005798 11544 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.153981  8675 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:02.132099 11647 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.968866 11572 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.132084  8579 catalog_manager.cc:5697] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 1 to 3, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:02.151175 11416 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:02.032442 11644 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:03:02.213268 11504 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.675s	user 0.014s	sys 0.002s
W20260709 13:03:02.247059  8677 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.006s	sys 0.006s
W20260709 13:03:02.298900  8677 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.006s	sys 0.006s
I20260709 13:03:02.250052  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:02.245582  8679 rpcz_store.cc:276] 0709 13:03:00.956789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.957510 (+   721us) service_pool.cc:224] Handling call
0709 13:03:00.958042 (+   532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.179494 (+221452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.179547 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.754223 (+574676us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b2c000014c8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e0df2a11 00007f32e0e02372 00007f32e2534a01 00007f32e24f943a 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2
0709 13:03:01.754846 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.133529 (+378683us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:02.133585 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.133703 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.188912 (+ 55209us) raft_consensus.cc:1685] finished
0709 13:03:02.188938 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.189236 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4119,"raft.queue_time_us":20585,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":12315,"spinlock_wait_cycles":2908928,"thread_start_us":523357,"threads_started":2,"wal-append.queue_time_us":407988}
W20260709 13:03:02.254189  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.248793  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:02.069800 11621 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:02.241592 11382 consensus_meta.cc:287] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.000s
W20260709 13:03:02.263324  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 197859us
W20260709 13:03:02.240023 11420 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.582s	user 0.008s	sys 0.004s
I20260709 13:03:02.076088 11623 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.194s
W20260709 13:03:02.263043 11649 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:03:02.157472 11549 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.239622 11590 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:02.296944  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:02.250198 11652 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:02.255478  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 193875us
I20260709 13:03:02.302183  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1592) took 1205 ms. Trace:
W20260709 13:03:02.305043  8647 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.671s (SENT)
I20260709 13:03:02.356014  8794 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:02.355769  8677 rpcz_store.cc:276] 0709 13:03:01.096430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.380740 (+284310us) service_pool.cc:224] Handling call
0709 13:03:01.408163 (+ 27423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.408469 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:01.425750 (+ 17281us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.299278 (+873528us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.300946 (+  1668us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:02.301023 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.301106 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.302116 (+  1010us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17091,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":32706,"raft.queue_time_us":14561,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":872944,"threads_started":2,"wal-append.queue_time_us":395677}
I20260709 13:03:02.358482  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:02.359468  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:02.360164  8679 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
W20260709 13:03:02.334120  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 84150.7us
I20260709 13:03:02.360327  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:02.361061  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:02.361280  8677 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:02.361658  8679 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:02.362845  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:02.339310 11650 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.141s	user 0.002s	sys 0.000s
I20260709 13:03:02.339656  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:02.365202  8756 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:02.364089  8722 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:02.341092 11276 consensus_meta.cc:287] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.002s
I20260709 13:03:02.340740 11382 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:02.330862 11226 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:02.343364 11420 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:02.343856 11327 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.939s	user 0.016s	sys 0.000s
I20260709 13:03:02.296428 11660 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:02.270530 11647 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:02.345643 11544 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:02.345887 11645 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:03:02.347747 11548 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:02.349550  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:02.310819 11393 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:02.301748  8579 catalog_manager.cc:5697] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 2 to 4, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 4 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.349678 11637 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:02.399399 11607 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.400620  8579 catalog_manager.cc:5697] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index 3 to 4, VOTER 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) evicted. New cstate: current_term: 1 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.401793 11607 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:02.402717 11327 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.402827  8579 catalog_manager.cc:5697] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 1 to 2, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 2 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
W20260709 13:03:02.365734  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.351846 11658 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.053s	user 0.002s	sys 0.000s
I20260709 13:03:02.329355 11621 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:02.405683 11621 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:02.299968 11644 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:02.407069  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:02.351665 11653 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.101s	user 0.002s	sys 0.000s
I20260709 13:03:02.368170 11636 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:02.366564 11590 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:02.377944  8794 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:02.411139  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 473) took 1205 ms. Trace:
I20260709 13:03:02.411580  8826 rpcz_store.cc:276] 0709 13:03:01.206012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.547297 (+341285us) service_pool.cc:224] Handling call
0709 13:03:01.547858 (+   561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.548266 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.548301 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.548943 (+   642us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.762871 (+213928us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:01.762917 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.763072 (+   155us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.410881 (+647809us) raft_consensus.cc:1685] finished
0709 13:03:02.410907 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.411074 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19744,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":16066,"thread_start_us":213465,"threads_started":1,"wal-append.queue_time_us":464118}
I20260709 13:03:02.412979 11607 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:03:02.381280  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 76303.8us
I20260709 13:03:02.392421  8679 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
W20260709 13:03:02.415079  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51066.2us
I20260709 13:03:02.416137 11636 consensus_queue.cc:1059] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:02.417062 11636 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:02.391462  8825 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:02.329842 11504 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.404857  8579 catalog_manager.cc:5697] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 1 to 4, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 4 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.428288  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 451) took 1122 ms. Trace:
I20260709 13:03:02.345671 11541 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:02.397507 11657 log.cc:927] Time spent T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:03:02.403683  8748 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:02.412268  8794 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:02.417783  8722 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:02.367882 11667 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.071s	user 0.002s	sys 0.000s
I20260709 13:03:02.449577 11636 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:03:02.393711 11360 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.857s	user 0.011s	sys 0.004s
I20260709 13:03:02.450070 11636 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.450546 11636 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:02.450501 11360 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.453207 11360 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:02.397090 11506 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.653s	user 0.012s	sys 0.000s
I20260709 13:03:02.454041 11506 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.456423 11506 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:02.457267  8680 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:02.457913  8822 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:02.432354  8672 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
W20260709 13:03:02.395705 11619 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.741s	user 0.005s	sys 0.009s
I20260709 13:03:02.459117  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:02.459621 11619 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:02.460120 11457 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:02.435180  8676 rpcz_store.cc:276] 0709 13:03:01.309352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.969952 (+660600us) service_pool.cc:224] Handling call
0709 13:03:02.001337 (+ 31385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.297155 (+295818us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.297186 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.297229 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:02.297272 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.428022 (+130750us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b5400257bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:02.428072 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.428220 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679796608}
W20260709 13:03:02.415050  8647 leader_election.cc:336] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.672s (SENT)
I20260709 13:03:02.462384  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:02.463193  8830 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:02.463794  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 48784.1us
I20260709 13:03:02.465801  8823 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:02.466257  8579 catalog_manager.cc:5697] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.466320  8823 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:02.470279  8794 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:02.425307 11662 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:03:02.376924 11565 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:02.471735  8647 leader_election.cc:336] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.753s (SENT)
I20260709 13:03:02.390175 11556 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.415023 11211 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:02.390512 11623 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:02.473315  8794 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:02.488966 11623 raft_consensus.cc:697] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:02.472648  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:02.490386 11623 consensus_queue.cc:237] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:02.491324 11360 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:02.491652 11515 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:02.477584  8827 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:02.418251 11576 raft_consensus.cc:3055] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:02.475660  8823 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:02.493108 11576 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:02.493417  8823 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:02.478809  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:02.421260 11327 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:02.419147 11590 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.480895  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:02.377996 11276 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:02.529042  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:02.530108  8747 raft_consensus.cc:2368] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7.
I20260709 13:03:02.533501  8756 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:02.486855 11556 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.537861  8823 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:02.548779  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1657) took 1500 ms. Trace:
I20260709 13:03:02.549259  8823 rpcz_store.cc:276] 0709 13:03:01.048243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.207163 (+158920us) service_pool.cc:224] Handling call
0709 13:03:01.207468 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.465697 (+1258229us) spinlock_profiling.cc:243] Waited 65.3 ms on lock 0x7b54001d4d20. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2470dc0 00007f32e24fb1c0 00007f32e24fc231 00007f32e24f8d8f 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c
0709 13:03:02.548339 (+ 82642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.548681 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150107904}
I20260709 13:03:02.547259  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:02.552178  8751 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:02.475757 11506 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:02.432051 11382 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:02.443389 11483 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.430158 11647 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:02.506232  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:02.509063  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:02.566896  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:02.528440  9382 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:02.510145  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:02.568320  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:02.569288  8826 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:02.470825 11619 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:02.471974 11596 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:02.494257 11576 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:02.527940  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 67195.4us
I20260709 13:03:02.477205 11541 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:02.577104  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:02.554644  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:02.564805 11493 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:03:02.487233 11636 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:02.588431  9381 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:03:02.558501  8829 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:02.559491  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:02.591763  8750 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:02.592967  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:02.375988 11652 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.565452  8824 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:02.596778 11258 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:02.601030 11258 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.574858 11483 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:02.602512 11622 consensus_meta.cc:287] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.745s	user 0.011s	sys 0.003s
I20260709 13:03:02.574719 11596 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:03:02.606487  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.535720 11515 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.578050  8749 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:02.584931 11556 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:02.632753  8754 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:02.615664  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 452) took 1059 ms. Trace:
I20260709 13:03:02.633414  8674 rpcz_store.cc:276] 0709 13:03:01.556123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.010969 (+454846us) service_pool.cc:224] Handling call
0709 13:03:02.029652 (+ 18683us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.030188 (+   536us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:02.268822 (+238634us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.498459 (+229637us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.499020 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.499182 (+   162us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:02.499219 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.499297 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.615343 (+116046us) raft_consensus.cc:1685] finished
0709 13:03:02.615374 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.615581 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":334048,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":145886,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":25248,"thread_start_us":358333,"threads_started":2,"wal-append.queue_time_us":285378}
I20260709 13:03:02.615805  9389 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:02.572762  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:02.594414  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:02.578387  8823 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:02.632304  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.556597  8825 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:03:02.639458  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:02.640450  8825 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:02.644797  8749 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:02.645969  8747 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:02.646927  9355 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:02.652331  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:02.652645  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:02.653825  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:02.655555  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:02.655616  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:02.656579  8754 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:02.657860  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:02.659186  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:02.659894  8750 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:02.663406 11555 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:02.663698  8747 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:02.663929 11555 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:02.665081 11555 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.666208 11555 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:02.668095  8673 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:02.669741  8673 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:03:02.670789  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:02.671624  9380 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:02.672438 11679 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:03:02.672993 11342 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:02.673568 11342 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:02.674361 11680 log.cc:927] Time spent T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:03:02.610600 11416 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:02.675817  8753 raft_consensus.cc:2376] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7. Re-sending same reply.
I20260709 13:03:02.675921 11416 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:02.608630 11596 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:02.678067  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:02.678910 11665 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:02.679288 11258 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:02.679648 11665 raft_consensus.cc:697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:02.679821  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1593) took 1583 ms. Trace:
I20260709 13:03:02.680752 11665 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.682819 11576 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:02.670867  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:02.680351  8678 rpcz_store.cc:276] 0709 13:03:01.096781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.390437 (+293656us) service_pool.cc:224] Handling call
0709 13:03:01.391007 (+   570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.502846 (+111839us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.502897 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.571546 (+ 68649us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.824093 (+252547us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:01.824138 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.679311 (+855173us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540033a4d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:02.679377 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.679430 (+    53us) raft_consensus.cc:1685] finished
0709 13:03:02.679445 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.679718 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7465,"raft.queue_time_us":8448,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":31839,"spinlock_wait_cycles":238090496,"thread_start_us":148485,"threads_started":1,"wal-append.queue_time_us":518205}
I20260709 13:03:02.686726  8755 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: [5.4-8.5]
I20260709 13:03:02.698114  8750 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:02.701542  8674 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:02.695886  8719 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:02.687798  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:02.701262  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:02.705241  8826 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:02.733986  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 46257.1us
I20260709 13:03:02.736148  8645 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:02.706203  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:02.741422  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:02.758131  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 53690.5us
I20260709 13:03:02.742465  8823 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:02.760761  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:02.761695  8823 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
W20260709 13:03:02.763842  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 268045us
W20260709 13:03:02.770329  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.771065  8647 leader_election.cc:336] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.671s (SENT)
I20260709 13:03:02.771593  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:02.777549  8825 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:02.772046  8722 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:02.706458 11420 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:02.703440 11457 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:02.646188 11515 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.703436 11666 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:02.712806 11226 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:02.796887  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:02.812794  8826 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:02.813987  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:02.814957  8826 raft_consensus.cc:2410] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:02.794021  8719 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:02.802287  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:02.796171  8647 leader_election.cc:336] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.735s (SENT)
I20260709 13:03:02.818387  9381 raft_consensus.cc:2410] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 8 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
W20260709 13:03:02.819010  8647 leader_election.cc:336] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.948s (SENT)
I20260709 13:03:02.819749  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:02.755900 11623 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:02.763353 11556 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:02.711267 11360 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.099s
I20260709 13:03:02.784204 11483 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:02.831579  9386 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:02.763269 11382 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:03:02.795333 11625 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:02.810920 11393 raft_consensus.cc:2764] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:03:02.636633 11622 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.826404 11623 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:02.828377  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:02.790655 11506 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:03:02.791442 11607 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:03:02.839828  9387 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:02.840513  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:02.842254  8826 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:02.819250 11457 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:02.815637 11644 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:02.815809 11579 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:03:02.836762 11327 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:02.862231  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:02.863377  8829 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:02.838125 11625 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.821748 11666 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:02.841897 11652 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:02.867394 11258 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.837347 11636 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:02.887745  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:02.852254 11646 consensus_meta.cc:287] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.709s	user 0.011s	sys 0.011s
I20260709 13:03:02.887260 11544 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:02.893919  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 74962.4us
I20260709 13:03:02.875078 11556 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:02.892973  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 99068.5us
I20260709 13:03:02.909286 11556 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:02.917999 11666 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:02.904778 11661 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:03:02.919998 11644 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:02.920203  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:02.920892 11515 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:02.901232 11457 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:02.905202 11258 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:02.923278 11682 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:03:02.889950 11625 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:02.925575 11622 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:02.907820  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:02.924829 11258 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:02.933970  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:02.934214  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:02.907316  8719 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:02.934860  8681 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:02.934948  8678 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:02.888408 11393 raft_consensus.cc:3055] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:02.869823 11327 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.945416  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:02.953378  8673 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:02.949504 11593 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:02.951618  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1459) took 1676 ms. Trace:
I20260709 13:03:02.959625  8824 rpcz_store.cc:276] 0709 13:03:01.275646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.602634 (+326988us) service_pool.cc:224] Handling call
0709 13:03:01.665043 (+ 62409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.596355 (+931312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.951272 (+354917us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b5400126954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:02.951540 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2070311808}
I20260709 13:03:02.960331  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 476) took 1176 ms. Trace:
I20260709 13:03:02.960816  8822 rpcz_store.cc:276] 0709 13:03:01.783843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.784004 (+   161us) service_pool.cc:224] Handling call
0709 13:03:01.792498 (+  8494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.458349 (+665851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.960026 (+501677us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b5400126454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:02.960260 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1529913216}
I20260709 13:03:02.943984  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:02.957792 11382 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:02.958545 11327 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:02.952787 11393 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:03:02.967651  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 59919.5us
W20260709 13:03:02.965826  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 58608.3us
I20260709 13:03:02.914192  9380 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:02.970356  8719 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:02.979329  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:02.979569  8647 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:02.914098  9382 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:02.983546 11360 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:02.930769 11623 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:02.991008  9383 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:03:02.966455  8829 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:02.976128 11393 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:02.977443 11636 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:03.002645  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:03.003127  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:02.978715 11622 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:03.006449  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:03.008013 11393 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:03.003569 11258 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
W20260709 13:03:03.010130 11655 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.836s	user 0.000s	sys 0.011s
I20260709 13:03:03.009886 11596 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.020102 11636 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:03.011312 11621 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.004s
I20260709 13:03:02.999784 11360 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:03.005594  8822 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:03.003298  9385 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:02.907577 11646 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:03.019119  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 50555.6us
I20260709 13:03:03.021868  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1597) took 1923 ms. Trace:
W20260709 13:03:03.028964  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 48033.9us
I20260709 13:03:03.076670  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:02.994738 11686 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:03.011111 11276 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:03.011615 11622 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:03.034121  8673 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:03.076253  8672 rpcz_store.cc:276] 0709 13:03:01.098059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.561878 (+463819us) service_pool.cc:224] Handling call
0709 13:03:01.637208 (+ 75330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.458863 (+821655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.021244 (+562381us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b54003395d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:03.021770 (+   526us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1828313856}
I20260709 13:03:03.075253 11493 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.083716  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:03.028909  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:03.088338  8829 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:03:03.092669  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:03.043242 11636 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:03.095765  8829 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:03.097641  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:03.050911 11621 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.050519 11360 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:03.210397  8756 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:03:03.161417  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 77805us
I20260709 13:03:03.231181  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:03.155891  8680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1601) took 2030 ms (client timeout 1872 ms). Trace:
W20260709 13:03:03.233217  8680 rpcz_store.cc:269] 0709 13:03:01.125373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.285273 (+159900us) service_pool.cc:224] Handling call
0709 13:03:03.155612 (+1870339us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540033aed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:03.155802 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2490260736}
I20260709 13:03:03.051540 11226 raft_consensus.cc:697] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:03:03.127198 11656 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.857s	user 0.004s	sys 0.010s
I20260709 13:03:03.126520  8672 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:03:03.132804  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.227995  8822 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:03.149998  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1734) took 1180 ms. Trace:
I20260709 13:03:03.248831  8748 rpcz_store.cc:276] 0709 13:03:01.969339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.969489 (+   150us) service_pool.cc:224] Handling call
0709 13:03:01.969794 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.446108 (+476314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.149669 (+703561us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b5400157cd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:03.149882 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":997485824}
I20260709 13:03:03.090169 11555 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:03.090271 11622 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:03.250306 11555 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:03.259902 11555 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:03.222162  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 410409us
I20260709 13:03:03.214517 11250 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:03.262212 11250 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:03.263803 11250 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.082530 11646 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:03.269529  8821 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:03.093083 11276 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:03.206573 11663 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.287143  8829 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:03:03.315610  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:03.234722 11382 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:03.317504  9355 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:03.313827  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:03.279971  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.321398  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:03.299881  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68834.8us
I20260709 13:03:03.095664 11493 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.351854  8822 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:03.319255  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:03.308609 11695 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:03:03.247503 11652 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:03.319906 11276 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.310798 11621 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:03.247495 11204 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.355787  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:03.355996  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:03.349731  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:03.319972 11250 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:03.103536 11625 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.311208 11637 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:03.358477  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:03.311262 11226 consensus_queue.cc:237] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.364262  8829 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:03.297909 11686 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:03.368568 11204 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.370842 11596 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:03.364779  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:03.351472  9382 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:03.349711 11646 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:03.348867 11666 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:03.377687 11688 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:03.350924 11690 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:03.384557 11416 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.353397 11623 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:03:03.359226 11660 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:03.392360 11493 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:03.398207  8752 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:03.398470  8647 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:03.398742  8822 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:03.420473  8829 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:03:03.434263  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:03.437331  8677 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:03.437525  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:03.432803  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:03.438470  8829 raft_consensus.cc:2410] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:03:03.403985 11413 raft_consensus.cc:2764] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:03:03.397665 11652 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:03.328990 11663 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.422024 11544 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:03.452219  8829 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:03.456548  8822 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:03.456712  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:03.440129 11660 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.435316 11646 raft_consensus.cc:697] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:03.425110 11327 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.482995  8823 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:03:03.459653 11413 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:03.433539 11666 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.460161 11544 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:03.469558 11416 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:03.470199 11660 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:03.490522  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:03.477172  8722 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:03.478631 11493 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:03.496330 11688 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:03.472414 11598 consensus_queue.cc:1059] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:03:03.513675  8722 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:03.503039 11623 raft_consensus.cc:2764] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:03:03.481972 11576 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.023s
W20260709 13:03:03.521857 11692 log.cc:927] Time spent T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 13:03:03.537508  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.540535  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1455) took 2271 ms. Trace:
I20260709 13:03:03.541666  8828 rpcz_store.cc:276] 0709 13:03:01.268942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.550218 (+281276us) service_pool.cc:224] Handling call
0709 13:03:01.550646 (+   428us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:01.551086 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.551128 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:01.551769 (+   641us) log.cc:844] Serialized 52 byte log entry
0709 13:03:01.746044 (+194275us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b48000cd158. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:03:01.973785 (+227741us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:02.064251 (+ 90466us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.082431 (+ 18180us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.667685 (+585254us) spinlock_profiling.cc:243] Waited 4.77 ms on lock 0x7b1800178360. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:03:03.487671 (+819986us) spinlock_profiling.cc:243] Waited 9.7 ms on lock 0x7b1800178360. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:03:03.539729 (+ 52058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.540201 (+   472us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.540264 (+    63us) raft_consensus.cc:1685] finished
0709 13:03:03.540282 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.540465 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":165628,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":1047,"raft.queue_time_us":216441,"raft.run_cpu_time_us":6405,"raft.run_wall_time_us":247845,"spinlock_wait_cycles":57034880,"thread_start_us":473381,"threads_started":2,"wal-append.queue_time_us":597026}
I20260709 13:03:03.495529 11622 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.522275 11675 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:03.547992  8722 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.807s (SENT) [suppressed 39 similar messages]
I20260709 13:03:03.521253 11652 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.478793 11686 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.538501 11276 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:03.566404  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 168018us
I20260709 13:03:03.575162 11563 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:03.547518 11565 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:03.577713  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:03.520200 11493 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.525625 11688 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.588017  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:03.575908 11563 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.533192 11696 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:03:03.567340 11598 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:03.594748  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:03.587572 11204 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:03.599215 11706 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:03.599949 11706 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:03.601083 11563 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:03.601187 11706 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:03.602834  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.595973 11555 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:03:03.603452  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.598675  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:03.523149 11646 consensus_queue.cc:237] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.608880  8647 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:03.580049 11665 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.577533 11675 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:03.588701 11690 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:03.612828  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:03.614030 11576 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:03.614804 11710 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:03.615319 11493 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:03.616510 11565 raft_consensus.cc:697] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:03.617689 11565 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.620693  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:03.590116 11544 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.621582  8672 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 2.
I20260709 13:03:03.623813 11596 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.625463  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:03.593892 11625 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:03.581671 11413 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:03:03.593633 11598 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:03.637485 11493 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:03.632404 11596 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.607297 11555 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.571246 11705 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
W20260709 13:03:03.644729  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.593598 11707 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:03:03.610116 11663 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:03.635739 11393 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:03.671650  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:03.646463 11493 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:03.681219  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:03.682312  8680 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:03.685117 11636 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.534s	user 0.006s	sys 0.007s
I20260709 13:03:03.672540 11393 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.685659 11636 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.679008 11493 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:03.651234 11555 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.687453 11393 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:03.687709 11360 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.689018 11555 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:03.670958 11327 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:03.690001 11360 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:03.691231 11622 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:03.691599 11636 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:03.692014 11598 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.692554 11590 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:03.693593 11598 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:03.665944 11707 consensus_queue.cc:1243] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:03:03.631083 11665 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.710733 11596 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:03.717291 11708 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:03:03.720881  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1463) took 1086 ms. Trace:
I20260709 13:03:03.653322 11710 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:03.721771  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:03.730536  8674 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:03.721570  8829 rpcz_store.cc:276] 0709 13:03:02.634295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.455149 (+820854us) service_pool.cc:224] Handling call
0709 13:03:03.455577 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:03.455830 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.455854 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:03.470824 (+ 14970us) log.cc:844] Serialized 26 byte log entry
0709 13:03:03.572036 (+101212us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:03.606914 (+ 34878us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.607225 (+   311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.607309 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.720561 (+113252us) raft_consensus.cc:1685] finished
0709 13:03:03.720595 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.720788 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37359,"prepare.run_wall_time_us":44,"raft.queue_time_us":2691,"spinlock_wait_cycles":29531904,"thread_start_us":73329,"threads_started":1,"wal-append.queue_time_us":149474}
I20260709 13:03:03.726560  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:03.732501  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:03.676350 11666 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:03.741392  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:03.742478  8829 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:03.754941  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:03.781715  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1596) took 2684 ms. Trace:
I20260709 13:03:03.782526  8675 rpcz_store.cc:276] 0709 13:03:01.097747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.406591 (+308844us) service_pool.cc:224] Handling call
0709 13:03:01.407030 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.260308 (+853278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.781312 (+1521004us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b54002413d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:03.781613 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1717430144}
W20260709 13:03:03.788810  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 125379us
I20260709 13:03:03.791697  8680 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:03.751909 11367 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.795923  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:03.796702  8824 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:03.812088 11690 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:03.812546  9385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 446) took 1020 ms. Trace:
I20260709 13:03:03.813202  9385 rpcz_store.cc:276] 0709 13:03:02.792528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.795294 (+  2766us) service_pool.cc:224] Handling call
0709 13:03:02.978975 (+183681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.087165 (+108190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.812188 (+725023us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54002369d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:03.812429 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324111104}
I20260709 13:03:03.779218 11622 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:03.814949 11622 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.815376 11622 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:03.815788 11563 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:03.816280 11563 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:03.817377  8829 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:03.817251 11563 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.755931 11625 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:03.823323  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.835285  8722 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:03.837098  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:03.840546  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:03.844412  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.849151  8824 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:03:03.850941  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 242193us
I20260709 13:03:03.851944  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:03.853611  8722 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:03.818589 11563 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:03.853703  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:03.855719  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:03.857367  8824 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:03.722790 11675 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.847782 11276 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:03.860530 11493 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.891373  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:03.839135 11590 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.889315 11705 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:03.897855  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:03.853247 11565 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:03.853730 11226 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:03.838668 11367 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.889622 11413 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:03.865952 11636 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:03.852792 11688 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:03.913230 11544 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:03.917631  8826 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:03.916996  9355 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:03.869534 11652 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:03.914813  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:03.869926 11690 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:03.921757  8824 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
W20260709 13:03:03.923117  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.927579  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1468) took 1279 ms. Trace:
I20260709 13:03:03.927881  8824 rpcz_store.cc:276] 0709 13:03:02.648613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.921261 (+1272648us) service_pool.cc:224] Handling call
0709 13:03:03.921567 (+   306us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:03.927326 (+  5759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.927528 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:03.929919  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:03.938325  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:03.916281 11705 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:03.818742 11710 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.941979 11705 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:03.907301 11706 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:03.947963 11636 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:03.948096 11226 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:03.949518 11636 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.950515 11226 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.953855 11576 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:03.907521 11276 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:03.919548 11565 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:03.920171 11413 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:03.971134  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:03.983013  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:03.925686 11483 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:03.994459 11367 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:03.995297 11499 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:04.024060  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:04.000249 11493 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:04.001951  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1467) took 1353 ms. Trace:
I20260709 13:03:03.962826 11723 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:03.957041 11706 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:03.907765 11590 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:04.009618  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:04.032488  8678 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:04.033764  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:03.925320 11544 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:04.025245  8829 rpcz_store.cc:276] 0709 13:03:02.648250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.818182 (+1169932us) service_pool.cc:224] Handling call
0709 13:03:03.824847 (+  6665us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:03.835469 (+ 10622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.835504 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:03.854833 (+ 19329us) log.cc:844] Serialized 52 byte log entry
0709 13:03:03.911257 (+ 56424us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:03.912184 (+   927us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.912991 (+   807us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.914405 (+  1414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.914502 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:04.001602 (+ 87100us) raft_consensus.cc:1685] finished
0709 13:03:04.001637 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.001860 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31216,"prepare.queue_time_us":377,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":97,"raft.queue_time_us":17297,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":15414,"spinlock_wait_cycles":15464192,"thread_start_us":52225,"threads_started":1,"wal-append.queue_time_us":52681}
I20260709 13:03:03.964039 11689 raft_consensus.cc:3055] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:03.947510 11688 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:04.053191  8796 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:03.942795 11690 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:03.983878 11707 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:04.016549  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:04.046803  9389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 443) took 1556 ms. Trace:
I20260709 13:03:04.015139 11413 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:04.055102  8796 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:04.042327  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:04.057807  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:04.030372 11499 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.041558 11706 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:04.079612  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 162746us
W20260709 13:03:04.091645  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.107510  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 52545.2us
W20260709 13:03:04.058975  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 56010.7us
I20260709 13:03:04.074086  8678 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
W20260709 13:03:04.047526 11737 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:03:04.112576  8794 leader_election.cc:336] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.575s (SENT)
I20260709 13:03:04.029843 11483 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:04.039633 11665 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:04.040143 11723 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:04.107184  9389 rpcz_store.cc:276] 0709 13:03:02.479866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.484537 (+  4671us) service_pool.cc:224] Handling call
0709 13:03:02.484981 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.635399 (+150418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.035494 (+1400095us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54002fb554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:04.035751 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300356480}
I20260709 13:03:04.403771  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:04.148300  9383 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
W20260709 13:03:04.391171  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.146109  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:04.395996  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:04.407146  8829 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
W20260709 13:03:04.165748  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.226529  9381 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
W20260709 13:03:04.164253  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.207195  9387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 439) took 2366 ms. Trace:
I20260709 13:03:04.412042  9387 rpcz_store.cc:276] 0709 13:03:01.822364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.822537 (+   173us) service_pool.cc:224] Handling call
0709 13:03:01.822985 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.842085 (+1019100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.187487 (+1345402us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400510254. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:04.188240 (+   753us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2338205440}
I20260709 13:03:04.047356 11544 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:04.436378  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.521965  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:04.400888  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.119979  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:04.053040 11689 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:03:04.125906 11733 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:03:04.386914 11660 consensus_meta.cc:287] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.866s	user 0.014s	sys 0.001s
W20260709 13:03:04.333911 11735 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:03:04.530275  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:04.532572  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 412759us
W20260709 13:03:04.535629  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 423089us
I20260709 13:03:04.535939  8825 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:04.537475  8794 leader_election.cc:336] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.605s (SENT)
I20260709 13:03:04.058566 11493 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:03:04.332974 11731 log.cc:927] Time spent T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:03:04.110829 11499 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:04.109112  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:04.532974  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:04.192742 11690 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:04.113351 11416 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:04.127527 11590 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:04.524874 11623 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:03:04.542415  8830 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1457) took 3272 ms (client timeout 1807 ms). Trace:
I20260709 13:03:04.183020 11504 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:04.142441 11696 raft_consensus.cc:3055] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:03:04.555722  8830 rpcz_store.cc:269] 0709 13:03:01.269645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.337346 (+ 67701us) service_pool.cc:224] Handling call
0709 13:03:04.542206 (+3204860us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54002288d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:04.542337 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2503357312}
I20260709 13:03:04.538100  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:04.557322  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:04.558341  8830 raft_consensus.cc:2410] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:04.563807  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:04.564973  8830 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:04.565358  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1466) took 1929 ms. Trace:
I20260709 13:03:04.566148  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:04.567488  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:04.568126  8830 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:03:04.538115 11712 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:04.581306 11723 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:04.585685  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:04.586860  8829 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:04.565796  8822 rpcz_store.cc:276] 0709 13:03:02.636139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.807906 (+1171767us) service_pool.cc:224] Handling call
0709 13:03:03.808398 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:03.808900 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.808951 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:03.809590 (+   639us) log.cc:844] Serialized 26 byte log entry
0709 13:03:03.943708 (+134118us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:03.962650 (+ 18942us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.962989 (+   339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.963109 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:04.564896 (+601787us) raft_consensus.cc:1685] finished
0709 13:03:04.564918 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.565269 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19489,"raft.queue_time_us":13585,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":8809,"spinlock_wait_cycles":22833536,"thread_start_us":133527,"threads_started":1,"wal-append.queue_time_us":222931}
W20260709 13:03:04.591463  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.598512  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 61071.9us
I20260709 13:03:04.117524 11707 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:04.407013 11706 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:04.582355 11723 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:04.607918  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:04.609156  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.615547  8825 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:04.125172 11276 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:04.585716 11590 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.184526 11565 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:04.539650 11660 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:04.585803 11696 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:04.597116 11741 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.605746 11493 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:04.636147  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:04.668385  8829 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:04.669035  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:04.665591  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 143729us
I20260709 13:03:04.669432  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:04.669765  8748 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:04.670176  8752 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:04.672818  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:04.673738  8825 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:04.673753  8755 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:04.674546  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1474) took 1971 ms. Trace:
I20260709 13:03:04.674945  8825 rpcz_store.cc:276] 0709 13:03:02.702823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.618796 (+1915973us) service_pool.cc:224] Handling call
0709 13:03:04.666973 (+ 48177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.674248 (+  7275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.674487 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:04.675475  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:04.662875  8823 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:04.677387  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 488) took 1842 ms. Trace:
I20260709 13:03:04.677618  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:04.677721  8823 rpcz_store.cc:276] 0709 13:03:02.835196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.661760 (+1826564us) service_pool.cc:224] Handling call
0709 13:03:04.662290 (+   530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.677109 (+ 14819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.677326 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:04.679718  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:04.680550  8753 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:03:04.682778  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:04.683743  8754 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:04.685534  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:04.691376 11557 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:04.693055 11709 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.099s	user 0.002s	sys 0.003s
I20260709 13:03:04.694173 11499 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:04.695315  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:04.696555  8748 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:04.696667 11499 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:04.698251  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:04.699007  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:04.700304  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:04.705547  8826 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1670) took 1935 ms (client timeout 1981 ms). Trace:
I20260709 13:03:04.706283  8751 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:03:04.670200 11327 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:04.634449 11413 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.504s	user 0.006s	sys 0.006s
W20260709 13:03:04.653712 11622 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.838s	user 0.012s	sys 0.003s
I20260709 13:03:04.716907  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:04.734267  8754 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:04.736366  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 482) took 2405 ms. Trace:
W20260709 13:03:04.706023  8826 rpcz_store.cc:269] 0709 13:03:02.770225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.854919 (+ 84694us) service_pool.cc:224] Handling call
0709 13:03:04.705280 (+1850361us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54002292d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:04.705473 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2333213440}
I20260709 13:03:04.762018  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 489) took 1894 ms. Trace:
I20260709 13:03:04.761612  8827 rpcz_store.cc:276] 0709 13:03:02.331236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.331562 (+   326us) service_pool.cc:224] Handling call
0709 13:03:02.331898 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.492766 (+160868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.736051 (+2243285us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b540032df54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:04.736287 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334613888}
I20260709 13:03:04.762444  8829 rpcz_store.cc:276] 0709 13:03:02.868054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.669230 (+1801176us) service_pool.cc:224] Handling call
0709 13:03:04.761435 (+ 92205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.761655 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.761689 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:04.761736 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:04.761768 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.761826 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.761956 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:04.765125  8829 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:04.768280  8829 raft_consensus.cc:1217] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[5.4-9.6]   Dedup: 5.4->[7.5-9.6]
I20260709 13:03:04.769089  8829 pending_rounds.cc:85] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 4
I20260709 13:03:04.779361  8829 pending_rounds.cc:107] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:03:04.633630 11723 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:04.755542  8755 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:04.754017  8750 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:04.780501  8829 raft_consensus.cc:2889] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:04.750114  8753 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:04.645051 11675 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:04.782223  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:04.655896 11342 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:04.770475 11557 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:04.784009  8826 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:04.788662  8752 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:04.789245  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1480) took 1241 ms. Trace:
I20260709 13:03:04.712024 11636 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.789660  8826 rpcz_store.cc:276] 0709 13:03:03.547537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.762448 (+1214911us) service_pool.cc:224] Handling call
0709 13:03:04.783646 (+ 21198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.788931 (+  5285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.789179 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:04.717495 11572 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.792261  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1477) took 1473 ms. Trace:
I20260709 13:03:04.812827  8825 rpcz_store.cc:276] 0709 13:03:03.318970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.746735 (+1427765us) service_pool.cc:224] Handling call
0709 13:03:04.786856 (+ 40121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.787064 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.787089 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:04.791822 (+  4733us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:04.791897 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.791993 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.792171 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:04.802314  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:04.788687 11504 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.814853  8752 raft_consensus.cc:2376] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8. Re-sending same reply.
I20260709 13:03:04.757231 11416 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.771061 11493 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.649061 11590 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:04.823305  8752 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:04.817680  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:04.712375 11741 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:04.831262  8754 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:04.838649  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1612) took 2501 ms. Trace:
I20260709 13:03:04.772053 11413 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:04.772303 11327 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:04.777691 11622 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:04.839490  8679 rpcz_store.cc:276] 0709 13:03:02.336895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.558165 (+221270us) service_pool.cc:224] Handling call
0709 13:03:02.558693 (+   528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.558936 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:02.577084 (+ 18148us) log.cc:844] Serialized 14 byte log entry
0709 13:03:04.614583 (+2037499us) spinlock_profiling.cc:243] Waited 9 ms on lock 0x7b18003348a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:03:04.837832 (+223249us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:04.837921 (+    89us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:04.837998 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.838320 (+   322us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.838532 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21573,"prepare.queue_time_us":17744,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":7538,"spinlock_wait_cycles":20691200,"thread_start_us":215591,"threads_started":1}
I20260709 13:03:04.844318  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:04.851441  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.851984  8826 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:04.855207  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:04.634231 11689 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:04.856890  8823 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:04.857111  8794 leader_election.cc:336] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.678s (SENT)
I20260709 13:03:04.857789  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:04.840701 11541 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:04.864018  8826 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:03:04.869374  8748 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:04.819119 11504 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:04.819370 11636 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:04.817234 11557 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:04.816964 11675 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:04.865641  8794 leader_election.cc:336] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.772s (SENT)
I20260709 13:03:04.884173  8794 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:04.827023 11416 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:04.828291 11493 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:04.892005 11660 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:04.892282 11327 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:04.877398 11689 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:04.819662 11572 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:04.896127  8826 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:04.879734  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:04.880820  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 209418us
I20260709 13:03:04.899032  8647 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:04.882266  9385 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 8: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:04.858752 11342 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.819177 11696 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:04.906443  8753 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:04.903539  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:04.908147  8647 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:04.886488  8754 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:04.911335 11342 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:04.834434 11723 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:04.879045 11712 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:04.916512  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:04.921101  8754 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:04.933238  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:04.934340  9386 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:04.912899 11660 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:04.913393 11493 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:04.951215  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:04.912094 11706 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:04.921171 11276 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.966171 11276 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:04.968518 11755 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:03:04.968704 11276 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:04.925479 11718 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:03:04.973011 11718 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:04.975184  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:04.918411 11747 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:04.976768  8825 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:04.977001 11747 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:04.977444 11747 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:04.977792  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1478) took 1656 ms. Trace:
I20260709 13:03:04.919685 11696 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:04.915431 11748 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:04.963799 11593 raft_consensus.cc:3055] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:04.978364  8823 rpcz_store.cc:276] 0709 13:03:03.321370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.739521 (+1418151us) service_pool.cc:224] Handling call
0709 13:03:04.739882 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.860888 (+121006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.977529 (+116641us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540041c154. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:04.977724 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268476800}
I20260709 13:03:05.004244  8823 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:04.979527  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1610) took 2643 ms. Trace:
I20260709 13:03:04.978291  8825 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:04.926896 11746 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:03:05.011490 11746 raft_consensus.cc:3055] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:05.012012 11746 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:05.013072 11746 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.014439 11636 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.014539 11622 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:04.979713 11752 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:03:05.018484 11504 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:04.980211 11712 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.021126  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1471) took 2371 ms. Trace:
I20260709 13:03:05.021629  8830 rpcz_store.cc:276] 0709 13:03:02.649566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.568856 (+1919290us) service_pool.cc:224] Handling call
0709 13:03:04.569226 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:04.598492 (+ 29266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.598543 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:04.609109 (+ 10566us) log.cc:844] Serialized 26 byte log entry
0709 13:03:04.901791 (+292682us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:04.901874 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.901991 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.020836 (+118845us) raft_consensus.cc:1685] finished
0709 13:03:05.020866 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.021049 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28698,"raft.queue_time_us":39676,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":9605,"thread_start_us":234168,"threads_started":1,"wal-append.queue_time_us":387808}
I20260709 13:03:04.928399 11753 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:05.024261  8830 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:03:04.973678 11718 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:05.027208  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:05.036610  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.006085  8676 rpcz_store.cc:276] 0709 13:03:02.336417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.462382 (+125965us) service_pool.cc:224] Handling call
0709 13:03:02.462779 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.597063 (+134284us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:02.652674 (+ 55611us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.817495 (+164821us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.817550 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:02.817616 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.690505 (+872889us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b18000baee0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:03:04.979170 (+1288665us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54001f9d54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:04.979244 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.979435 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59548,"prepare.queue_time_us":86533,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":1487,"raft.queue_time_us":32327,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":7025,"spinlock_wait_cycles":317066496,"thread_start_us":136131,"threads_started":1,"wal-append.queue_time_us":174861,"wal-append.run_cpu_time_us":1676,"wal-append.run_wall_time_us":1060124}
I20260709 13:03:05.036826  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:05.037581  8830 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:05.039608  8830 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:03:05.037528  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.040714  8753 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:05.043860  8823 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:05.046358  8823 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:05.048444  8823 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:04.925876 11382 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.109s
I20260709 13:03:05.002831 11593 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:05.039279 11276 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:05.022264 11590 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.049898 11413 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:05.051509 11416 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:05.069162 11748 raft_consensus.cc:697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:04.991380 11707 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:04.987964 11557 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:05.082866 11652 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.013309 11544 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.132158 11748 consensus_queue.cc:237] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.134002  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:05.059039  8825 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:05.135721 11765 raft_consensus.cc:3055] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:05.136173 11765 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:05.137020 11765 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:04.973151 11342 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.140108  8830 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:05.141067  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:05.141798  8754 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:05.067492  9383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 641) took 1146 ms. Trace:
I20260709 13:03:05.144160  9383 rpcz_store.cc:276] 0709 13:03:03.920952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.921084 (+   132us) service_pool.cc:224] Handling call
0709 13:03:05.067254 (+1146170us) spinlock_profiling.cc:243] Waited 76.4 ms on lock 0x7b540061b554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:05.067410 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":175692032}
W20260709 13:03:05.145607  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.068029 11504 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.071260  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.157383  8751 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:04.990998 11675 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.073622  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.166224  8753 raft_consensus.cc:2410] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:03:05.166224  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.155601  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:05.076237  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 170923us
W20260709 13:03:05.077389  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.167941  8675 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:05.169016  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:04.979598 11741 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:05.072956 11747 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.112013  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1472) took 2462 ms. Trace:
I20260709 13:03:05.014539 11746 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:05.116441  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.118232 11663 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:05.074396 11660 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.052838 11572 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:05.130117  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1677) took 1172 ms. Trace:
I20260709 13:03:05.127271 11382 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:05.067636 11753 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.137701  8825 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:05.200843  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:05.154381  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.201499  8747 raft_consensus.cc:2376] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7. Re-sending same reply.
I20260709 13:03:05.202111  8681 raft_consensus.cc:2410] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:03:05.154718  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:05.203431  8678 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:05.134425 11544 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:05.138350 11765 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:05.131929  8823 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:05.158504  8827 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:05.159018  9353 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:05.160926  9380 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:05.161608  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1465) took 2526 ms. Trace:
I20260709 13:03:05.074178  9386 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:05.173614  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.178989  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.208016  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:05.171270  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:05.208982  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 50109.7us
I20260709 13:03:05.185351  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.184701  8822 rpcz_store.cc:276] 0709 13:03:02.649869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.590497 (+1940628us) service_pool.cc:224] Handling call
0709 13:03:04.591172 (+   675us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:04.622006 (+ 30834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.622066 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:04.638968 (+ 16902us) log.cc:844] Serialized 26 byte log entry
0709 13:03:04.883880 (+244912us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:04.914248 (+ 30368us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.111480 (+197232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.111623 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.111691 (+    68us) raft_consensus.cc:1685] finished
0709 13:03:05.111713 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.111929 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21766,"prepare.queue_time_us":137167,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":135834,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":41137,"thread_start_us":329175,"threads_started":3,"wal-append.queue_time_us":253282}
I20260709 13:03:05.210286  8755 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:05.190148  8748 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:03:05.190553  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.190596  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:05.191759  8794 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:05.207223  8828 rpcz_store.cc:276] 0709 13:03:02.635563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.568100 (+932537us) service_pool.cc:224] Handling call
0709 13:03:03.568737 (+   637us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:03.569408 (+   671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.569466 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:03.574832 (+  5366us) log.cc:844] Serialized 52 byte log entry
0709 13:03:03.648907 (+ 74075us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:04.709746 (+1060839us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b180000f460. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df409455 00007f32e5ef13c7 00007f32e25039a7 00007f32e25130bf 00007f32e2513038
0709 13:03:04.923244 (+213498us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.131800 (+208556us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.150477 (+ 18677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.150584 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.161354 (+ 10770us) raft_consensus.cc:1685] finished
0709 13:03:05.161378 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.161548 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18235,"prepare.queue_time_us":229913,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":97,"raft.queue_time_us":48324,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":50620,"spinlock_wait_cycles":70329856,"thread_start_us":455377,"threads_started":3,"wal-append.queue_time_us":284033,"wal-append.run_cpu_time_us":2701,"wal-append.run_wall_time_us":1013900}
I20260709 13:03:05.131485 11413 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:05.194615  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.195392  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.227473  8676 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 5: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:05.196419  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.228240  8680 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:05.197803  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:05.197755  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:05.185917 11646 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.071s	user 0.016s	sys 0.038s
I20260709 13:03:05.153038  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.116767 11686 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.116427 11276 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.232344 11712 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:05.247516 11747 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:05.248260 11747 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:05.204288 11327 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:03:05.248739 11747 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.249176 11327 consensus_queue.cc:1243] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:03:05.240902 11686 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.166313 11652 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.194999 11393 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:05.148864 11718 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.250898 11393 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:05.251379 11393 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:05.253494 11483 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.253762 11565 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.230186  8756 raft_consensus.cc:2410] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:03:05.204684 11660 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.255682  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.256773  8679 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 3.
I20260709 13:03:05.257050  8751 raft_consensus.cc:2376] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7. Re-sending same reply.
I20260709 13:03:05.223158  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:05.211328 11637 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.198859 11549 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:03:05.206382 11753 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:05.236105 11646 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.123s	user 0.016s	sys 0.038s
I20260709 13:03:05.232486 11707 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:05.276387  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:05.241205 11548 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:05.252242 11675 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:05.229842  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:05.205106 11663 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.185092 11504 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.238312  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:05.272822  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 49775.6us
I20260709 13:03:05.198604  8829 rpcz_store.cc:276] 0709 13:03:03.957301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.767583 (+810282us) service_pool.cc:224] Handling call
0709 13:03:04.767998 (+   415us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:04.786769 (+ 18771us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:04.787076 (+   307us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:04.787930 (+   854us) log.cc:844] Serialized 52 byte log entry
0709 13:03:05.125391 (+337461us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:05.125467 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.125588 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.125664 (+    76us) raft_consensus.cc:1685] finished
0709 13:03:05.125689 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.130014 (+  4325us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":38986,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":5570,"thread_start_us":336919,"threads_started":1,"wal-append.queue_time_us":331753}
I20260709 13:03:05.274442 11763 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:05.254387 11565 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.274695  8678 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:05.206565 11382 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.277102  8747 raft_consensus.cc:2376] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8. Re-sending same reply.
I20260709 13:03:05.304479  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:05.304798 11382 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.305544 11382 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:05.305369  8747 raft_consensus.cc:2376] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6. Re-sending same reply.
W20260709 13:03:05.283470  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 53759.8us
I20260709 13:03:05.307255 11747 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.307833 11747 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.227694 11416 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:05.286427 11712 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:05.283972 11707 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.308442 11416 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.308950 11416 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:05.310437 11707 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.238119 11413 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.308502 11712 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.311486 11712 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:05.312769 11413 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.294715 11579 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:05.314904 11579 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:05.315356 11579 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:05.315536 11663 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:05.316929  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 492) took 1482 ms. Trace:
I20260709 13:03:05.317534 11504 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:05.194509 11593 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.286767  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.319988 11593 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:05.320400 11751 consensus_queue.cc:1175] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" responder_term: 7 status { last_received { term: 6 index: 6 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:05.287096 11766 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:03:05.322724  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1486) took 1313 ms. Trace:
I20260709 13:03:05.323441 11766 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:05.323941 11766 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:05.324801 11652 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.324918 11766 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:05.289846  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51650.6us
I20260709 13:03:05.317353  8827 rpcz_store.cc:276] 0709 13:03:03.834181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.763163 (+928982us) service_pool.cc:224] Handling call
0709 13:03:04.763478 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.208642 (+445164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.316617 (+107975us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b5400267ad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:05.316852 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":908009728}
I20260709 13:03:05.327340  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 485) took 2860 ms. Trace:
I20260709 13:03:05.295539 11675 consensus_queue.cc:1243] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:05.328135 11675 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:05.328938 11226 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:05.329505 11636 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:05.327812  8821 rpcz_store.cc:276] 0709 13:03:02.467205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.467340 (+   135us) service_pool.cc:224] Handling call
0709 13:03:02.480292 (+ 12952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.318689 (+838397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.327064 (+2008375us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b54003ab6d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:05.327271 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1814569728}
I20260709 13:03:05.330943 11327 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:05.331451 11327 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:05.332362 11327 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.332634 11687 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:05.333720 11327 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:05.333762  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:05.291193  8794 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:05.335017  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:05.335376  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 44338us
I20260709 13:03:05.335672  8821 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:05.334451 11687 raft_consensus.cc:3055] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:05.336827 11687 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:05.306879 11660 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.318907 11771 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:03:05.337103  8794 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:05.338140 11771 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:05.338361 11763 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:05.338546 11771 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:05.339208 11763 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:05.339682 11771 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.341275 11763 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:05.341766  8825 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:05.341985 11763 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:05.342315 11753 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.277741 11637 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.343168 11763 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.346124  8821 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:05.346506 11687 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.349023 11687 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:05.349598  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.341233 11771 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:05.326319 11766 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:05.300132  8754 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:05.323186  8826 rpcz_store.cc:276] 0709 13:03:04.008960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.869746 (+860786us) service_pool.cc:224] Handling call
0709 13:03:04.883036 (+ 13290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.896792 (+ 13756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.322331 (+425539us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b540032f354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:05.322566 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28927232}
I20260709 13:03:05.356217  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:05.338882  8794 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:05.367413 11565 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:05.368286 11686 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.368906  9355 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:05.369244 11276 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.369460 11646 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:05.369874 11646 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:05.370363 11416 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.370846 11646 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.371863  8827 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:05.308318 11747 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.375725 11416 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:05.375984 11747 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.386885  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:05.387918  8821 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:05.379666  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:05.380466  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 107666us
I20260709 13:03:05.363420 11636 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:05.404897  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:05.490921  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 48341.4us
I20260709 13:03:05.401563 11342 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:03:05.510748  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.519006  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.520519  8672 raft_consensus.cc:2410] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:03:05.397572 11250 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:05.525992  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:05.412716 11646 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:05.363335 11226 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.128s
I20260709 13:03:05.405642 11747 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:05.527213  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:05.580592  8676 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:05.534751  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.489596 11636 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:05.525615  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 146044us
W20260709 13:03:05.578943  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46704.9us
I20260709 13:03:05.540753  8796 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:05.589478  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:05.591270  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 359996us
I20260709 13:03:05.526257 11250 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:05.596161  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 55530.2us
I20260709 13:03:05.593272  8647 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:05.396786 11663 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.136s
W20260709 13:03:05.583513 11756 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:03:05.626652  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:05.405382 11686 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.178s
I20260709 13:03:05.600634  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.633559  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:05.583150 11342 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:05.626658  8796 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:05.628093  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.543814 11666 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:05.631964  8794 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:05.596621  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 311567us
I20260709 13:03:05.400722  9386 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
W20260709 13:03:05.647411  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 54243.5us
W20260709 13:03:05.648468  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.648386  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.648722  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.649283  8747 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:03:05.627369 11590 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:05.634164  8754 raft_consensus.cc:2376] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8. Re-sending same reply.
I20260709 13:03:05.507267 11665 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:03:05.592948 11636 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:05.489859 11741 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 13:03:05.631819 11637 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.659178  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.625852 11689 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.526575 11743 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 13:03:05.658869 11660 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:05.653564 11416 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:05.690969  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:05.701089  8754 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:05.591244 11675 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:03:05.649571 11504 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:05.696178  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.591401 11747 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.684206 11413 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:05.735594  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.698715  8676 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:05.652966 11565 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:05.625231 11712 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.684538 11652 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.632038 11607 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:05.630472 11250 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:05.746542  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.791832  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:05.792832  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.793118  8680 raft_consensus.cc:2410] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:05.660002 11753 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:05.727809  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:05.661223 11775 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:05.753100 11565 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.802457 11775 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.804144 11565 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:05.746642 11416 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:05.729368 11504 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:05.805148 11504 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:05.805641 11416 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.811533 11775 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.701608 11689 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.763603 11747 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.692842 11707 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:03:05.772678 11342 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.816481 11689 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.773334  9386 raft_consensus.cc:606] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:05.817133  9386 raft_consensus.cc:614] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:05.778934  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.781376  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:05.819002  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:05.819727  8747 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:05.783545  8754 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:05.822395  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.824211  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.824589  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.825479  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:05.786471  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:05.827015  8676 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:05.828024  8675 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:05.789299  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.831247  8678 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:05.786310  8679 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:05.796391  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.832964  8680 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:03:05.837215  8825 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:03:05.842116  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:05.843226  9383 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:05.850227  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:05.849527  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.849715  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:05.849578  9387 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:05.849200  8676 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:03:05.853521  8675 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:05.853816  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:05.789551 11556 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.857782  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:05.858865  9389 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:05.859045  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:05.857854  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.862144  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:05.862995  8754 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:05.842170 11393 consensus_meta.cc:287] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.588s	user 0.014s	sys 0.000s
I20260709 13:03:05.864602  8827 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
W20260709 13:03:05.864926  8719 connection.cc:256] RPC call timeout handler was delayed by -1.1022778770000059s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
I20260709 13:03:05.865429  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.751211 11413 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.866657  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:05.878427  8719 connection.cc:256] RPC call timeout handler was delayed by -1.0716778769999564s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16610000000000003s for a response
W20260709 13:03:05.869130  9380 consensus_meta.cc:287] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.632s	user 0.015s	sys 0.000s
W20260709 13:03:05.882781  8719 connection.cc:256] RPC call timeout handler was delayed by -1.0509778770000366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1684s for a response
I20260709 13:03:05.877804  8678 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:05.869763  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:05.850093 11665 consensus_queue.cc:1243] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:05.911118  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:05.911633  9385 raft_consensus.cc:2376] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7. Re-sending same reply.
I20260709 13:03:05.911986  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.912464  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:05.912735  8676 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 7: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:05.914793  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.915086  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:05.916040  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.916664  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.920208  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:05.921072  8673 raft_consensus.cc:2376] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4. Re-sending same reply.
W20260709 13:03:05.925762  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.922271  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:05.917438  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.927670  8755 raft_consensus.cc:2410] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:03:05.928174  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.928674  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:05.929273  8755 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:05.929136  8679 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:05.858111 11593 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.913570  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:05.942929  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.943744  8676 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:05.913633  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.932724  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:05.931180  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.930464  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:05.864130 11772 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.953024 11607 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.953778 11607 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.956182 11607 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:05.943104 11753 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:05.957592  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:05.957448 11753 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.959086  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:05.909633 11590 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:05.959865 11753 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:05.960944 11590 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.963106  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:05.882758 11393 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.787287 11652 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:05.966503 11652 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:05.967325  8678 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:05.968911 11772 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.966503 11393 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.971514 11772 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:05.973397 11579 consensus_meta.cc:287] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.658s	user 0.010s	sys 0.000s
I20260709 13:03:05.973893 11579 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.975608  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:05.976670 11499 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:05.976814  9353 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:05.977272 11499 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.978835 11772 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:05.979547 11665 raft_consensus.cc:2764] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:03:05.980275 11772 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:05.980561 11499 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:05.980715 11712 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:05.981490 11772 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:05.946269  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:05.987277 11773 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.239s	user 0.000s	sys 0.000s
I20260709 13:03:05.988193  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:05.909039 11276 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:05.949570  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:05.939685  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:05.866889 11556 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:05.950646 11778 log.cc:927] Time spent T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:03:05.996953  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:05.997874  8680 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:03:05.951079  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:05.998564  8747 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:05.952196  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.999439  8752 raft_consensus.cc:2410] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 9 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:03:06.000317  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:05.963073  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:05.998476  8679 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:05.982439 11327 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.648s	user 0.011s	sys 0.006s
I20260709 13:03:05.943787  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:05.984153 11766 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.633s	user 0.004s	sys 0.006s
W20260709 13:03:05.949816 11779 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:03:05.871459 11689 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.011608 11689 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.012866 11579 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:05.993863  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 44998.7us
I20260709 13:03:06.014820  8821 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:06.015782  8794 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:05.910435  9380 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
W20260709 13:03:06.016886  9380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 644) took 2059 ms (client timeout 1766 ms). Trace:
W20260709 13:03:06.017305  9380 rpcz_store.cc:269] 0709 13:03:03.957794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.958483 (+   689us) service_pool.cc:224] Handling call
0709 13:03:05.133808 (+1175325us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:03:06.016717 (+882909us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46355712}
I20260709 13:03:05.879915 11549 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.947614  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.018736 11549 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:06.019569  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.020854  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.021198 11549 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:06.014351 11689 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:06.024497 11556 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:05.947515 11593 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.026350  8753 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:06.026777  8674 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:06.027383 11593 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:06.028205 11523 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:05.996394 11777 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.030190  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:06.030038 11777 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:06.031682  8827 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:06.032656  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:06.032794  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:06.033352 11777 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:06.034148  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:06.006307 11276 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:06.038203 11276 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:06.005036  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.061465  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.063015  8680 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:06.039427  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:06.040570  8645 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:06.033643  8828 raft_consensus.cc:2393] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 4: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:06.064320  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:06.065186  8828 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 7: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:06.059230  8674 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:03:06.034148  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:06.067196  8827 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:06.043215  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:06.044361 11548 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:06.046012  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:06.073092  8749 raft_consensus.cc:2410] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:06.040467 11544 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.047891  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:06.076220  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:06.075343  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:06.003003  8755 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:05.995909 11660 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:06.052299  9380 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:06.081794  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:06.049821  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:06.052984  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:06.022336  8676 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:06.053527  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:06.054338  8750 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:06.049784 11623 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:06.054852  9381 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:06.055393 11504 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.072777 11777 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:06.057519 11413 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:06.059197  8754 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: []
W20260709 13:03:06.040719 11781 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:03:06.006052  8756 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:06.041666  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:06.068126  8753 raft_consensus.cc:2376] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7. Re-sending same reply.
I20260709 13:03:06.068784  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:06.070350  8821 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:06.074496  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:06.075152  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2154) took 1387 ms. Trace:
I20260709 13:03:06.076601  8794 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:06.049675  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:06.044956 11250 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.084208  9389 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:06.085053  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.098927  8749 rpcz_store.cc:276] 0709 13:03:04.687814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.045651 (+1357837us) service_pool.cc:224] Handling call
0709 13:03:06.075079 (+ 29428us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:06.101105  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.101105  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:06.102281  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.085495  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:06.102535  8749 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:06.082741  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:06.086491  8823 raft_consensus.cc:2393] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:06.087116  9380 raft_consensus.cc:2376] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5. Re-sending same reply.
I20260709 13:03:06.103925  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:06.104348  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:06.105069 11549 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:06.105436  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:06.105994  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:06.088318  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.088637  8796 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:06.090041  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:06.056497  8679 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:06.106577  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:06.108232  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.074846 11548 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:06.077762 11276 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:06.114167  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:06.115707  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.115928  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:06.108682  9380 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:06.109656  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.106895  9387 raft_consensus.cc:2376] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8. Re-sending same reply.
I20260709 13:03:06.117022  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:06.110930  8755 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:06.116580  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:06.117022  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.092312 11556 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:06.121066  8679 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:06.121583  8645 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:06.122864  8680 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:06.124186  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.125298  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:06.126228  8680 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:03:06.127871  8645 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:06.124286  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:06.129168  8679 raft_consensus.cc:2376] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7. Re-sending same reply.
I20260709 13:03:06.130193  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:06.131587  8679 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
W20260709 13:03:06.114029 11765 consensus_meta.cc:287] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.905s	user 0.009s	sys 0.009s
I20260709 13:03:06.123061  8796 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:06.079713 11544 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.089181 11723 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:03:06.133678 11790 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:06.148272 11790 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:06.143007 11792 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:06.135820 11652 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:06.120883 11712 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:06.094143 11689 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:03:06.155179 11276 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:06.155995 11276 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.156208 11250 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:06.156562 11276 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:06.157112  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.134301  8754 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:06.161970  8754 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:06.118221 11786 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:06.164649 11660 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.168300 11504 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:06.143201 11753 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:06.171631 11753 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:06.172376 11718 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:06.173051 11753 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.175235  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1469) took 3526 ms. Trace:
I20260709 13:03:06.142030 11777 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:06.177696 11548 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.138505  9385 raft_consensus.cc:2368] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8.
I20260709 13:03:06.136636  8794 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:06.107400 11413 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:06.093956 11647 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:03:06.143693  8673 raft_consensus.cc:2410] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:06.142076  8677 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:06.132969  8827 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:06.198202  8827 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:06.199685  8827 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:03:06.142874 11556 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:06.134258  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:06.201797  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 67686.7us
W20260709 13:03:06.186578  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 70098.1us
I20260709 13:03:06.204495  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:06.205035  8677 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:03:06.207013  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:06.209460  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:06.187640  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 51091.8us
I20260709 13:03:06.211105  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:06.140542  8680 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:03:06.179757 11687 consensus_meta.cc:287] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.830s	user 0.009s	sys 0.004s
I20260709 13:03:06.215691  8825 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:06.175868  8824 rpcz_store.cc:276] 0709 13:03:02.648920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.928555 (+1279635us) service_pool.cc:224] Handling call
0709 13:03:03.928911 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:03.957425 (+ 28514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.957469 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:03.958056 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 13:03:05.366525 (+1408469us) spinlock_profiling.cc:243] Waited 8.39 ms on lock 0x7b18003b4be0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:03:05.660120 (+293595us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:05.889988 (+229868us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.894431 (+  4443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.894564 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:06.174725 (+280161us) raft_consensus.cc:1685] finished
0709 13:03:06.174871 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.175128 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27045,"prepare.queue_time_us":32619,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":28373,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":52353,"spinlock_wait_cycles":19292032,"thread_start_us":494560,"threads_started":2,"wal-append.queue_time_us":1812051}
I20260709 13:03:06.218956  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:06.220391  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:06.222594  8754 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:03:06.204023 11548 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:06.204331 11504 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:06.203469 11327 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:06.218564 11579 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:06.245857 11749 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:06.225943 11790 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:06.218562 11652 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:06.253273  8645 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:06.221741 11765 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:06.226173 11416 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:06.268224  9389 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:06.280016  8796 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:06.252451 11579 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:06.254806 11544 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:06.304337  8825 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:06.294765  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:06.308522  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:06.308884  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:06.309713  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:06.270884 11646 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.691s	user 0.009s	sys 0.006s
I20260709 13:03:06.260262 11416 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:06.316669  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:06.318497  8825 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:06.314635  9388 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:06.283630 11504 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:06.303233 11590 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:06.287930 11749 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:06.337963  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.338667  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.353827  8674 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:06.356096  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:06.358495  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:06.375675  8750 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:06.377092  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:06.378355  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:06.379166  8679 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:06.379909  8750 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:06.380525  8679 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:06.359941  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:06.341274 11749 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.361311  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:06.362689  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:06.340258 11504 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.385540 11579 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:06.365140 11636 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.634s	user 0.008s	sys 0.003s
W20260709 13:03:06.338088 11800 log.cc:927] Time spent T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 13:03:06.367098  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.341660 11590 raft_consensus.cc:697] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:06.388082 11636 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.412891  8750 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:06.382421 11777 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:06.414402 11556 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:06.417549  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:06.418320 11504 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:06.419090  8680 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:06.457012  8680 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:06.420504 11636 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:06.393308 11416 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:06.421387  8748 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:06.424407 11276 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.373099 11623 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:06.426890 11711 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.427567  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:06.427708  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 223310us
I20260709 13:03:06.428113 11749 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:06.428860 11647 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:06.429960  8756 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:06.432096 11549 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.435324  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2160) took 1260 ms. Trace:
I20260709 13:03:06.453986  8675 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:06.444584  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:06.472048  8680 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:06.624261  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 500) took 1256 ms. Trace:
I20260709 13:03:06.752612 11647 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:06.752488  8821 rpcz_store.cc:276] 0709 13:03:05.367861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.590575 (+222714us) service_pool.cc:224] Handling call
0709 13:03:06.624022 (+1033447us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b540032ee54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:06.624182 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":874575616}
I20260709 13:03:06.754050 11647 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:06.623550  9386 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:33196 (request call id 0) took 1263 ms. Trace:
I20260709 13:03:06.755486  9386 rpcz_store.cc:276] 0709 13:03:05.379690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.386469 (+  6779us) service_pool.cc:224] Handling call
0709 13:03:06.623171 (+1236702us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b540025ce54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f3853 00007f32e74c160a 00007f32e0a477d4 00007f32e0a47762 00007f32e0a47691 00007f32e0a4760c 00007f32e0a468c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:06.623383 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288632576}
I20260709 13:03:06.705080  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2173) took 1007 ms. Trace:
I20260709 13:03:06.751175  8752 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:06.652696  8673 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:06.757390  8756 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:03:06.750375  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 307774us
I20260709 13:03:06.784440  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:06.750964  8677 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:03:06.615499  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 190698us
I20260709 13:03:06.765729  9355 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:06.785866  8677 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:06.786849  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:06.787009  8677 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:06.788141  8677 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:06.780879  8750 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:06.749962  8679 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:03:06.752535  8751 rpcz_store.cc:276] 0709 13:03:05.174796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.174956 (+   160us) service_pool.cc:224] Handling call
0709 13:03:06.435045 (+1260089us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b54002abf54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:06.435221 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64029952}
W20260709 13:03:06.766935  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.789783  8679 pending_rounds.cc:85] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 2
I20260709 13:03:06.790125  8679 pending_rounds.cc:107] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 3.3
W20260709 13:03:06.790256  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.790480  8679 raft_consensus.cc:2889] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:06.756860  8755 rpcz_store.cc:276] 0709 13:03:05.697871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.134065 (+436194us) service_pool.cc:224] Handling call
0709 13:03:06.134404 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:06.134795 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.134826 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:06.135382 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 13:03:06.195093 (+ 59711us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:06.196990 (+  1897us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.383605 (+186615us) spinlock_profiling.cc:243] Waited 4.23 ms on lock 0x7b18003aeac0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:03:06.436046 (+ 52441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.436138 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:06.670541 (+234403us) spinlock_profiling.cc:243] Waited 7.02 ms on lock 0x7b18003aeac0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:03:06.704728 (+ 34187us) raft_consensus.cc:1685] finished
0709 13:03:06.704762 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.704981 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62221,"prepare.queue_time_us":1579,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":235490,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":16757,"spinlock_wait_cycles":25891968,"thread_start_us":59346,"threads_started":1,"wal-append.queue_time_us":120892}
I20260709 13:03:06.755602 11647 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:06.757752 11794 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.601s	user 0.000s	sys 0.003s
I20260709 13:03:06.796938  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:06.799171 11646 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:06.801077 11666 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.102s	user 0.000s	sys 0.012s
I20260709 13:03:06.758097 11549 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:06.513782 11548 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.801383 11579 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:06.807317 11751 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:06.834489  8677 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:03:06.816767 11743 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.085s	user 0.020s	sys 0.054s
I20260709 13:03:06.790313 11711 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:06.834494 11666 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.143s	user 0.004s	sys 0.012s
W20260709 13:03:06.853986 11743 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.023s	sys 0.060s
I20260709 13:03:06.841641 11623 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:06.858712 11718 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.686s	user 0.016s	sys 0.000s
I20260709 13:03:06.872153  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:06.875730  8674 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:06.877882  9388 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 3.
I20260709 13:03:06.880321  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1506) took 1518 ms. Trace:
I20260709 13:03:06.855846 11666 raft_consensus.cc:697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:06.882581  8672 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:06.884231  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:06.885198  8822 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:06.840602 11548 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.880858  8826 rpcz_store.cc:276] 0709 13:03:05.361824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.364810 (+  2986us) service_pool.cc:224] Handling call
0709 13:03:05.365229 (+   419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:05.387047 (+ 21818us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:05.387083 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:05.387780 (+   697us) log.cc:844] Serialized 52 byte log entry
0709 13:03:06.416876 (+1029096us) spinlock_profiling.cc:243] Waited 6.66 ms on lock 0x7b18003c3640. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:03:06.573816 (+156940us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:06.573879 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.573991 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:06.868296 (+294305us) raft_consensus.cc:1685] finished
0709 13:03:06.868317 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.880241 (+ 11924us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33335,"raft.queue_time_us":36211,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":83953,"spinlock_wait_cycles":15313024,"thread_start_us":156622,"threads_started":1,"wal-append.queue_time_us":1317774}
W20260709 13:03:06.887581  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 91350.7us
I20260709 13:03:06.888850  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:06.889604  8826 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:06.863941 11786 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:06.463141 11590 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.893466  8822 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:06.881153 11812 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.904634 11548 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:06.905224  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:06.893154 11743 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:03:06.911090 11666 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:06.913388 11746 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:03:06.913627  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 660519us
I20260709 13:03:07.038388  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:07.040612  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:06.914367 11771 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:06.915202  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.915251  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:06.913388 11549 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:06.918231  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:07.042533 11690 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:03:07.043140  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 125046us
I20260709 13:03:07.043663 11690 consensus_queue.cc:1243] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:03:07.045683 11690 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:07.046689 11548 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:06.919096  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 122238us
W20260709 13:03:06.923658 11793 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.768s	user 0.003s	sys 0.000s
W20260709 13:03:06.924247 11806 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:03:06.925616  8826 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:06.926260 11777 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.059046 11777 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:06.926955 11748 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.927379 11276 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:06.927667  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:07.062801  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 135237us
I20260709 13:03:07.054713 11549 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.063685  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:07.063849 11549 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:06.927997  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.066288 11549 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:07.068084  8647 leader_election.cc:400] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8.
I20260709 13:03:07.068468  8647 leader_election.cc:403] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:07.069552  8647 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:06.880671 11623 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.072014 11623 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:07.072707 11623 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:07.073236 11623 raft_consensus.cc:697] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:06.928411 11710 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:07.075209 11710 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.077612 11710 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:06.929101 11557 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:07.077824 11646 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:07.078529 11557 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.079596 11777 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:07.081033 11557 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:06.933527 11811 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:03:06.900315 11786 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.046476 11690 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:07.084841 11690 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:07.085377 11690 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:07.085742 11786 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:06.940649 11814 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:06.941340  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1507) took 1579 ms. Trace:
I20260709 13:03:07.074246 11623 consensus_queue.cc:237] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:06.944890  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:06.947685  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1640) took 1021 ms. Trace:
I20260709 13:03:06.954911 11743 consensus_queue.cc:1059] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:06.956779  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1627) took 1041 ms. Trace:
I20260709 13:03:07.103994  8751 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:07.104544  8676 rpcz_store.cc:276] 0709 13:03:05.915735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.133583 (+217848us) service_pool.cc:224] Handling call
0709 13:03:06.133887 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.159276 (+ 25389us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:06.159643 (+   367us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.225242 (+ 65599us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:06.225301 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:06.225359 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.913098 (+687739us) spinlock_profiling.cc:243] Waited 6.24 ms on lock 0x7b18003d35a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32e63cd58f 00007f32df40c571 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd
0709 13:03:06.937658 (+ 24560us) spinlock_profiling.cc:243] Waited 9.09 ms on lock 0x7b18003d35a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:03:06.956479 (+ 18821us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b540033a4d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:06.956538 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.956701 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":946,"prepare.queue_time_us":549,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":89,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":7216,"spinlock_wait_cycles":92948864,"thread_start_us":65057,"threads_started":1,"wal-append.queue_time_us":83213}
I20260709 13:03:06.959949 11741 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:06.968683 11751 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:06.975831  9388 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:06.976751  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:06.978271  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1631) took 1055 ms. Trace:
I20260709 13:03:07.110791  8675 rpcz_store.cc:276] 0709 13:03:05.923176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.168624 (+245448us) service_pool.cc:224] Handling call
0709 13:03:06.169142 (+   518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.779912 (+610770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.977964 (+198052us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b54002aa654. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:06.978199 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654348032}
I20260709 13:03:06.981702 11687 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:06.983115  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.115119  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.983668 11636 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:06.983856  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 186988us
I20260709 13:03:07.120252  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:06.984405 11696 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:06.984797 11647 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:06.984488  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 200193us
I20260709 13:03:06.987085  9386 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:06.994731 11408 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:07.000118 11753 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:07.006891  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1617) took 1094 ms. Trace:
I20260709 13:03:07.008917  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 457) took 1222 ms. Trace:
I20260709 13:03:07.140290  8672 rpcz_store.cc:276] 0709 13:03:05.787623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.788021 (+   398us) service_pool.cc:224] Handling call
0709 13:03:07.008709 (+1220688us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400430dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:07.008847 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2370907008}
I20260709 13:03:07.010960  8681 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:07.017289  8576 catalog_manager.cc:5697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 7 to 8, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 8 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:07.050565  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.139765  8678 rpcz_store.cc:276] 0709 13:03:05.912924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.913049 (+   125us) service_pool.cc:224] Handling call
0709 13:03:05.913339 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.967826 (+ 54487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.006572 (+1038746us) spinlock_profiling.cc:243] Waited 53.7 ms on lock 0x7b54003cfd54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:07.006819 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123599232}
I20260709 13:03:07.052541  8645 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:07.146297  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 93888.3us
I20260709 13:03:07.147279  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:07.148275  8722 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:07.149397  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.062458 11748 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:06.935796 11711 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:07.086885  8823 rpcz_store.cc:276] 0709 13:03:05.362089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.112769 (+750680us) service_pool.cc:224] Handling call
0709 13:03:06.113168 (+   399us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:06.119241 (+  6073us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.119264 (+    23us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:06.119769 (+   505us) log.cc:844] Serialized 52 byte log entry
0709 13:03:06.759342 (+639573us) spinlock_profiling.cc:243] Waited 42.8 ms on lock 0x7b1800395c80. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:03:06.787108 (+ 27766us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:06.787164 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.787259 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:06.916553 (+129294us) raft_consensus.cc:1685] finished
0709 13:03:06.916577 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.941256 (+ 24679us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11218,"raft.queue_time_us":19510,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":7236,"spinlock_wait_cycles":98448640,"thread_start_us":27325,"threads_started":1,"wal-append.queue_time_us":711944}
I20260709 13:03:06.933980 11812 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.088165 11652 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:03:07.090343 11718 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:07.092556  8753 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:07.092978  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:07.109622 11741 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:07.110598  9381 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:07.112733  9353 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:07.151671  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:07.114544 11786 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:07.157877  8678 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:07.158109  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:07.158861  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.150130  8677 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:03:07.162789  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1647) took 1235 ms. Trace:
I20260709 13:03:07.155619  8576 catalog_manager.cc:5697] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.161880 11226 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:07.151682 11579 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:07.103802  8679 rpcz_store.cc:276] 0709 13:03:05.925903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.381222 (+455319us) service_pool.cc:224] Handling call
0709 13:03:06.749447 (+368225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.791284 (+ 41837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.947311 (+156027us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54003a2fd0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:06.947574 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606245632}
I20260709 13:03:07.154729 11504 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:03:07.216280  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.197837 11541 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:07.217257  8679 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:07.167117  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:03:07.217623  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 109585us
I20260709 13:03:07.162556 11382 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:07.164048  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.164896  8755 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:03:07.139997 11777 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.165284  8829 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:03:07.154922  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:07.173926  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:07.163522  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:07.167043  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.234490  9388 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:07.167791  9385 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:07.221480  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.171011  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:07.221812  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:07.237282  8679 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:07.237737  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.220204  8576 catalog_manager.cc:5697] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 4, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 4 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.171598  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:07.245944  9389 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:07.247596  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.248492  8672 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:07.219774  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.165870  8823 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:07.134718 11696 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.167114  8827 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:07.178191  9386 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:07.163166 11746 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:07.172557  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 79850.5us
I20260709 13:03:07.162896 11557 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:07.235862  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:07.220715  8681 rpcz_store.cc:276] 0709 13:03:05.927805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.626012 (+698207us) service_pool.cc:224] Handling call
0709 13:03:07.010392 (+384380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.141855 (+131463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.162473 (+ 20618us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b5400430dd0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:07.162704 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882853248}
I20260709 13:03:07.240916  8576 catalog_manager.cc:5697] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.262667  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:07.255103 11816 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:07.245289 11523 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:03:07.150993 11710 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:07.177304 11718 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.271807  8576 catalog_manager.cc:5697] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.272833  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:07.273363  8829 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:07.266702  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:07.163384 11792 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.268082 11557 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.172101 11698 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:07.162809 11771 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:07.256268 11777 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.275722  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:07.285509  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 440565us
I20260709 13:03:07.284529  8576 catalog_manager.cc:5697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.217110 11541 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.242585 11382 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.141271 11276 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.127s
I20260709 13:03:07.320641  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.221701 11711 raft_consensus.cc:3055] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:07.135030 11636 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.166988  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:07.288391  8753 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:07.164505 11563 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.124s
I20260709 13:03:07.265525 11816 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.267786 11696 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:07.291524  8576 catalog_manager.cc:5697] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 3 to 4, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 4 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.299743  8751 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:07.305835  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:07.168730 11812 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:07.313879  8794 leader_election.cc:336] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.699s (SENT)
I20260709 13:03:07.279819  8647 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:07.344427  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:07.345652  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 66052us
I20260709 13:03:07.346537  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:07.348186  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:07.211997 11741 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:07.350018  8579 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43482 (request call id 227) took 27209 ms (client timeout 15000 ms). Trace:
I20260709 13:03:07.350232 11741 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:07.327828 11711 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:07.351599 11711 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:07.352942 11711 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:07.277024 11710 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.282078 11557 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:07.320192 11382 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:07.356082 11710 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:07.356899  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:07.358124 11607 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.358682 11607 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:07.291465 11792 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.360864 11607 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:07.361340 11792 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:07.361720 11707 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:07.362074 11748 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:07.343883 11816 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:07.363062 11504 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:07.278240 11718 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:07.219165 11807 consensus_queue.cc:1059] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:03:07.260643 11790 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.289556 11749 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:07.323408  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:07.389434  8829 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:07.366914 11814 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:07.362491 11706 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:07.368793  8677 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:03:07.312031  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:07.318007  9381 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:07.375823  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:07.452536  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 76796.7us
I20260709 13:03:07.453399  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:07.280834 11765 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:07.282938 11698 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.366768  9389 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:07.379804  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:07.468472  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 89031.7us
I20260709 13:03:07.243584 11786 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.381975  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:07.289646 11541 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:07.283097 11556 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:07.487736  9381 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 13:03:07.325922  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.385447  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 335147us
I20260709 13:03:07.165617 11753 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.403266  8679 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:07.423308  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:07.429971  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.431376 11777 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:07.434252  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.436129 11749 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:07.508215 11749 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:07.378935 11812 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.510952 11812 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.386003 11748 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:07.513857  8827 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:07.269682 11746 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.448596  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.516208  8681 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:07.451179 11636 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:07.454145  8796 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:07.517787  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 63787.9us
I20260709 13:03:07.455536  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.483366  8674 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:07.491875  9353 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:07.320326 11771 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.502096  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:07.521039  8678 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:07.522095 11504 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:07.523890 11504 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:07.524354 11751 raft_consensus.cc:3055] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:07.524968 11751 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:07.530048 11786 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:07.530793 11786 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:07.532403  8672 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:03:07.533207  9355 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:07.536024  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:07.485937 11276 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:07.546679  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:07.524456 11504 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:07.538321  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 401685us
W20260709 13:03:07.538591  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 46874us
I20260709 13:03:07.506367 11607 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:07.504534  9384 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
W20260709 13:03:07.545094  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.468678 11706 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:07.467595 11698 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:07.482149 11792 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:07.486503 11556 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.580065  8822 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:07.424265 11790 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.591336 11549 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:07.592406 11746 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:07.593008 11549 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:07.494154 11557 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.088s
I20260709 13:03:07.488945 11710 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 13:03:07.526064 11751 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:07.597049 11771 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:07.501473 11590 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
W20260709 13:03:07.605144 11690 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.519s	user 0.005s	sys 0.006s
I20260709 13:03:07.605485 11792 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:07.605783 11690 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:07.627874 11652 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8.
I20260709 13:03:07.608482 11382 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.575500 11687 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:03:07.587194 11563 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:03:07.612685 11753 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:07.570149 11523 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:07.459741 11814 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.664614 11226 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:07.665779 11814 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:07.580641 11636 consensus_queue.cc:1059] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:03:07.587281 11812 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:07.668637  9352 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.021s	sys 0.026s
I20260709 13:03:07.623365 11504 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:07.586798 11556 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:07.624650 11751 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:07.628984 11690 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:07.606668 11792 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.632449 11790 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:07.636291 11541 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:07.621690 11816 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:07.653424 11687 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:07.660071 11749 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:07.660071 11696 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:07.662282 11382 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:07.657895 11718 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:07.718397  9384 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: []
W20260709 13:03:07.726025 11821 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:03:07.684156 11698 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:07.717124 11816 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:07.684538 11741 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:07.722785 11523 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:07.711153 11687 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:07.720296 11541 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:07.687489  9382 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:03:07.746742 11815 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.000s	sys 0.024s
W20260709 13:03:07.749526 11623 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.013s	sys 0.013s
I20260709 13:03:07.742523 11226 raft_consensus.cc:3055] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:03:07.756666 11666 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.003s	sys 0.023s
W20260709 13:03:07.775656 11666 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.003s	sys 0.023s
I20260709 13:03:07.775521 11718 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:07.778360 11696 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:07.770319 11687 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:07.780481 11698 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:07.783414 11786 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:07.790459 11687 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:07.788761 11741 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:07.795239 11741 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:07.800446 11749 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:07.340444  8576 catalog_manager.cc:5697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 5 to 6, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 6 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:07.350538  8579 rpcz_store.cc:269] 0709 13:02:40.196322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.203114 (+  6792us) service_pool.cc:224] Handling call
0709 13:03:07.349894 (+27146780us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:02.897219 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:02.898329 (+  1110us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:02.898413 (+    84us) write_op.cc:435] Acquired schema lock
0709 13:03:02.898444 (+    31us) tablet.cc:662] Decoding operations
0709 13:03:02.957912 (+ 59468us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:02.958038 (+   126us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:02.958071 (+    33us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:03:02.975135 (+ 17064us) tablet.cc:701] Row locks acquired
0709 13:03:02.975209 (+    74us) write_op.cc:260] PREPARE: finished
0709 13:03:02.975450 (+   241us) write_op.cc:270] Start()
0709 13:03:02.975659 (+   209us) write_op.cc:276] Timestamp: P: 1783602182975357 usec, L: 0
0709 13:03:02.975693 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:03:02.976720 (+  1027us) log.cc:844] Serialized 6285 byte log entry
0709 13:03:03.351236 (+374516us) op_driver.cc:464] REPLICATION: finished
0709 13:03:03.497094 (+145858us) write_op.cc:301] APPLY: starting
0709 13:03:03.503251 (+  6157us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:03.521944 (+ 18693us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:03.554855 (+ 32911us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:03.800696 (+245841us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:03.801185 (+   489us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:03:03.801236 (+    51us) write_op.cc:312] APPLY: finished
0709 13:03:03.801774 (+   538us) log.cc:844] Serialized 113 byte log entry
0709 13:03:03.802017 (+   243us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:03.802212 (+   195us) write_op.cc:454] Released schema lock
0709 13:03:03.802294 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:03.857230 (+ 54936us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:03.866501 (+  9271us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":168931,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":126519,"apply.run_cpu_time_us":8363,"apply.run_wall_time_us":354496,"num_ops":16,"prepare.queue_time_us":220992,"prepare.run_cpu_time_us":13881,"prepare.run_wall_time_us":178840,"raft.queue_time_us":195415,"raft.run_cpu_time_us":25976,"raft.run_wall_time_us":96389,"replication_time_us":375343,"thread_start_us":382134,"threads_started":4,"wal-append.queue_time_us":113079,"wal-append.run_cpu_time_us":10778,"wal-append.run_wall_time_us":1709826}]]}
I20260709 13:03:07.845007  8576 catalog_manager.cc:5697] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 4 to 7, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 7 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:03:07.852980  8576 catalog_manager.cc:5697] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 6, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 6 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.856006  8576 catalog_manager.cc:5697] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 3 to 5, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 5 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.471963  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:07.861799  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 389711us
I20260709 13:03:07.866593  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:07.875906  8576 catalog_manager.cc:5697] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 1 to 4, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 4 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.884166  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.882822  8576 catalog_manager.cc:5697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.885632  8672 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:07.887137  8673 raft_consensus.cc:2393] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:07.887995  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.889037  8672 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:07.889219  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.891068  8672 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:07.889395  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.892038  8673 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:07.893668  8674 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:07.893810  8673 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:07.896487  8673 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:07.898346  8673 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:03:07.899387  8674 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:03:07.902408  8675 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:07.910202  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:07.642974  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 54122.6us
W20260709 13:03:07.942384  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.943548  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.944543  8722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.027s	sys 0.029s
W20260709 13:03:07.945153  8722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.027s	sys 0.029s
W20260709 13:03:07.945588  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.955883  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 304722us
W20260709 13:03:07.963121  8824 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.012s	sys 0.008s
W20260709 13:03:07.963908  8824 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.013s	sys 0.009s
I20260709 13:03:07.967274  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:07.972613  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:07.974509 11846 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:07.675557  8830 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.020s	sys 0.051s
I20260709 13:03:07.978116  8681 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:07.979004  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:07.980042  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 506) took 1096 ms. Trace:
I20260709 13:03:07.980810  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1693) took 2036 ms. Trace:
I20260709 13:03:07.981572  8828 rpcz_store.cc:276] 0709 13:03:05.944005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.082391 (+138386us) service_pool.cc:224] Handling call
0709 13:03:07.980065 (+1897674us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b54002689d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:07.980316 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4275148672}
I20260709 13:03:07.980849  8830 rpcz_store.cc:276] 0709 13:03:06.883445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.883640 (+   195us) service_pool.cc:224] Handling call
0709 13:03:06.884145 (+   505us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:07.094079 (+209934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:07.094135 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:07.109883 (+ 15748us) log.cc:844] Serialized 52 byte log entry
0709 13:03:07.979204 (+869321us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:07.979337 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.979433 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.979502 (+    69us) raft_consensus.cc:1685] finished
0709 13:03:07.979515 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.979775 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18951,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":1012040,"threads_started":2,"wal-append.queue_time_us":630770}
W20260709 13:03:07.669617  9352 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.021s	sys 0.026s
W20260709 13:03:07.997278  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 945290us
I20260709 13:03:07.998358 11226 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:08.001137 11226 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.004559 11226 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:08.007023  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:08.007615  9384 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:08.001173  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:08.008240  8750 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:07.977180 11846 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
W20260709 13:03:07.769335 11623 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.013s	sys 0.013s
I20260709 13:03:08.034040  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:08.034837  8794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.005s	sys 0.015s
W20260709 13:03:08.035422  8794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.005s	sys 0.015s
I20260709 13:03:08.035331  8681 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:08.038409 11718 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:08.040072 11718 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.041486 11563 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:08.043201 11765 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.044255 11765 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.045360  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:08.052836 11822 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.054333 11822 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.055253 11666 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:07.759822 11556 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.056538 11666 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.040020 11696 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:03:08.042064 11698 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:03:08.098054  8830 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:08.126959  8755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.003s	sys 0.011s
I20260709 13:03:08.062458 11749 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:08.046311 11707 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:08.123063  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:08.109093  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:08.111143  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:08.073573 11666 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:08.285655  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 146934us
I20260709 13:03:08.286532 11826 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.068971  9387 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:03:08.253511  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 145675us
I20260709 13:03:08.301156  8681 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:08.060185  8796 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:08.107925  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:08.259699 11687 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.042097 11718 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:08.311611  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 281472us
I20260709 13:03:08.041605 11831 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:08.075086 11765 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:08.079684 11541 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:08.237720 11690 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.313395  9381 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:08.236506 11660 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:08.335361 11660 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.331830  9389 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:08.337476  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 835516us
I20260709 13:03:08.103550 11652 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.138924 11563 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:08.328719  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.330027  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:08.338714 11652 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.339469  8794 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:08.339951  8828 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:08.339499 11846 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.106166 11822 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:08.343056 11741 consensus_meta.cc:287] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.009s
I20260709 13:03:08.338716 11563 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:08.343746 11838 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.308s	user 0.001s	sys 0.001s
I20260709 13:03:08.344305 11504 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:08.344795 11504 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:08.345290 11504 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:08.345752 11563 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:08.345635 11416 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:08.347781  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 347003us
I20260709 13:03:08.349316  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:08.325897 11666 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:08.350157 11831 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.351001 11666 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.351446 11666 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:08.352154  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.352958 11831 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:08.353125 11822 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:08.302850 11707 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.238250 11382 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.118s
I20260709 13:03:08.356958  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:08.357137 11382 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:08.049571 11276 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.310s
I20260709 13:03:08.360093 11276 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:08.360625 11276 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:08.361769 11276 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.364774 11647 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:08.365396 11647 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:08.365646 11541 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:08.366346 11541 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:08.331240  8749 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 8: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:08.075019 11636 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.292s
I20260709 13:03:08.343703 11741 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:08.331337  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 357750us
I20260709 13:03:08.327593 11765 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:08.073482 11792 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.243s
I20260709 13:03:08.333868  8722 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:08.342012 11846 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.083494 11523 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.330585 11823 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:08.067044 11751 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.265s
I20260709 13:03:08.071758 11549 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.291s
I20260709 13:03:08.384531  8793 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:08.397552  9355 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:08.386418  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 58543.8us
W20260709 13:03:08.391920  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 58271.1us
W20260709 13:03:08.395306  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 52853.8us
I20260709 13:03:08.391243  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:08.409238  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 151318us
W20260709 13:03:08.411769  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 82484.9us
I20260709 13:03:08.390530  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:08.415946  8751 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:08.416105  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:08.419013  8794 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:08.366580 11647 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.420011 11792 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:08.421445 11647 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:08.420203 11696 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:08.422894  9384 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 8: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:08.427412  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:08.393265 11276 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:08.428061  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 43675.7us
I20260709 13:03:08.428356 11342 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:08.429179  8793 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:08.430054 11711 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:08.393076 11846 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:08.430581 11711 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.431126 11711 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:08.383080 11741 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:08.074725 11579 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.381s
I20260709 13:03:08.398392 11823 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:08.439145 11848 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:08.441720 11652 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:08.442049 11749 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:08.442884  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:08.393338 11416 consensus_queue.cc:1175] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" responder_term: 9 status { last_received { term: 7 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:08.386782 11623 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:03:08.445582  8747 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:08.414245 11226 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:08.384866 11765 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.447868  9385 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:08.379027 11636 consensus_queue.cc:1243] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:03:08.448158 11556 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:08.433686 11687 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.407989 11690 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.414482 11822 raft_consensus.cc:2764] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:03:08.457099  9381 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
W20260709 13:03:08.457716  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.317058 11826 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.463981  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:08.464969  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:08.465348  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 108589us
I20260709 13:03:08.465981  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:08.466427 11660 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:08.467586  8829 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:08.467890 11504 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.468767  8827 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:08.433427 11696 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.469410  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:08.469904  9388 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:08.471478  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:08.472694 11718 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.473345 11382 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.474908 11707 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:08.490957  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 44146us
I20260709 13:03:08.470504  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:08.494515  8750 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:08.459278 11823 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:08.476132 11579 consensus_queue.cc:1243] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:03:08.502209  9389 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:08.446442 11666 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.531489  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:08.550989 11504 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:08.552500 11666 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:08.553223 11765 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:08.507519  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.554132  8751 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:03:08.555490 11687 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:08.557528  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:08.559269 11847 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:08.559597  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:08.559963 11847 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:08.560695 11687 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:08.562052  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:08.563180  8827 raft_consensus.cc:2410] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:08.513526 11541 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.506083 11607 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:08.519508  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:08.520243  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:08.521488 11826 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:08.522548  9352 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:08.573468  8681 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:08.530445  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:08.546808 11382 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.524554 11807 consensus_queue.cc:1059] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:03:08.524839 11696 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:08.527897 11749 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
W20260709 13:03:08.565588  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 136532us
I20260709 13:03:08.518314  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:08.565371  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:08.605579  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:08.557622 11690 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:08.615569  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:08.615849  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:08.616349  8674 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:08.595135 11751 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:08.616199  8828 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:08.617004  9386 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:08.620020  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 94470.5us
I20260709 13:03:08.625707  8827 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:03:08.617002  8675 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: [3.2-6.4]
I20260709 13:03:08.659495 11743 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:08.605173 11666 consensus_queue.cc:1059] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:03:08.631477 11690 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:08.676941  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:08.613546 11826 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
W20260709 13:03:08.691402  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 133971us
I20260709 13:03:08.700573  9353 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:08.561273 11847 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:08.662568  8647 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.645s (SENT) [suppressed 36 similar messages]
I20260709 13:03:08.699156  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:08.677371 11226 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.703981 11690 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:08.748093  8676 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:03:08.744375  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:08.729849 11647 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:08.758523  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 138945us
W20260709 13:03:08.759593  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 59125.1us
I20260709 13:03:08.760445  9353 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:08.754539  8679 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:08.713024 11851 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.237s
I20260709 13:03:08.767081  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:08.746239 11855 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.750015 11687 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:08.770205  8751 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:08.756244 11416 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:08.742105 11226 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.791481  8828 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:08.764235 11847 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:08.792608  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:08.797598 11829 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:03:08.770262 11718 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:08.803645  8750 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:08.812363 11646 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:08.812788  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:08.813227  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:08.814227  8822 raft_consensus.cc:2410] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:03:08.813901 11690 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.809788 11849 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.818769 11382 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:08.819334 11849 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.821759 11849 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:08.805567 11647 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.825805 11647 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:08.827287 11853 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
W20260709 13:03:08.829438 11858 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:03:08.807037 11687 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:08.801751 11416 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:08.833214 11416 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.801117 11871 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:03:08.835359 11861 log.cc:927] Time spent T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:03:08.836769 11860 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:03:08.837524 11751 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:08.838174 11751 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:08.839262 11831 raft_consensus.cc:3055] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:08.808952 11855 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.839603 11623 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:08.839740 11831 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:08.830276 11666 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:08.840289 11623 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.840742 11666 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.840916 11831 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.814107 11276 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.842764 11623 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:08.843029 11276 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.843815 11646 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:08.845750 11871 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:08.846626 11276 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:08.846951 11871 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:08.850338 11850 log.cc:927] Time spent T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Append to log took a long time: real 0.334s	user 0.003s	sys 0.000s
I20260709 13:03:08.851243 11646 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.831786 11687 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:08.809192 11851 consensus_queue.cc:1243] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:03:08.854867  8793 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:08.870708  8674 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:08.881407  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:08.842383 11855 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:08.842386 11831 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:08.893199  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 271072us
I20260709 13:03:08.902479  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:08.904515  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:08.907687 11607 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:08.914502  8751 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:08.863255 11342 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:08.915477 11342 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:08.916841 11342 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.918815  8755 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:08.920814  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:08.921829  8747 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
W20260709 13:03:08.928982  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 126732us
I20260709 13:03:08.933833  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:08.896903 11666 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:08.939548  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:08.942269  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:08.942819 11687 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:08.943472  8722 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:08.945115 11875 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:08.943574 11687 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:08.945667 11875 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.959595  8825 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:08.962492  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:08.962823  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:08.948791 11687 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:08.967335  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 444996us
I20260709 13:03:08.967046  8747 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 4.
I20260709 13:03:08.946116 11855 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:08.958117 11875 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:08.971660 11687 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:08.976145  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:08.998507  8747 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:08.978907 11855 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:08.979472 11711 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.003s
W20260709 13:03:09.002831  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 340298us
I20260709 13:03:08.980244 11823 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:08.987232 11408 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:09.010861  9386 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:09.023989  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:09.026151  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.015272  8647 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:09.007581 11711 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:09.030022  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.030381  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.030793  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.031497  8749 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:09.033644  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:09.036587  9386 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:09.038345  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:09.042742  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.042514  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.043406 11792 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:09.044009 11792 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:09.046792  8822 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:09.048977  8821 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
W20260709 13:03:09.061283  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 56886.6us
I20260709 13:03:09.064069  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:09.064997  9380 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:09.065840  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.045238 11792 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:09.067698  8750 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:03:09.071240 11607 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:09.073160 11823 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:09.074319  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:09.074935  9382 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:09.078840 11855 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:09.083021  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 143600us
I20260709 13:03:09.084105  8793 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:09.085311  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:09.086200  8752 raft_consensus.cc:2368] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9.
I20260709 13:03:09.087026  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:09.088264  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:09.089192  8827 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:09.089546 11749 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:09.075044  8756 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:09.093559  9386 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:03:09.101863 11877 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:03:09.118848  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:09.118818 11711 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:09.118813 11590 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:09.120429 11590 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:09.120880 11590 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:09.129341  8756 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:09.128693  9380 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:09.141798  8750 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:09.142966  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:09.137465  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:09.144203 11872 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:09.145030 11872 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:09.106871 11636 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:09.149966  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:03:09.150185 11636 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:09.152802 11636 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:09.153576  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:03:09.156705  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.092681 11749 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:09.158593  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:09.105274 11607 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:09.161608  9386 raft_consensus.cc:2410] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:03:09.164028  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:09.165359  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.166648  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.167903  8752 raft_consensus.cc:2410] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:03:09.169680  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:09.171893  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.173094  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.173647  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:09.173851  8749 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:09.168301 11875 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:09.175027  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:09.176128  8747 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:03:09.176891 11875 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:09.175513  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.173888  8754 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:09.185554  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:09.187093  8830 raft_consensus.cc:2410] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:03:09.188272 11828 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:09.188985 11828 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:09.189429  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:09.190310  8827 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:09.190387  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:09.191964  8821 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:09.195709  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:09.197316  8830 raft_consensus.cc:2410] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
W20260709 13:03:09.197456 11880 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:03:09.160292 11749 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:09.205796 11541 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:09.195783 11342 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:09.217686  8678 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:03:09.218240 11226 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:09.217651  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:09.218894 11882 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.101s	user 0.000s	sys 0.002s
I20260709 13:03:09.221824  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:09.222883  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:09.223559  8830 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:09.229056  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.230154 11751 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:09.232535 11872 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:09.234289  8676 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:03:09.190315 11828 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:09.237534 11828 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:09.243340  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.244839  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:09.245575  8822 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:09.246912  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.247684  8752 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:03:09.251580  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.252333  8752 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:09.253249  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.242395  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.249944  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:09.255111 11880 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.056s	user 0.000s	sys 0.002s
I20260709 13:03:09.255447 11751 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:09.255632  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.161953 11607 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:09.257478  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.257562 11607 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:09.258276 11879 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.259064 11879 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:09.262035 11879 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:09.264583  8821 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:09.266672  8750 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:03:09.267805  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.257045  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.277828  8754 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:09.287046  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:09.289294  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 128644us
I20260709 13:03:09.293227  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:09.276512  8830 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
W20260709 13:03:09.264828 11696 consensus_meta.cc:287] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.651s	user 0.012s	sys 0.003s
I20260709 13:03:09.301796  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.287789  8677 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:09.319672  9355 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:09.296370 11765 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.341145 11765 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:09.344199 11765 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:09.345239 11623 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:09.346321 11623 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:09.346715 11623 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:09.348412  8796 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:09.357649  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:09.360497  8719 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:09.367069  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:09.250599 11687 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:09.382992 11891 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:03:09.382750 11590 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:09.409147  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:09.384130  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 76385.1us
I20260709 13:03:09.385133 11541 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:09.386175  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:09.307361 11690 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.359133 11848 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:09.360193 11847 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:03:09.393414  8738 tablet_service.cc:1558] Processing DeleteTablet for tablet abd86f4e9b544531a7cbfa0042b4fa6c with delete_type TABLET_DATA_TOMBSTONED (TS 0e9b0850145a4c66bfe983ffdfa93a57 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39018
I20260709 13:03:09.393951 11504 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:03:09.395475  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:09.395586  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:09.396262 11822 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.365311 11666 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:09.398758 11873 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:09.366075 11886 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:03:09.402896  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 60336.7us
I20260709 13:03:09.403116 11226 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:09.319025 11696 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:09.402915  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:09.336997 11741 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:03:09.407719 11875 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:09.439167  8748 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:09.363703 11416 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:03:09.445360  8681 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:09.460220  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.453823  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:09.425137 11848 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.455657  9386 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:09.462256  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:09.434315 11831 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.465101 11855 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:03:09.466921  8680 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:09.473022  8673 raft_consensus.cc:1217] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[4.2-7.4]   Dedup: 4.2->[5.3-7.4]
I20260709 13:03:09.360890 11623 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:09.476598  8751 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10. Ops: [5.4-9.6]
I20260709 13:03:09.476627  8673 pending_rounds.cc:85] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 2
I20260709 13:03:09.442916 11873 raft_consensus.cc:697] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:03:09.503366  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 51392.5us
I20260709 13:03:09.454800 11875 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.505934  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:09.459671 11792 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:09.511793  8680 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:09.500324  8673 pending_rounds.cc:107] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 13:03:09.513027  8673 raft_consensus.cc:2889] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:09.466929 11636 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:03:09.465490 11848 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:09.465009 11872 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:09.517168  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:09.519011  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 132959us
I20260709 13:03:09.427577 11690 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:09.525849  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 123014us
I20260709 13:03:09.530192  8752 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:09.477525 11822 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:09.527774  8681 pending_rounds.cc:85] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 4
I20260709 13:03:09.694204  8681 pending_rounds.cc:107] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 6.5
W20260709 13:03:09.583611  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 48304.2us
W20260709 13:03:09.584986  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.585332 11879 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:09.509970 11541 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:09.608567  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 84624.1us
I20260709 13:03:09.556170  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.515836 11636 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:09.694867  8681 raft_consensus.cc:2889] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:09.665190  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:09.564129  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:09.668457  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 256714us
W20260709 13:03:09.709143  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.669734  8749 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:09.536365  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:09.566035  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:09.563581  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:09.596781  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 79722.4us
I20260709 13:03:09.533870 11698 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:09.720844  8645 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:09.529630 11590 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:09.514590 11696 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:09.514845 11831 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:09.518826 11875 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:09.522629 11872 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:09.722240  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:09.617081 11749 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:09.644323 11848 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:09.735688 11848 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:09.735859 11749 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:09.726218 11718 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:03:09.566428 11504 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.171s
I20260709 13:03:09.538968 11607 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:09.515832 11687 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:09.637379 11855 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.716150  8754 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 8: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:09.596968 11792 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:09.731479 11698 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:09.527984 11226 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:09.719483  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:09.738219  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:09.749014  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:09.742494  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 311948us
I20260709 13:03:09.742859  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:09.755623  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.528343 11712 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.218s
I20260709 13:03:09.744773  9352 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:09.608737 11765 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 13:03:09.770810  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:09.544898 11690 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:09.706352 11636 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:09.753275  8676 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:09.705379 11879 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:09.706410 11541 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:09.745561 11666 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:09.800664  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.537972 11873 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:09.796268  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:09.804708  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:09.746125 11792 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:09.799291  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47664.4us
I20260709 13:03:09.795439  8750 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:09.813151  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:09.810966  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:09.823204  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 118800us
I20260709 13:03:09.824337  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:09.825616  8750 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:09.772682 11563 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:09.828890  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:09.845305  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:09.829322  8828 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:03:09.847389  8828 pending_rounds.cc:85] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 7
I20260709 13:03:09.847685  8828 pending_rounds.cc:107] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:03:09.847922  8828 raft_consensus.cc:2889] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:09.848270  8645 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:09.848579  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 80274.3us
I20260709 13:03:09.795485 11846 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.850284  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:09.856941  8675 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:09.797638 11743 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:09.816614 11792 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:09.809604 11690 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:09.767982 11855 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:09.825294 11226 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.835161 11563 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.884303  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:09.885185  8675 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:09.886477  8752 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:09.867689 11382 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:09.862269 11743 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:10.008989  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:10.003612  8722 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:10.004937  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:09.871116 11590 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.875821 11718 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:09.892119 11563 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:10.023496  8675 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:03:10.042523  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 194355us
I20260709 13:03:10.044963  8750 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
W20260709 13:03:10.027809  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 179691us
I20260709 13:03:10.005915 11636 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:10.021705 11900 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 13:03:10.049520  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:09.859642 11872 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:09.864229 11846 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:10.060424  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:10.058779  8645 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:10.022765 11743 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:09.890127 11226 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:10.105253  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:10.101223  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 226767us
I20260709 13:03:10.058488 11636 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:10.046221 11382 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:03:10.112202  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 107441us
I20260709 13:03:10.114811 11607 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:10.117727 11915 tablet_replica.cc:333] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:03:10.126013  8677 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:10.127775  8677 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:03:10.131196  8750 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:03:10.116860  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.116806 11912 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:03:10.118327 11822 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:10.128365  8755 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:10.039096 11590 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:10.103726 11765 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:10.098903 11872 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:10.066740 11749 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:03:10.099961 11914 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:10.190907  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 115205us
I20260709 13:03:10.181922  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:10.110196 11382 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:10.061647 11849 consensus_queue.cc:1175] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7aff04f53d4845a6a4fa491fd2a06145" responder_term: 9 status { last_received { term: 5 index: 6 } last_committed_idx: 5 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:10.186826  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:10.188457  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 306203us
I20260709 13:03:10.059373 11918 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.131s
I20260709 13:03:10.031301 11698 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:10.170197 11831 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:10.216641  8722 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:10.185946 11765 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:10.069913 11666 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:10.199412 11382 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:10.226354  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 167720us
I20260709 13:03:10.198485 11607 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:10.228130  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 49668.1us
I20260709 13:03:10.223682  8719 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:10.190153 11541 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:10.253496  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.209369 11913 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:10.278656  8645 leader_election.cc:400] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9.
I20260709 13:03:10.279899  8645 leader_election.cc:403] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:03:10.223443 11831 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.233182 11879 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:10.281616 11911 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.277175 11913 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:10.291015  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 83340.3us
I20260709 13:03:10.267227  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:10.224223 11765 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:10.240597 11924 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:10.271353 11925 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:10.272040 11579 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:10.273576 11541 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:10.307703  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 79870.3us
I20260709 13:03:10.309969  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:10.233693 11914 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:03:10.308905  8645 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:10.314109  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1678) took 1023 ms. Trace:
I20260709 13:03:10.292110 11913 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:10.315660  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 48891.9us
I20260709 13:03:10.317487 11914 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:10.319481  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:10.322825 11879 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:10.323413 11579 consensus_queue.cc:1059] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:10.327222 11590 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:10.328876 11875 consensus_meta.cc:287] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.577s	user 0.011s	sys 0.000s
I20260709 13:03:10.315354  8678 rpcz_store.cc:276] 0709 13:03:09.290757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.324200 (+ 33443us) service_pool.cc:224] Handling call
0709 13:03:09.343138 (+ 18938us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:09.369275 (+ 26137us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.369325 (+    50us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:09.404496 (+ 35171us) log.cc:844] Serialized 78 byte log entry
0709 13:03:09.694222 (+289726us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:09.694264 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.694364 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.313774 (+619410us) raft_consensus.cc:1685] finished
0709 13:03:10.313804 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.314028 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25441,"raft.queue_time_us":31998,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":274,"thread_start_us":289215,"threads_started":1,"wal-append.queue_time_us":407973}
I20260709 13:03:10.333527  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:10.336992 11914 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:10.337975 11342 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:10.337595 11741 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:10.341251 11871 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.290174 11855 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:10.349195  9353 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:10.400873  8748 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
W20260709 13:03:10.358539 11909 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.318s	user 0.001s	sys 0.001s
I20260709 13:03:10.303728 11924 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:10.357920  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 135665us
I20260709 13:03:10.359601  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:10.418349  8794 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:10.393724  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 84888.6us
W20260709 13:03:10.401616  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 52568.3us
I20260709 13:03:10.290120 11933 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:10.292479 11831 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:10.403386 11579 raft_consensus.cc:3055] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:10.307477 11690 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:03:10.191208  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:10.366971  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:10.456091  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:10.292157 11911 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:10.398155 11914 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:10.395915 11541 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:10.464133  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:10.416239 11823 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:10.471011  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:10.473574  8756 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:10.397883 11342 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:10.474169  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54290.6us
W20260709 13:03:10.475293  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 108511us
I20260709 13:03:10.475445  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:10.425518  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 233516us
I20260709 13:03:10.423892 11924 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:10.426736 11579 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:10.424342 11933 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:03:10.477916  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 254294us
I20260709 13:03:10.400357 11875 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:10.463281 11922 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:03:10.452458 11590 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:10.493959  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:10.427222  8672 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:10.497522  9353 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:10.499754  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:10.426714 11847 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:03:10.408205 11846 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:10.421942 11871 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:10.475026 11541 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:10.490269 11504 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:10.466636 11712 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:10.500813 11792 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.620s	user 0.009s	sys 0.003s
I20260709 13:03:10.536533  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:10.496017 11696 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.539245  8647 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:10.597348  8751 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:10.539309  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1689) took 1066 ms. Trace:
I20260709 13:03:10.490653 11933 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:10.550676  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2195) took 1296 ms. Trace:
I20260709 13:03:10.566517  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1692) took 1045 ms. Trace:
I20260709 13:03:10.567404 11935 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:10.567974  8575 catalog_manager.cc:5697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 4 to 6, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 6 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:10.568909 11541 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:10.478389 11920 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:03:10.503723 11823 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.527859 11931 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:10.675752 11931 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:10.502547 11927 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:10.679553 11927 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:10.680222 11927 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:10.680320 11928 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:03:10.680756 11927 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:10.686604 11871 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:10.598295  8672 rpcz_store.cc:276] 0709 13:03:09.461316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.461470 (+   154us) service_pool.cc:224] Handling call
0709 13:03:10.359384 (+897914us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:03:10.528046 (+168662us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2380160}
I20260709 13:03:10.602190  8681 rpcz_store.cc:276] 0709 13:03:09.520998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.521211 (+   213us) service_pool.cc:224] Handling call
0709 13:03:09.521690 (+   479us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:09.750101 (+228411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.750150 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:09.752782 (+  2632us) log.cc:844] Serialized 52 byte log entry
0709 13:03:09.873724 (+120942us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:09.873776 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.873867 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.566206 (+692339us) raft_consensus.cc:1685] finished
0709 13:03:10.566233 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.566437 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":17906,"thread_start_us":40838,"threads_started":1,"wal-append.queue_time_us":383679}
I20260709 13:03:10.601739  8747 rpcz_store.cc:276] 0709 13:03:09.245498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.263618 (+ 18120us) service_pool.cc:224] Handling call
0709 13:03:09.263994 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:09.280813 (+ 16819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.280850 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:09.329392 (+ 48542us) log.cc:844] Serialized 26 byte log entry
0709 13:03:09.640606 (+311214us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:09.640659 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.640734 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.541210 (+900476us) raft_consensus.cc:1685] finished
0709 13:03:10.541244 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.541596 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40186,"raft.queue_time_us":18998,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":30771,"thread_start_us":216195,"threads_started":1,"wal-append.queue_time_us":477117}
W20260709 13:03:10.690790  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.539008 11918 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:10.711902  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:10.674993 11563 consensus_meta.cc:287] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.598s	user 0.001s	sys 0.012s
W20260709 13:03:10.667519  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 128370us
I20260709 13:03:10.600824 11712 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:10.599134 11342 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:10.601981 11504 raft_consensus.cc:697] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:10.669721 11920 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:10.528841 11855 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:03:10.653290 11873 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:10.693748  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.651791  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 125239us
I20260709 13:03:10.548550 11822 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:10.684270 11930 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.371s	user 0.001s	sys 0.000s
I20260709 13:03:10.677389 11931 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:10.723752  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 243919us
I20260709 13:03:10.749691  8645 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:10.725988  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.606113 11751 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:10.556474 11590 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:10.606529 11690 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:10.719774 11918 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.547995 11711 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.140s
I20260709 13:03:10.741053  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:10.762113 11636 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.635s	user 0.010s	sys 0.003s
I20260709 13:03:10.734764  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:10.738185  8719 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:10.788372  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50261.5us
I20260709 13:03:10.789491  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:10.551870 11932 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9.
I20260709 13:03:10.792260 11932 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:10.793501 11932 raft_consensus.cc:3055] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:10.793965 11932 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:10.794883 11925 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:10.795506 11925 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.796082 11925 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:10.798910 11946 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:10.799654 11946 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:10.800144 11946 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:10.765070 11751 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.802980 11751 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:10.728302  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:10.807732  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 79563.2us
I20260709 13:03:10.808804  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:10.727873 11607 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:10.606886 11792 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:10.647989 11696 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:10.774904 11918 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:10.740132 11822 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.684409 11911 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:10.722872 11823 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:10.718356  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.831431 11923 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:10.832432 11936 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.833532 11416 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.822834 11934 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:10.736330 11563 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:10.654294 11935 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:10.992646  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:10.839264  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1702) took 1115 ms. Trace:
I20260709 13:03:10.993607  8825 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:10.994856 11953 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.993848  8673 rpcz_store.cc:276] 0709 13:03:09.724293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.125211 (+400918us) service_pool.cc:224] Handling call
0709 13:03:10.125623 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.125857 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:10.473584 (+347727us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.838807 (+365223us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.838862 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:10.838931 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.839015 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.839180 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55860,"prepare.queue_time_us":417197,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":75,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":80551,"spinlock_wait_cycles":2432,"thread_start_us":701261,"threads_started":2}
I20260709 13:03:10.995471 11953 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:10.654066 11579 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:10.998195 11953 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:10.814697 11690 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:11.001998 11690 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.759986 11873 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:11.005591 11873 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:10.820513 11930 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:03:10.835858 11743 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.693s	user 0.008s	sys 0.012s
W20260709 13:03:10.928877  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 57499.4us
W20260709 13:03:10.906603  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 60233.2us
I20260709 13:03:10.904511  8675 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:10.926525  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:11.008304 11690 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:11.009035  8822 raft_consensus.cc:2410] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:03:10.933573  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:11.010332  8830 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:03:10.760406 11920 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:11.011466 11935 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:10.998221 11226 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:11.016865  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:11.016170  8575 catalog_manager.cc:5697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 3 to 9, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 9 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.017686  8674 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:10.842744 11822 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:11.019590  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:11.020617  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:11.021409  8673 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:11.014248 11920 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:10.915657 11831 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:10.999469  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:11.000067 11945 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:03:10.781589 11879 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:03:11.046401 11944 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 13:03:11.047132 11941 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:03:11.047343 11879 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:11.049361 11563 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:11.000833 11666 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.007122 11743 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:11.050545 11666 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:11.052776 11666 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:11.056910  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 560) took 1662 ms. Trace:
I20260709 13:03:11.067047 11711 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:11.067919 11873 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:11.071118 11873 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:11.072847 11342 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:11.073796 11873 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:11.075961 11579 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:11.078893  9384 rpcz_store.cc:276] 0709 13:03:09.394716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.395088 (+   372us) service_pool.cc:224] Handling call
0709 13:03:11.056556 (+1661468us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540025ce54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:11.056775 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2470141696}
I20260709 13:03:11.068171 11741 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:11.068480  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 48997.3us
I20260709 13:03:10.948487 11875 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:11.069070 11920 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:11.034328  8575 catalog_manager.cc:5697] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 7 to 8, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 8 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.091856  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:11.062359 11919 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.903s	user 0.005s	sys 0.000s
I20260709 13:03:11.114005  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:11.105962  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:11.148146  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 593132us
W20260709 13:03:11.149689  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 341009us
I20260709 13:03:11.208395  9384 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
W20260709 13:03:11.222151  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.987478 11416 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:11.100775 11920 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:11.366576  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.414983  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.414136  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.015s	sys 0.033s
W20260709 13:03:11.416262  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.015s	sys 0.034s
W20260709 13:03:11.370666  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 127299us
W20260709 13:03:11.418215  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 71965.2us
I20260709 13:03:10.938918 11504 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:10.954777 11936 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:10.929026 11932 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:10.948132 11590 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:11.086907 11690 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:11.605574  9355 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.013s	sys 0.052s
W20260709 13:03:11.606269  9355 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.013s	sys 0.052s
I20260709 13:03:11.210639 11923 raft_consensus.cc:697] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:11.099866 11741 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:11.465894  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 84822.1us
I20260709 13:03:11.374991 11911 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.423372 11920 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:11.657200  8828 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:03:11.415385 11855 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:11.668295  8719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.016s	sys 0.021s
W20260709 13:03:11.680576  8719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.016s	sys 0.021s
W20260709 13:03:11.690980  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.618992  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0324436299999888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
I20260709 13:03:11.752988 11923 consensus_queue.cc:237] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:11.758074  9353 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.018s	sys 0.021s
W20260709 13:03:11.758774  9353 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.018s	sys 0.021s
I20260709 13:03:11.679512 11951 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:11.718245 11698 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.039s	sys 0.041s
I20260709 13:03:11.678014 11935 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:11.678939 11917 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:11.719767 11848 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.019s	sys 0.055s
W20260709 13:03:11.789149 11952 log.cc:927] Time spent T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.803s	user 0.001s	sys 0.000s
I20260709 13:03:11.791325 11875 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:11.795823 11955 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.421s	user 0.000s	sys 0.000s
I20260709 13:03:11.797917  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:11.798588  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:11.687611 11712 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:11.805846  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1687) took 2353 ms. Trace:
I20260709 13:03:11.722647 11792 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:11.855368  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:11.800822 11698 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.039s	sys 0.041s
W20260709 13:03:11.807931 11541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.134s	user 0.026s	sys 0.036s
W20260709 13:03:11.808704 11848 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.020s	sys 0.056s
I20260709 13:03:11.838989  8679 rpcz_store.cc:276] 0709 13:03:09.432542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.432704 (+   162us) service_pool.cc:224] Handling call
0709 13:03:09.452036 (+ 19332us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:09.597417 (+145381us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:09.597471 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:09.598440 (+   969us) log.cc:844] Serialized 52 byte log entry
0709 13:03:09.746536 (+148096us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:09.746579 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.760378 (+1013799us) spinlock_profiling.cc:243] Waited 78.5 ms on lock 0x7b54003a20d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:10.760485 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:11.061230 (+300745us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:11.784954 (+723724us) raft_consensus.cc:1685] finished
0709 13:03:11.784995 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.785263 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66246,"raft.queue_time_us":68334,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":172,"spinlock_wait_cycles":223386752,"thread_start_us":147519,"threads_started":1,"wal-append.queue_time_us":461270}
I20260709 13:03:11.876087  9383 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:11.877028  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:11.884034 11416 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:11.850598 11823 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:11.886802 11958 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:11.849076 11935 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:11.848871 11917 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:11.707511 11911 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:11.719807 11855 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:11.777823 11953 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.005s	sys 0.035s
I20260709 13:03:11.913470  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:11.912983  8827 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.923s	user 0.032s	sys 0.069s
W20260709 13:03:11.857514 11954 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.434s	user 0.000s	sys 0.001s
I20260709 13:03:11.807248 11951 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:11.969151  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:12.613853  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 1.60633e+06us
W20260709 13:03:12.130154  8644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.016s	sys 0.012s
W20260709 13:03:12.615306  8827 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.625s	user 0.051s	sys 0.091s
I20260709 13:03:11.980995  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:12.616142  9355 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.006s	sys 0.006s
W20260709 13:03:12.616446  8681 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.003s	sys 0.012s
W20260709 13:03:12.617003  8681 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.004s	sys 0.012s
I20260709 13:03:12.618263  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:12.619088  8752 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:11.858656 11875 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:12.559376 11382 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 2.280s	user 0.009s	sys 0.016s
W20260709 13:03:12.559224  8795 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.903s	user 0.021s	sys 0.034s
I20260709 13:03:11.978574  9388 raft_consensus.cc:2368] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11.
W20260709 13:03:12.623018  8795 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.022s	sys 0.034s
W20260709 13:03:12.623687  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 1.84294e+06us
I20260709 13:03:11.980014  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:12.016436  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1699) took 2221 ms. Trace:
I20260709 13:03:11.980585  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:12.365830 11936 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:12.412096  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.004s	sys 0.008s
W20260709 13:03:12.627324  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.971s	user 0.008s	sys 0.008s
I20260709 13:03:12.615499  8719 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:12.627964  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:11.898504 11927 consensus_meta.cc:287] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.217s	user 0.007s	sys 0.006s
W20260709 13:03:12.559172  8647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.001s	sys 0.014s
W20260709 13:03:12.413229  9387 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.966s	user 0.002s	sys 0.010s
I20260709 13:03:12.622509  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2210) took 3089 ms. Trace:
I20260709 13:03:12.628857  8752 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
W20260709 13:03:11.973181 11872 consensus_meta.cc:287] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.781s	user 0.018s	sys 0.008s
I20260709 13:03:12.624933  8645 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:11.911197 11541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.237s	user 0.027s	sys 0.037s
I20260709 13:03:12.626332  8675 rpcz_store.cc:276] 0709 13:03:09.699362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.057258 (+357896us) service_pool.cc:224] Handling call
0709 13:03:10.057690 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.008132 (+950442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.920604 (+912472us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b5400240ed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:11.920949 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1813164416}
I20260709 13:03:12.624595 11382 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:12.086977  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.411753  8722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.007s	sys 0.004s
W20260709 13:03:12.615520  8644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.195s	user 0.016s	sys 0.026s
W20260709 13:03:12.430152  8747 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.000s	sys 0.017s
W20260709 13:03:12.641656  8747 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.002s	sys 0.018s
I20260709 13:03:12.641817  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:12.629153  8647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.002s	sys 0.015s
W20260709 13:03:12.616603  9355 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.007s	sys 0.006s
W20260709 13:03:12.643483  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 928040us
I20260709 13:03:12.629869  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 502) took 1939 ms. Trace:
W20260709 13:03:12.623970 11920 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.904s	user 0.015s	sys 0.000s
W20260709 13:03:12.630699  9387 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.184s	user 0.005s	sys 0.010s
I20260709 13:03:12.630914  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:12.629487  8751 rpcz_store.cc:276] 0709 13:03:09.532701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.855114 (+322413us) service_pool.cc:224] Handling call
0709 13:03:09.875577 (+ 20463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.597967 (+722390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.622177 (+2024210us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b54001577d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:12.622405 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1574154496}
I20260709 13:03:11.979481  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:11.907084 11849 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:12.544652 11931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.007s	sys 0.007s
W20260709 13:03:12.653313 11931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.009s	sys 0.008s
I20260709 13:03:12.649513 11948 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.654114  9382 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:12.654222 11948 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:11.905480 11636 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:11.978361 11933 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.274s	user 0.011s	sys 0.002s
W20260709 13:03:11.977669 11918 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.073s	user 0.012s	sys 0.000s
W20260709 13:03:12.212589 11879 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.004s	sys 0.018s
W20260709 13:03:12.095333 11914 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.612s	user 0.012s	sys 0.003s
W20260709 13:03:12.580451 11342 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.507s	user 0.018s	sys 0.000s
W20260709 13:03:12.481412 11751 consensus_meta.cc:287] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.678s	user 0.012s	sys 0.004s
W20260709 13:03:12.147264 11822 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.128s	user 0.010s	sys 0.003s
I20260709 13:03:12.649178  8755 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
W20260709 13:03:12.697990  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 182195us
W20260709 13:03:12.543818 11946 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.743s	user 0.017s	sys 0.009s
W20260709 13:03:12.628129 11953 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.599s	user 0.017s	sys 0.047s
W20260709 13:03:12.130156 11913 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.732s	user 0.008s	sys 0.015s
I20260709 13:03:12.651901  9355 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:12.485700 11923 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.004s	sys 0.011s
W20260709 13:03:12.575177 11934 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.000s	sys 0.012s
I20260709 13:03:11.985802 11792 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:12.640733  8722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.008s	sys 0.008s
I20260709 13:03:12.657431  8748 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:12.616017 11932 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:12.668390 11983 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.658556  8645 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:12.655020 11765 consensus_meta.cc:287] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 2.342s	user 0.016s	sys 0.010s
W20260709 13:03:12.703269 11934 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.001s	sys 0.013s
I20260709 13:03:12.655959 11920 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:12.676820 11623 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:12.627556 11696 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:12.628599 11741 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:12.651619 11924 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 2.121s	user 0.018s	sys 0.007s
I20260709 13:03:12.744959 11924 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.746361  8677 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:12.672866  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:12.636539 11690 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:12.747207  8750 raft_consensus.cc:2410] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:03:12.747411 11690 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:12.747990 11690 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.675941  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2216) took 2221 ms. Trace:
I20260709 13:03:12.672417 11918 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:12.667968 11636 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:12.701907  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 50114.6us
W20260709 13:03:12.668741 11879 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.062s	user 0.010s	sys 0.030s
W20260709 13:03:12.114658 11947 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.997s	user 0.029s	sys 0.043s
I20260709 13:03:12.711162  8751 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:03:12.702813 11923 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.006s	sys 0.012s
I20260709 13:03:12.716423  8722 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:12.700721 11978 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.675890 11914 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.666561 11950 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:12.671090 11541 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.704516 11848 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.702906 11946 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.717839 11743 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:12.709336 11913 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.642953 11927 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.781819 11911 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:12.671908 11973 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:12.664507 11382 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:12.637017 11958 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:12.674976 11977 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:12.700075 11849 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:12.743086  9354 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.012s	sys 0.016s
W20260709 13:03:12.832405  9354 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.025s	user 0.013s	sys 0.017s
I20260709 13:03:12.744387 11915 raft_consensus.cc:2243] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.659660 11936 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:12.747642 11924 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:12.702628 11822 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:12.714741 11980 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:12.731245 11925 consensus_meta.cc:287] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.935s	user 0.025s	sys 0.001s
I20260709 13:03:12.782826  9380 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:12.703063 11751 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:12.670315 11933 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:12.791002  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 74675.2us
I20260709 13:03:12.834491 11879 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:12.793221  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 134774us
I20260709 13:03:12.797881 11541 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.831105 11977 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:12.719383 11792 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.648165  8681 rpcz_store.cc:276] 0709 13:03:10.690107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.694906 (+  4799us) service_pool.cc:224] Handling call
0709 13:03:10.695414 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:10.767292 (+ 71878us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:10.832965 (+ 65673us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.401055 (+568090us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:11.422235 (+ 21180us) log.cc:844] Serialized 26 byte log entry
0709 13:03:11.422475 (+   240us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:12.617496 (+1195021us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.629526 (+ 12030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.629614 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:12.629662 (+    48us) raft_consensus.cc:1685] finished
0709 13:03:12.629673 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.629812 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":308910,"prepare.queue_time_us":57868,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":289125,"raft.run_cpu_time_us":510,"raft.run_wall_time_us":53899,"thread_start_us":1132347,"threads_started":2,"wal-append.queue_time_us":1293025}
I20260709 13:03:12.737633 11765 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.765190 11961 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:12.793078 11947 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.713s	user 0.035s	sys 0.047s
I20260709 13:03:12.802554 11973 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:12.796067 11848 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.784348 11741 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:12.802170 11971 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.831266  8748 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:12.836815  9382 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:12.837653 11964 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.837760 11951 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:12.839835 11918 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:12.840579 11913 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:12.841457 11920 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:12.800012 11923 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:12.836297  8672 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:12.833014 11959 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.057s	user 0.000s	sys 0.004s
I20260709 13:03:12.836052 11969 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:12.849539 11980 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.850355  8754 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
W20260709 13:03:12.855057  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0733646509999062s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.315s for a response
I20260709 13:03:12.749687  8753 rpcz_store.cc:276] 0709 13:03:10.455039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.457864 (+  2825us) service_pool.cc:224] Handling call
0709 13:03:10.472145 (+ 14281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:10.472982 (+   837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.473008 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:10.481757 (+  8749us) log.cc:844] Serialized 26 byte log entry
0709 13:03:10.846747 (+364990us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:10.846801 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.846893 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:12.675654 (+1828761us) raft_consensus.cc:1685] finished
0709 13:03:12.675679 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.675867 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":353445,"threads_started":1,"wal-append.queue_time_us":542313}
I20260709 13:03:12.860652  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:12.749500 11935 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:12.738178 11983 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:12.826056 11911 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:12.917758  9353 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:12.936847  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 248759us
I20260709 13:03:12.942240  9386 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:03:12.927470  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0234065829998826s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
I20260709 13:03:12.920361  8676 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:12.938863  9353 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:12.951849  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 91301.4us
W20260709 13:03:12.944840  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 304924us
I20260709 13:03:12.956099  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:12.935715  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 544) took 2014 ms. Trace:
I20260709 13:03:12.963142  8752 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:12.965085  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.848261 11875 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:12.979089  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:12.976080  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:12.796873 11978 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.799294 11950 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.849566 11936 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.852838 11925 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.972494 11690 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:13.028507 11690 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:13.029583 11690 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.012821  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:13.033075 11541 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:12.951375 11382 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:13.033887 11541 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:13.034925 11875 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:13.036001 11579 consensus_meta.cc:287] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.959s	user 0.021s	sys 0.005s
I20260709 13:03:13.035624 11875 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.031121  8823 raft_consensus.cc:2410] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:13.038517 11749 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.034557 11382 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.039265 11749 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.038260 11875 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:12.918365 11961 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.041851 11849 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:12.919466 11765 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:13.042846 11936 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:13.043292 11983 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:13.015312  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:13.014897  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:13.033905  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:13.013516  8829 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:12.945987 11920 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:13.013782  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:12.918771 11973 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:13.049997  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:12.924710 11923 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.053751  9382 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:12.956894  8827 rpcz_store.cc:276] 0709 13:03:10.921803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.921971 (+   168us) service_pool.cc:224] Handling call
0709 13:03:10.922361 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.922918 (+   557us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:12.615773 (+1692855us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.934949 (+319176us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.935204 (+   255us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:12.935270 (+    66us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:12.935342 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.935442 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.935621 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1291483,"prepare.run_cpu_time_us":145,"prepare.run_wall_time_us":2783,"raft.queue_time_us":598,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":241,"thread_start_us":1941013,"threads_started":2}
I20260709 13:03:12.914156 11977 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:13.057348  8756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.073s	user 0.010s	sys 0.012s
W20260709 13:03:13.058898  8756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.075s	user 0.010s	sys 0.012s
I20260709 13:03:12.925911 11980 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:13.088011  9387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 688) took 1926 ms. Trace:
W20260709 13:03:13.014640 11987 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:03:13.109970  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:13.110571  8830 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:13.110925  8824 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:13.111793  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:13.031113 11915 raft_consensus.cc:2272] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.971701 11951 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:12.925644 11969 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:12.920985 11741 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.026556 11933 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:12.982471 11934 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:12.987187 11935 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:13.075771 11636 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:13.013039 11993 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.046154 11879 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.088651  9387 rpcz_store.cc:276] 0709 13:03:11.133581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.203313 (+ 69732us) service_pool.cc:224] Handling call
0709 13:03:11.380492 (+177179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.380826 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:12.648442 (+1267616us) log.cc:844] Serialized 14 byte log entry
0709 13:03:13.058209 (+409767us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:13.058300 (+    91us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:13.058389 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.058502 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.059366 (+   864us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1407866,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"thread_start_us":1610666,"threads_started":2}
I20260709 13:03:13.110149  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:13.128077  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 171600us
I20260709 13:03:13.074237 11961 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:13.131668  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:13.125766  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 169769us
I20260709 13:03:12.928932 11911 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.121625  8829 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:03:13.077363 11923 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:13.138422  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:13.138918  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 91665.3us
I20260709 13:03:12.952308 11947 consensus_queue.cc:1059] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.176s
I20260709 13:03:13.087643 11958 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:13.030267 11690 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.124851 11951 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:03:13.163798  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:13.145896  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 189275us
I20260709 13:03:13.147429  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:13.133018 11541 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:12.921382 11971 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.158092  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:13.189795  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:13.190867 11953 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:13.191335 11953 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:13.191802 11953 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:13.141446 11934 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.141052 11935 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:12.923178 11964 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.169165  8753 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:13.169678  9353 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:13.172008 11636 raft_consensus.cc:3055] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:13.195259  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:13.191387 11961 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:13.160086  8647 leader_election.cc:400] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11.
I20260709 13:03:13.241418  8647 leader_election.cc:403] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:13.145444 11927 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:13.197916  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 497) took 1291 ms. Trace:
I20260709 13:03:13.245704  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:13.197292  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 500) took 1286 ms. Trace:
I20260709 13:03:13.246734  8674 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:13.244637  8747 rpcz_store.cc:276] 0709 13:03:11.906848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.907182 (+   334us) service_pool.cc:224] Handling call
0709 13:03:11.907683 (+   501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:11.908143 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.908186 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:11.908854 (+   668us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.642260 (+733406us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:12.642313 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.642422 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.197623 (+555201us) raft_consensus.cc:1685] finished
0709 13:03:13.197657 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.197836 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":75373,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":129,"thread_start_us":732898,"threads_started":1,"wal-append.queue_time_us":868039}
I20260709 13:03:13.251389  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:13.273852  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:13.194326 11950 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:13.256628  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:13.257809  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 101012us
I20260709 13:03:13.194399 11936 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:13.322220  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:13.175366 11920 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:03:13.259217  8794 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:13.174655 11951 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:13.255419  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:13.376417  8675 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:13.240044 11947 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:03:13.283895  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:13.265039 11958 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:13.393791  8673 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:03:13.222195 11541 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:13.299212  8747 raft_consensus.cc:2368] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10.
I20260709 13:03:13.246977  8756 rpcz_store.cc:276] 0709 13:03:11.910712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.910846 (+   134us) service_pool.cc:224] Handling call
0709 13:03:11.981591 (+ 70745us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:11.982221 (+   630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:11.982428 (+   207us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:11.983376 (+   948us) log.cc:844] Serialized 52 byte log entry
0709 13:03:13.059694 (+1076318us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:13.059756 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.059876 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.196931 (+137055us) raft_consensus.cc:1685] finished
0709 13:03:13.196955 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.197195 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1650,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":61,"spinlock_wait_cycles":9088,"thread_start_us":1075760,"threads_started":1,"wal-append.queue_time_us":1183690}
I20260709 13:03:13.245103 11946 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:13.286365  8681 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:03:13.384207  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 125092us
I20260709 13:03:13.146095 11879 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.387477 11950 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:13.388516 12009 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:13.389384  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:13.390039 12005 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:13.484400  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:13.485363  8673 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:13.390359 11978 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:13.144328 11993 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:13.391575 12007 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:03:13.275820 11927 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:03:13.160157 12003 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.178s
I20260709 13:03:13.222391 11636 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:13.258968 11690 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:13.273662 11875 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:13.460110  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:13.460490  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 314907us
I20260709 13:03:13.333984 11936 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:13.468421  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.465080  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:13.495484  8674 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:13.332458 11934 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:13.404801 11958 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:13.372900 11924 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.159s
W20260709 13:03:13.571681  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.603893  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:13.527385  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:13.491937  8719 connection.cc:256] RPC call timeout handler was delayed by -1.2508357770000202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
W20260709 13:03:13.600394  8754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 498) took 1690 ms (client timeout 1669 ms). Trace:
I20260709 13:03:13.605371  8747 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
W20260709 13:03:13.607156  8754 rpcz_store.cc:269] 0709 13:03:11.909705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.909867 (+   162us) service_pool.cc:224] Handling call
0709 13:03:13.599987 (+1690120us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54000901d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:13.600183 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":515552512}
W20260709 13:03:13.609690  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 440150us
I20260709 13:03:13.610174  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:13.611068  8747 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:13.613317  8645 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:13.613595  8829 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:13.616444  8824 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:13.617942 11579 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:13.618533 11579 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:13.619040  8719 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:13.621121  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:13.621328 11579 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:13.623447 11946 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:13.624047 11946 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:13.624522 11946 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:13.627104 11947 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:13.627772 11947 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.628254 11947 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:13.527022 12004 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:13.491910 11932 consensus_meta.cc:287] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.762s	user 0.008s	sys 0.005s
I20260709 13:03:13.644001  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:13.649893  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:13.490631 11382 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:13.664999  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:13.496881 12005 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:13.617255 11936 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:13.677630  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:13.556680 11971 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:13.651472  8753 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:03:13.701804  8829 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:13.689508  8674 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:13.496829 11950 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.730736  8753 pending_rounds.cc:85] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 3
I20260709 13:03:13.661602 11908 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:13.732754  8753 pending_rounds.cc:107] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:03:13.680436 11690 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:13.650239 11741 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:13.698549 12005 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:13.701584 11623 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.639438 11875 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.743532  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:13.744395  8829 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:13.649729 12004 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.735469  8753 raft_consensus.cc:2889] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:13.752533  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:13.755697  8674 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:03:13.702407 11920 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 13:03:13.762542  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 313805us
I20260709 13:03:13.743204 11623 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:13.796157  8719 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:13.804095  9384 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:13.512252 12009 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:13.820487  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:13.766845 11579 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:13.824024  9387 raft_consensus.cc:2410] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:03:13.824503  9355 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.105s (SENT) [suppressed 26 similar messages]
I20260709 13:03:13.824695  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:13.719491 11958 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:13.833910 12008 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:13.834442 11958 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:13.834878 11984 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 1.060s	user 0.004s	sys 0.009s
I20260709 13:03:13.820166 11911 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:13.842305 12004 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:13.843202 11950 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:13.844022 11950 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:13.845475  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:13.847404 11993 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:13.847833  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:13.849735  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:13.819304 12021 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:13.854342 12021 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:13.854816 12021 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:13.821936 11935 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:13.810003 11636 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.857874 11636 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:13.859406 12009 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:13.829613 12010 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.125s	user 0.004s	sys 0.005s
I20260709 13:03:13.861644 11541 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:13.863576 11993 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:13.810308 11964 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:13.864353 11993 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:13.846556  9385 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:13.864871 12008 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.867475 12008 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:13.830763 12012 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:03:13.871628  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 507) took 1956 ms. Trace:
I20260709 13:03:13.872145  8751 rpcz_store.cc:276] 0709 13:03:11.915382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.652992 (+737610us) service_pool.cc:224] Handling call
0709 13:03:12.710788 (+ 57796us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.790442 (+ 79654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.871456 (+1081014us) spinlock_profiling.cc:243] Waited 57.3 ms on lock 0x7b54002abf50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:13.871572 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131883392}
I20260709 13:03:13.853402 11875 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:13.875191  8826 raft_consensus.cc:2410] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:03:13.878371  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.870414  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2221) took 1953 ms. Trace:
I20260709 13:03:13.818827 11924 consensus_queue.cc:1243] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:13.880821  8755 rpcz_store.cc:276] 0709 13:03:11.916567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.697647 (+781080us) service_pool.cc:224] Handling call
0709 13:03:12.702079 (+  4432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:13.122181 (+420102us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:13.122215 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:13.130889 (+  8674us) log.cc:844] Serialized 26 byte log entry
0709 13:03:13.333245 (+202356us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:13.333289 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.333373 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.870114 (+536741us) raft_consensus.cc:1685] finished
0709 13:03:13.870141 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.870319 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94311,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":3663,"thread_start_us":608909,"threads_started":2,"wal-append.queue_time_us":406806}
I20260709 13:03:13.882807  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:13.891844  9384 raft_consensus.cc:2410] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:13.882704  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:13.901665  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:13.798764 11951 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:13.914395  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:13.914906 11623 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:13.921193  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:13.922894  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:13.945322  8826 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:13.908962 12025 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:13.931444  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2219) took 2015 ms. Trace:
I20260709 13:03:13.926494 11947 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:13.935388  8675 consensus_meta.cc:287] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.530s	user 0.012s	sys 0.000s
I20260709 13:03:13.935029 11964 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:13.936370  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:13.937968 11980 consensus_meta.cc:287] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.803s	user 0.004s	sys 0.009s
I20260709 13:03:13.938550 11579 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:13.951658  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 338446us
I20260709 13:03:13.951558  8752 rpcz_store.cc:276] 0709 13:03:11.915985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.667377 (+751392us) service_pool.cc:224] Handling call
0709 13:03:12.692037 (+ 24660us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.971245 (+279208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.931004 (+959759us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b540008ff54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:13.931325 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":622802688}
I20260709 13:03:13.953459  8675 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:13.955158  8673 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:13.963865  8827 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:13.964026  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:13.947012 11960 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:13.975350  9382 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:13.968226  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:13.954397 11993 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.070063  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 499) took 2159 ms. Trace:
I20260709 13:03:14.177241  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1737) took 1217 ms. Trace:
W20260709 13:03:14.069839  8677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1710) took 3370 ms (client timeout 1557 ms). Trace:
I20260709 13:03:14.178013  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:14.108989  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.993278  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:14.181792  8675 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:03:14.079245  9389 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:14.181839  9380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 564) took 3047 ms. Trace:
I20260709 13:03:13.983047  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:14.074252  9386 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 685) took 2983 ms (client timeout 1586 ms). Trace:
I20260709 13:03:14.073406  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1709) took 3375 ms. Trace:
I20260709 13:03:14.184237  8673 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:14.184046  8826 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:14.162533  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1711) took 3462 ms. Trace:
I20260709 13:03:14.116986  8829 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:14.185711  8672 rpcz_store.cc:276] 0709 13:03:10.699709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.774211 (+ 74502us) service_pool.cc:224] Handling call
0709 13:03:10.817701 (+ 43490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.924831 (+2107130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.162036 (+1237205us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b54003e03d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:14.162419 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4641945856}
W20260709 13:03:14.179548  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 211466us
W20260709 13:03:14.180817  8677 rpcz_store.cc:269] 0709 13:03:10.699368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.740629 (+ 41261us) service_pool.cc:224] Handling call
0709 13:03:14.069513 (+3328884us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b5400175a54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:14.069691 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4529153792}
I20260709 13:03:14.183928  9380 rpcz_store.cc:276] 0709 13:03:11.123115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.136809 (+ 13694us) service_pool.cc:224] Handling call
0709 13:03:11.797925 (+661116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.847589 (+1049664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.170418 (+1322829us) spinlock_profiling.cc:243] Waited 985 ms on lock 0x7b54002369d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:14.170737 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2264615040}
I20260709 13:03:14.184954  8676 rpcz_store.cc:276] 0709 13:03:10.698041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.698307 (+   266us) service_pool.cc:224] Handling call
0709 13:03:10.698801 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.944012 (+2245211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.073092 (+1129080us) spinlock_profiling.cc:243] Waited 2.22 s on lock 0x7b540033a4d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:14.073317 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5108688128}
W20260709 13:03:14.167814 11946 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.543s	user 0.015s	sys 0.001s
I20260709 13:03:14.191787  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:14.169257 11923 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.001s	user 0.008s	sys 0.005s
I20260709 13:03:14.180486  8821 rpcz_store.cc:276] 0709 13:03:12.960119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.960268 (+   149us) service_pool.cc:224] Handling call
0709 13:03:12.960739 (+   471us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:12.999199 (+ 38460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.999256 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:13.020116 (+ 20860us) log.cc:844] Serialized 52 byte log entry
0709 13:03:13.355267 (+335151us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:13.473887 (+118620us) log.cc:844] Serialized 14 byte log entry
0709 13:03:13.474363 (+   476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.474463 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.869790 (+395327us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b180030fb20. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:03:13.947359 (+ 77569us) raft_consensus.cc:1685] finished
0709 13:03:13.953931 (+  6572us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.177135 (+223204us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":118241,"prepare.run_wall_time_us":61,"raft.queue_time_us":11604,"raft.run_cpu_time_us":481,"raft.run_wall_time_us":58279,"spinlock_wait_cycles":127967872,"thread_start_us":311154,"threads_started":1,"wal-append.queue_time_us":483635}
I20260709 13:03:14.158536 11951 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:14.184782  9386 rpcz_store.cc:269] 0709 13:03:11.090348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.090572 (+   224us) service_pool.cc:224] Handling call
0709 13:03:14.073929 (+2983357us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540064e954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:14.074112 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2538423808}
I20260709 13:03:13.968681 11579 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:13.982905 11924 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:14.156869 11927 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.519s	user 0.012s	sys 0.000s
W20260709 13:03:14.057974 11977 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.936s	user 0.013s	sys 0.000s
I20260709 13:03:13.955312 11969 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.199779  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:13.988543 11953 consensus_meta.cc:287] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.796s	user 0.009s	sys 0.003s
I20260709 13:03:14.227738  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.229143  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:14.229996  8675 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:14.230211  8753 raft_consensus.cc:2410] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:14.201550  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.204097  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.177819  8749 rpcz_store.cc:276] 0709 13:03:11.910222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.910374 (+   152us) service_pool.cc:224] Handling call
0709 13:03:12.029454 (+119080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.620715 (+591261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.620751 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:12.620807 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:12.620842 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.069768 (+1448926us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b5400139054. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:14.069836 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.069993 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1359264512}
I20260709 13:03:14.189913 11687 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:14.072080 11947 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.128593 11822 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.217270  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.217664 11992 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:03:14.218089  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.219319 11925 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.203330 12008 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:14.221704  8824 raft_consensus.cc:2410] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:14.240397  8751 raft_consensus.cc:2410] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:03:14.222189 11690 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:14.202692 11946 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:14.123378 12005 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:14.211404 11541 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:14.228267 11749 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:13.967989 11964 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:14.248502  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.246881 11992 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:14.197742 11936 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:14.056764 11973 consensus_meta.cc:287] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.929s	user 0.006s	sys 0.008s
I20260709 13:03:14.253716  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.233911  8829 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:13.969003 11980 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:14.204186 11993 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:14.243566  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.244606  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.245581  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.246873  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.214075 11927 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11.
I20260709 13:03:14.260834  8754 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:03:14.267930  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.206918 11923 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:14.263074 12005 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.250515 11964 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.246279 11687 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:14.259863 11973 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:14.272590  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.250092 11992 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:14.251228 11690 consensus_queue.cc:1243] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:03:14.240756 12008 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:14.227892 12021 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:14.247085 11792 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.239516 11953 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:14.214202 11924 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:14.247815 11950 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:14.274036  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.274307  8680 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:14.275102  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:14.322535  8796 connection.cc:256] RPC call timeout handler was delayed by -2.0793531119999216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
I20260709 13:03:14.328233  8753 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:14.294117  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.280524 11946 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:14.318599 12021 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.283039 11751 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:14.285638 11879 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:14.338598  8754 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:14.318203 11980 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:14.318409  9352 connection.cc:256] RPC call timeout handler was delayed by -1.4080496300000505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20190000000000002s for a response
I20260709 13:03:14.349766  8822 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:03:14.350219  8822 pending_rounds.cc:85] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 5
I20260709 13:03:14.350427  8822 pending_rounds.cc:107] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 3.6
W20260709 13:03:14.311903 12030 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.060s	user 0.002s	sys 0.000s
I20260709 13:03:14.312201  9383 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 8: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:14.312208 12002 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:14.351450 12002 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:14.327085  8752 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:14.275223  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.332597  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.239043 11977 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:14.322845 11666 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.358953 11666 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:14.352402 12002 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:14.360165  8747 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:14.360723 12002 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:14.337625 11749 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:03:14.270720 11936 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:14.350667  8822 raft_consensus.cc:2889] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:14.355690  8753 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:03:14.372452  8829 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:14.312109 11973 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.313167 12008 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:14.362699  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 546) took 3365 ms (client timeout 1678 ms). Trace:
I20260709 13:03:14.362170  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1716) took 1121 ms. Trace:
I20260709 13:03:14.294281  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.376670  8821 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:14.271031 11993 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:14.379271  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:14.375716  8823 rpcz_store.cc:269] 0709 13:03:10.990714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.999073 (+  8359us) service_pool.cc:224] Handling call
0709 13:03:14.349052 (+3349979us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b5400267ad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:14.355517 (+  6465us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4586508672}
I20260709 13:03:14.376398  8679 rpcz_store.cc:276] 0709 13:03:13.240837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.240998 (+   161us) service_pool.cc:224] Handling call
0709 13:03:13.259248 (+ 18250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.259523 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:13.320155 (+ 60632us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.361629 (+1041474us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b180036d8a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e20a14bc 00007f32df3ef48b 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a
0709 13:03:14.361864 (+   235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.361917 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:14.361965 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.362018 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.362113 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":134616,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":809,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":4895,"spinlock_wait_cycles":27062528,"thread_start_us":175962,"threads_started":1,"wal-append.queue_time_us":439230}
I20260709 13:03:14.378391  8675 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:14.378971 12028 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.384832 12028 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:14.387188  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:14.335806 11950 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:14.384074  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.273128 11687 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:14.388918  8675 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:14.389132  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.390007  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:14.393141  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.393800  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.394789  8821 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:14.383028 11690 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:14.397621  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.403053  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.403077  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.403316  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:14.404244  8829 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:14.404297  8675 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:03:14.403031  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.405722  8821 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:14.405900  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.404191  8753 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 8: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:03:14.406813  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.407655  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.408337  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:14.402637 11666 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:14.410382 11950 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:14.413457  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.414119  8823 raft_consensus.cc:2368] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9.
I20260709 13:03:14.402438 11690 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:14.427268  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.513826  8752 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:14.600180  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.608403  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.609107  8752 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:14.609184 11690 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:14.599700  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:14.608426  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.419183  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:14.598331 11993 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:14.565570  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 1.39712e+06us
W20260709 13:03:14.460933 11696 raft_consensus.cc:670] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 13:03:14.606871 11914 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:14.497107  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.420684 12028 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:14.617277  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 139019us
I20260709 13:03:14.556030 11579 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:14.619357 11915 ts_tablet_manager.cc:1916] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:14.620386 11579 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.459314  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 541) took 3618 ms. Trace:
I20260709 13:03:14.611756  8823 raft_consensus.cc:2410] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:14.610703  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.418200  8829 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:14.610703  8679 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:14.623512  8828 rpcz_store.cc:276] 0709 13:03:10.841310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.841424 (+   114us) service_pool.cc:224] Handling call
0709 13:03:10.917210 (+ 75786us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.679023 (+761813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.458943 (+2779920us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b540009c9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:14.459195 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1700687104}
I20260709 13:03:14.611555  8675 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:14.604348  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1744) took 1135 ms. Trace:
I20260709 13:03:14.612742 11924 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.447535  8826 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:14.625784  8645 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:14.626377  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1751) took 1154 ms. Trace:
I20260709 13:03:14.626968  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1748) took 1156 ms. Trace:
I20260709 13:03:14.606129  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.625311  8754 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:14.624804  8821 rpcz_store.cc:276] 0709 13:03:13.469266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.407993 (+938727us) service_pool.cc:224] Handling call
0709 13:03:14.603251 (+195258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.603888 (+   637us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:14.603916 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.603960 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:14.603998 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.604065 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.604247 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18582}
I20260709 13:03:14.458369  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:14.626844  8826 rpcz_store.cc:276] 0709 13:03:13.471597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.177517 (+705920us) service_pool.cc:224] Handling call
0709 13:03:14.626307 (+448790us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:14.629684  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.629815  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:14.628702  8829 rpcz_store.cc:276] 0709 13:03:13.470692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.412239 (+941547us) service_pool.cc:224] Handling call
0709 13:03:14.417930 (+  5691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.625755 (+207825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.626902 (+  1147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:14.631054  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.631902  8645 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:14.632175  8823 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:14.633244  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:14.634454  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.635172 11953 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:14.635257  8829 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:14.636457  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.637648  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.638716 12028 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:14.639477  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2222) took 2722 ms. Trace:
I20260709 13:03:14.639940  8748 rpcz_store.cc:276] 0709 13:03:11.916847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.716042 (+799195us) service_pool.cc:224] Handling call
0709 13:03:12.716534 (+   492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.921501 (+204967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.639213 (+1717712us) spinlock_profiling.cc:243] Waited 86 ms on lock 0x7b54002fe9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:14.639408 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197702528}
I20260709 13:03:14.640393  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.641007  8756 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:14.641235  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.641930  8748 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:14.643170  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.644456  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.645598  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.646202  8821 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:03:14.647078  8823 raft_consensus.cc:1275] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:14.647941 11948 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.649279 11579 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:14.649948 11579 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:14.651188  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.651933  8748 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:14.652556 12036 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:03:14.651938 12028 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:03:14.655017 12028 raft_consensus.cc:3055] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:14.655447 12028 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:14.631556 11924 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:14.656301 12028 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:14.655025  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:14.657680 12028 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:14.652477 11579 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:14.647992  9389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 565) took 1734 ms. Trace:
I20260709 13:03:14.658201  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:14.661245  8645 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:14.667076  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.675634  8676 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:14.676303  8829 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:14.644740  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.677121  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1749) took 1206 ms. Trace:
I20260709 13:03:14.679147  8756 raft_consensus.cc:2410] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:03:14.671677  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.679570  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.680099  8828 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 13:03:14.677599  8827 rpcz_store.cc:276] 0709 13:03:13.471030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.644475 (+1173445us) service_pool.cc:224] Handling call
0709 13:03:14.677055 (+ 32580us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:14.681782  8675 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:14.681972  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.677508  9389 rpcz_store.cc:276] 0709 13:03:12.913967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.914130 (+   163us) service_pool.cc:224] Handling call
0709 13:03:14.078970 (+1164840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.183320 (+104350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.647815 (+464495us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540064e950. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:14.647934 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2542054272}
I20260709 13:03:14.682880  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.682833  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:14.677492  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.686299  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.684301  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:14.689245  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.689667  8679 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:14.690074  8753 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:14.690652  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:14.691357  8679 raft_consensus.cc:2410] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:03:14.692917  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:14.693538 11541 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:14.693938 11541 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:14.694913  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.695963 11923 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.695637 11541 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:14.696516  8830 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:14.697052 11541 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:14.698122  8752 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:14.698606  8673 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:14.699429  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1727) took 1083 ms. Trace:
I20260709 13:03:14.699623  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.699819  8673 rpcz_store.cc:276] 0709 13:03:13.616076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.698080 (+1082004us) service_pool.cc:224] Handling call
0709 13:03:14.698429 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.699163 (+   734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.699368 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:14.700420 11848 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:14.700929  8752 raft_consensus.cc:2410] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:03:14.701748  9383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 568) took 1649 ms. Trace:
I20260709 13:03:14.702291  8752 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:03:14.687276  8754 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:14.688959  8823 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:14.698678  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1726) took 1088 ms. Trace:
I20260709 13:03:14.703840  8679 rpcz_store.cc:276] 0709 13:03:13.609405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.693642 (+1084237us) service_pool.cc:224] Handling call
0709 13:03:14.694449 (+   807us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.697079 (+  2630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.697107 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.697152 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:14.697183 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.697238 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.697338 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:14.691740  8829 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:14.685567  8827 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:14.706507  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.708129  8821 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
W20260709 13:03:14.708415  8722 connection.cc:256] RPC call timeout handler was delayed by -1.2812016299999414s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
W20260709 13:03:14.708961  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0715016299999434s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
I20260709 13:03:14.685357  8748 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:14.690927  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.711642  8748 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:03:14.711932  8753 raft_consensus.cc:2410] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:03:14.714094  8826 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:14.716063  8828 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:14.702175  9383 rpcz_store.cc:276] 0709 13:03:13.052290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.052478 (+   188us) service_pool.cc:224] Handling call
0709 13:03:14.701557 (+1649079us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54005b13d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:14.701688 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2654797696}
I20260709 13:03:14.686271 11792 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:14.723224  8827 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:03:14.733603  8821 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:14.735311  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.741401  8826 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:03:14.736017  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.737951  8823 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:14.738771  8828 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:14.741218  8756 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:14.738962  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.743264  8827 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:03:14.738447  8679 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:14.748911  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.744541  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.755553  8829 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:14.741140 11872 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.740695  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.758982  8828 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:03:14.757243  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.760210  8752 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:03:14.763767 11872 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:14.765383  8722 leader_election.cc:336] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.508s (SENT)
I20260709 13:03:14.737018 11931 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:03:14.770183 11917 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:14.770748  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:14.771517  8722 leader_election.cc:336] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.566s (SENT)
I20260709 13:03:14.808355  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:14.809616  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.771548  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.775122  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.810657  8826 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:14.762435  8821 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:14.784163 12038 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:03:14.794812  8647 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:14.804055  9355 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:14.799059 11855 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.805353  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2236) took 1565 ms. Trace:
I20260709 13:03:14.819262  8747 rpcz_store.cc:276] 0709 13:03:13.240333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.612059 (+371726us) service_pool.cc:224] Handling call
0709 13:03:13.612377 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.360631 (+748254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.804919 (+444288us) spinlock_profiling.cc:243] Waited 748 ms on lock 0x7b54001581d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:14.805262 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1719336704}
I20260709 13:03:14.810649  8750 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
W20260709 13:03:14.820540  8722 leader_election.cc:336] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.677s (SENT)
I20260709 13:03:14.821065  8722 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:14.815653  8794 leader_election.cc:400] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10.
I20260709 13:03:14.822211  8794 leader_election.cc:403] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:14.822680  9353 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:14.823380  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.831852  8823 raft_consensus.cc:2410] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:14.834713  9353 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:14.770856 11872 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:14.834584  8821 raft_consensus.cc:2410] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:03:14.824057  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:14.850492  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.851612  8827 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:14.852157  8829 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:14.814138 11993 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:14.840049  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:14.856637  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.859136  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.871368  8752 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:14.830444 11696 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:14.835098 11982 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:14.875145  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.881256  8754 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:14.835786 11917 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10.
I20260709 13:03:14.883060  8749 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:14.883693  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.822958 11711 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:03:14.851403 12015 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:14.865284 11749 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:03:14.855593 11276 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:14.885831  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.886251  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.862051 11953 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:14.889734  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.854768 11872 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:03:14.892488  8722 leader_election.cc:336] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.823s (SENT)
I20260709 13:03:14.910265  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:14.912474  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.896593  8754 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:14.881578  8823 raft_consensus.cc:2376] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8. Re-sending same reply.
I20260709 13:03:14.866099 11961 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:03:14.900035  8676 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:14.854429  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.935539  8821 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
W20260709 13:03:14.889357 12002 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.518s	user 0.012s	sys 0.000s
I20260709 13:03:14.903299  8826 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:14.881011 11982 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:14.905334 11953 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:14.904721 11931 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:03:14.904881 11749 consensus_queue.cc:1243] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:03:14.923174  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1729) took 1286 ms. Trace:
I20260709 13:03:14.928361  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.947264  8823 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:14.947950  8722 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.899s (SENT)
I20260709 13:03:14.931964  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.901626 12015 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.904269 11276 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.940627  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.946502  8673 rpcz_store.cc:276] 0709 13:03:13.636544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.700812 (+1064268us) service_pool.cc:224] Handling call
0709 13:03:14.701183 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.701441 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.701468 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:14.702029 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:03:14.839387 (+137358us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:14.839450 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.839560 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:14.922748 (+ 83188us) raft_consensus.cc:1685] finished
0709 13:03:14.922786 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.923071 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":134731,"threads_started":1,"wal-append.queue_time_us":213641}
I20260709 13:03:14.952230  8821 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
W20260709 13:03:14.953562  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 119055us
I20260709 13:03:14.872918 11914 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:03:14.943948 11982 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:14.956811  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.944424  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.929256  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.959337  8754 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:14.949797 11908 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:14.953334  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:14.960445  8821 raft_consensus.cc:2368] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9.
I20260709 13:03:14.961694  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.942140  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:14.953949  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.963228  8821 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:14.954556  9353 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:14.936911 11828 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:14.951246 11953 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:14.950973 12015 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:14.963299  8828 raft_consensus.cc:2376] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10. Re-sending same reply.
I20260709 13:03:14.965682  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:14.953065 11276 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:14.969154  8673 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:14.971132  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:14.971901  8752 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:14.972196 11872 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:14.972903 11872 raft_consensus.cc:697] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:14.976032  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1725) took 1368 ms. Trace:
I20260709 13:03:14.978518  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1717) took 1731 ms. Trace:
I20260709 13:03:14.979194  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:14.981889  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 297691us
I20260709 13:03:14.983489  8645 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:14.986997 12006 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:14.968397 11751 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:14.987993 11751 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:14.987076 11690 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:14.989123 11751 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:14.976508  8674 rpcz_store.cc:276] 0709 13:03:13.607256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.804562 (+197306us) service_pool.cc:224] Handling call
0709 13:03:13.809082 (+  4520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.809332 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:13.855805 (+ 46473us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.683456 (+827651us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.974908 (+291452us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b1800351820. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:03:14.975369 (+   461us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.975556 (+   187us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.975604 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:14.975662 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.975750 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.975967 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":721751,"prepare.queue_time_us":744315,"prepare.run_cpu_time_us":145,"prepare.run_wall_time_us":148,"raft.queue_time_us":789,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":143,"spinlock_wait_cycles":5236480,"thread_start_us":118338,"threads_started":1,"wal-append.queue_time_us":355764}
I20260709 13:03:14.970778  9353 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:14.979106  8678 rpcz_store.cc:276] 0709 13:03:13.247548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.247715 (+   167us) service_pool.cc:224] Handling call
0709 13:03:13.274266 (+ 26551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.274530 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:13.719783 (+445253us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.673163 (+953380us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b18003c5680. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e5ef13c7 00007f32e25039a7 00007f32e25130bf 00007f32e2513038
0709 13:03:14.673435 (+   272us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.718967 (+ 45532us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.977944 (+258977us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.978015 (+    71us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:14.978095 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.978189 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.978432 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":385170,"prepare.queue_time_us":1093124,"prepare.run_cpu_time_us":139,"prepare.run_wall_time_us":145,"raft.queue_time_us":16795,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":19222,"spinlock_wait_cycles":63346688,"thread_start_us":464082,"threads_started":2,"wal-append.queue_time_us":165904}
I20260709 13:03:14.992116  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:14.993216  8826 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 8: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:14.973897 11872 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:14.960847 11908 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:15.008630  8754 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:14.930374 11915 ts_tablet_manager.cc:1929] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:15.013842  8754 raft_consensus.cc:1275] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:15.020448  8674 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:15.019402  8673 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:03:15.023748  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.040035 11718 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:15.044085 12050 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:03:15.022439 11950 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:15.046553 11950 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:15.047075 12048 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:03:15.047703 11950 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:15.014919 11855 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:15.043562 11982 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:15.056663 11915 log.cc:1199] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e9b0850145a4c66bfe983ffdfa93a57: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-1-root/wals/abd86f4e9b544531a7cbfa0042b4fa6c
I20260709 13:03:15.058475 11915 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39018 (request call id 75) took 5952 ms. Trace:
I20260709 13:03:15.059027  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.058984 11915 rpcz_store.cc:276] 0709 13:03:09.307413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.393010 (+ 85597us) service_pool.cc:224] Handling call
0709 13:03:09.889149 (+496139us) ts_tablet_manager.cc:1151] Deleting tablet abd86f4e9b544531a7cbfa0042b4fa6c
0709 13:03:10.030185 (+141036us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:03:14.788713 (+4758528us) tablet_metadata.cc:665] Metadata flushed
0709 13:03:14.930125 (+141412us) tablet_metadata.cc:665] Metadata flushed
0709 13:03:15.058370 (+128245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2618338,"tablet-delete.queue_time_us":417375,"thread_start_us":361154,"threads_started":1}
I20260709 13:03:15.060521  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.061856  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.062980  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:15.040212  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 56884.1us
I20260709 13:03:15.064237  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.064649  8645 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:15.013748 12051 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:15.066260 12051 raft_consensus.cc:697] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:03:15.069672  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:15.071449  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:15.071992  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.072894  8678 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:03:15.077248  8722 leader_election.cc:336] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.910s (SENT)
I20260709 13:03:15.077823  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:15.079126 11855 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:15.080457  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.086938  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.022185 12037 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:15.121968  8673 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:03:15.124076  8756 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:03:15.135025  8756 pending_rounds.cc:85] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 7
I20260709 13:03:15.135425  8756 pending_rounds.cc:107] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:03:15.135763  8756 raft_consensus.cc:2889] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:15.137630  8674 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:15.141170  8753 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:03:15.124814  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 288530us
I20260709 13:03:15.132855 11982 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:15.147346  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.149192  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:15.150496  8826 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:15.157337  8756 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:15.160503  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:15.160571  8823 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:15.162103  8748 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:03:15.121773 11828 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:15.169437  8748 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:15.135874 11855 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:15.172521  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:15.176462  8826 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:15.186229  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 391594us
I20260709 13:03:15.186185  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:15.190280  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.167033 11982 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:15.193121  9384 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:15.208077  9383 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:15.209141  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:15.208128  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.214429  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:15.217221  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:15.217283  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:15.219270  8647 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:15.220069  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:15.245841  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.234704  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:15.182729 11855 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:15.267973  8826 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:03:15.269563  8748 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:15.271982  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.153604 12037 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:15.281203  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.278409  8828 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:15.285076  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.277669  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:15.287294  8826 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:15.291142  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.233336 11973 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:15.295634  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:15.266775 12009 raft_consensus.cc:2764] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:03:15.265515 11792 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:15.279562 12028 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:15.303725  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 143371us
I20260709 13:03:15.309823  9355 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:15.308655  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:15.317150  9355 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:15.304491 11923 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:15.322413  8828 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:03:15.323614  8752 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:15.309268 12028 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:15.204777 12051 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:15.312043 12053 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:15.305486 11541 consensus_queue.cc:1048] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:03:15.355219  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.305613 12006 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:03:15.315913 11792 raft_consensus.cc:697] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:15.354792 11923 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:15.323307 11980 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:03:15.366931 12059 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:03:15.367080  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1736) took 2407 ms. Trace:
I20260709 13:03:15.367568  8822 rpcz_store.cc:276] 0709 13:03:12.959744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.959922 (+   178us) service_pool.cc:224] Handling call
0709 13:03:14.349337 (+1389415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.375222 (+ 25885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.366806 (+991584us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400267ad0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:15.367002 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3194485504}
I20260709 13:03:15.367842 12006 raft_consensus.cc:3055] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:15.373258 12006 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:15.374222  8680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1730) took 1537 ms (client timeout 1706 ms). Trace:
I20260709 13:03:15.366451 11923 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:15.374852  8680 rpcz_store.cc:269] 0709 13:03:13.836411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.946525 (+110114us) service_pool.cc:224] Handling call
0709 13:03:15.373939 (+1427414us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54002567d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:15.374145 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":697096960}
I20260709 13:03:15.376576  8676 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:15.350548 12028 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:15.358729 12053 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.388962  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:15.380170 11690 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:15.387324 11980 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:15.417481  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.418421  8828 raft_consensus.cc:2393] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:15.415319  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.412278 11792 consensus_queue.cc:237] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:15.405118  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2243) took 1068 ms. Trace:
I20260709 13:03:15.423341  8749 rpcz_store.cc:276] 0709 13:03:14.336907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.802406 (+465499us) service_pool.cc:224] Handling call
0709 13:03:14.882056 (+ 79650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.885467 (+  3411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.404794 (+519327us) spinlock_profiling.cc:243] Waited 79.1 ms on lock 0x7b540034a650. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:15.405021 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181874432}
I20260709 13:03:15.394527 12053 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:15.374351 12006 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:15.418520 11934 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:15.453523 11934 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:15.416252 12061 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:03:15.355020 11948 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:03:15.421439 12037 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:15.431114 11855 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:15.431669 11276 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:15.433851  8722 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.167s (SENT)
I20260709 13:03:15.455243  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:15.456221 11933 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:15.458083  8823 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:15.458297  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.438462  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:15.504492  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 66185.1us
I20260709 13:03:15.076994  8564 catalog_manager.cc:5028] TS 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): tablet abd86f4e9b544531a7cbfa0042b4fa6c (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:03:15.447417 12005 raft_consensus.cc:3055] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:15.508468  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:15.517632  8677 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 4.
I20260709 13:03:15.449371 11977 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:15.450261 11961 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:15.519903  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:15.520576  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:15.519039  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:15.521413  8681 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:15.447468  8575 catalog_manager.cc:5697] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 5 to 8, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 8 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:15.427052 11917 raft_consensus.cc:3055] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:15.427960 12006 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:15.453797  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2140) took 1077 ms. Trace:
I20260709 13:03:15.562042  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:15.562286  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:15.562867  8678 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:15.430173 11908 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:15.435159 11718 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:15.459124  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:15.496407  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.470221  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.564643  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.565451  8823 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:15.459383 11276 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:15.483270  9383 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:15.566402 11934 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:15.566910  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:15.567597 11934 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:15.559772 11917 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:15.569437 11917 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:15.487879  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.570469 11917 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:15.571076  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:15.571395  8826 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:15.571772  8673 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:03:15.489259  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.572516  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.491076  8794 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:15.574203  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 83289us
I20260709 13:03:15.575033  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:15.575160  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:15.494714  8747 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:15.495220  8576 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40096 (request call id 140) took 14217 ms (client timeout 15000 ms). Trace:
W20260709 13:03:15.577426  8576 rpcz_store.cc:269] 0709 13:03:01.237396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.316581 (+ 79185us) service_pool.cc:224] Handling call
0709 13:03:15.445371 (+14128790us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:07.936422 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:07.937439 (+  1017us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:07.937512 (+    73us) write_op.cc:435] Acquired schema lock
0709 13:03:07.937558 (+    46us) tablet.cc:662] Decoding operations
0709 13:03:07.949282 (+ 11724us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:07.949451 (+   169us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:07.949508 (+    57us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:03:07.956003 (+  6495us) tablet.cc:701] Row locks acquired
0709 13:03:07.956086 (+    83us) write_op.cc:260] PREPARE: finished
0709 13:03:07.956420 (+   334us) write_op.cc:270] Start()
0709 13:03:07.956719 (+   299us) write_op.cc:276] Timestamp: P: 1783602187956299 usec, L: 0
0709 13:03:07.956771 (+    52us) op_driver.cc:348] REPLICATION: starting
0709 13:03:07.958363 (+  1592us) log.cc:844] Serialized 5090 byte log entry
0709 13:03:09.042793 (+1084430us) op_driver.cc:464] REPLICATION: finished
0709 13:03:09.346203 (+303410us) write_op.cc:301] APPLY: starting
0709 13:03:09.346320 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:09.346649 (+   329us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:09.346826 (+   177us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:09.385788 (+ 38962us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:09.386272 (+   484us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:03:09.386329 (+    57us) write_op.cc:312] APPLY: finished
0709 13:03:09.386838 (+   509us) log.cc:844] Serialized 95 byte log entry
0709 13:03:09.387077 (+   239us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:09.387266 (+   189us) write_op.cc:454] Released schema lock
0709 13:03:09.387339 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:09.399221 (+ 11882us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:09.413310 (+ 14089us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1769865,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":303086,"apply.run_cpu_time_us":5569,"apply.run_wall_time_us":41803,"num_ops":13,"prepare.queue_time_us":15588,"prepare.run_cpu_time_us":28094,"prepare.run_wall_time_us":382044,"raft.queue_time_us":131993,"raft.run_cpu_time_us":44762,"raft.run_wall_time_us":255271,"replication_time_us":1085776,"thread_start_us":626064,"threads_started":4,"wal-append.queue_time_us":453947,"wal-append.run_cpu_time_us":11090,"wal-append.run_wall_time_us":1955925}]]}
I20260709 13:03:15.575644  8828 raft_consensus.cc:2376] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10. Re-sending same reply.
I20260709 13:03:15.581662 12028 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:15.582530  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.583302  8825 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:15.583681  8825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1752) took 2078 ms (client timeout 2159 ms). Trace:
W20260709 13:03:15.584029  8825 rpcz_store.cc:269] 0709 13:03:13.505984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.582244 (+2076260us) service_pool.cc:224] Handling call
0709 13:03:15.583624 (+  1380us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:15.495406  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 202222us
I20260709 13:03:15.585085 12053 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:15.586452  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:15.563514  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:15.587491  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:15.496498  8824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1746) took 2026 ms (client timeout 1930 ms). Trace:
I20260709 13:03:15.589275  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:15.589709  8749 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:15.590149  8823 raft_consensus.cc:2410] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:15.561851  8676 rpcz_store.cc:276] 0709 13:03:14.376167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.939559 (+563392us) service_pool.cc:224] Handling call
0709 13:03:15.376372 (+436813us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.377212 (+   840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.453549 (+ 76337us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b54002567d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:15.453724 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1003580160}
I20260709 13:03:15.591082  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:15.592350  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:15.592586  9386 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:15.592904  8681 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:03:15.594946  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:15.523375  8564 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 446587us
I20260709 13:03:15.509176  9353 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:15.514886  8751 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:03:15.594192  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:15.515508  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.460108  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 515) took 1259 ms. Trace:
I20260709 13:03:15.487118  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.488590  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.557534  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.597318  8755 rpcz_store.cc:276] 0709 13:03:14.205070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.205285 (+   215us) service_pool.cc:224] Handling call
0709 13:03:14.213067 (+  7782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.393033 (+179966us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.393067 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.393113 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:14.393151 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.459828 (+1066677us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b540034b554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:15.459874 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.460033 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":413389056}
W20260709 13:03:15.561484  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.599272  8822 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:15.564893  9382 raft_consensus.cc:2410] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:03:15.470634  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:15.600991  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.484205  9384 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:15.566977  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:15.588346  8824 rpcz_store.cc:269] 0709 13:03:13.470021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.649585 (+179564us) service_pool.cc:224] Handling call
0709 13:03:15.496193 (+1846608us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b5400126954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:15.496398 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1312504704}
I20260709 13:03:15.596639  8681 raft_consensus.cc:2368] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10.
W20260709 13:03:15.506798  8722 leader_election.cc:336] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.218s (SENT)
W20260709 13:03:15.604625  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 97905.5us
I20260709 13:03:15.589612  8675 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
W20260709 13:03:15.605419  8722 leader_election.cc:336] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.233s (SENT)
I20260709 13:03:15.610050  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:15.610440  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.611022  8674 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:15.611693  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.611935  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:15.612514  9384 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:15.612610  8674 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:15.519902 12002 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:15.614044 12002 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:15.467756 11953 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:15.613581 11950 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:15.520129 11961 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:15.614310 11977 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:03:15.620428 11977 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:15.620872 11977 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:15.472477 12015 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:15.614494 12002 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:15.624900 12060 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 13:03:15.626447  8821 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:15.520404 11933 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 13:03:15.615310  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.627700 11933 raft_consensus.cc:3055] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:15.628082 11933 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:15.628247  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.628957 11933 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:15.629480  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:15.629907 12063 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 13:03:15.630270  9380 raft_consensus.cc:2368] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11.
I20260709 13:03:15.597304 11993 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:15.631568 11993 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:15.631961 11993 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:15.633162  8796 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:15.634913  8829 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:15.635180  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.636431  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:15.477007 11848 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.150s
I20260709 13:03:15.641639 12053 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:15.630951  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.617777 11961 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:15.638962  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.616528 11950 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.621846 11977 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:15.613524 12064 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.255s	user 0.000s	sys 0.004s
I20260709 13:03:15.603597  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:15.687311  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.643954  9388 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:15.716260  8674 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:15.795886 11828 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:03:15.714924  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2143) took 1337 ms. Trace:
W20260709 13:03:15.700317  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 125216us
W20260709 13:03:15.884475  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.787956  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1557) took 1058 ms. Trace:
I20260709 13:03:15.646624  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:15.630347 11933 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:15.645496  8678 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:15.704612  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:15.891016  8681 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:03:15.777755  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.660545  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:15.641981  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.895192  8822 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:03:15.885360  8827 rpcz_store.cc:276] 0709 13:03:14.729667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.894121 (+164454us) service_pool.cc:224] Handling call
0709 13:03:14.905821 (+ 11700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.907720 (+  1899us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:15.476670 (+568950us) log.cc:844] Serialized 14 byte log entry
0709 13:03:15.757286 (+280616us) log.cc:844] Serialized 14 byte log entry
0709 13:03:15.787500 (+ 30214us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.787569 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:15.787640 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.787722 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.787880 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":727263,"prepare.queue_time_us":570295,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":98,"raft.queue_time_us":179074,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":205,"thread_start_us":132194,"threads_started":1,"wal-append.queue_time_us":162776}
W20260709 13:03:15.708117  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 199068us
I20260709 13:03:15.891734  8677 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:15.886224  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:15.641717  9387 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 6: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:15.614081  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:15.884289  8675 rpcz_store.cc:276] 0709 13:03:14.377085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.687501 (+310416us) service_pool.cc:224] Handling call
0709 13:03:15.714649 (+1027148us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b54000dacd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:15.714824 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2001354496}
W20260709 13:03:15.882928  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.647754  8575 catalog_manager.cc:5697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 6 to 7, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 7 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:15.818313  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:15.736253  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 103194us
I20260709 13:03:15.887280  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2155) took 1274 ms. Trace:
I20260709 13:03:15.758945 11749 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:15.734513 11751 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:15.802692  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:15.644758 12001 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:15.699520  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:15.610085  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:15.929198  8680 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:15.901909  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:15.904989  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:15.930423  8678 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:15.737865 11961 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:15.787235 11276 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:15.517033 12005 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:15.624970 12015 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:15.916460 11996 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.954133  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:15.485304 11980 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:15.920780 11958 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:15.919427  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:15.919770  8827 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:03:15.617193 11908 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:15.928007 12001 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:15.786971 11666 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:15.648458  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:15.989364  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:15.990319  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:15.917888  8679 rpcz_store.cc:276] 0709 13:03:14.612945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.004394 (+391449us) service_pool.cc:224] Handling call
0709 13:03:15.026444 (+ 22050us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:15.121578 (+ 95134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.121624 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:15.133998 (+ 12374us) log.cc:844] Serialized 52 byte log entry
0709 13:03:15.513692 (+379694us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:15.513756 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.513863 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.691036 (+177173us) raft_consensus.cc:1685] finished
0709 13:03:15.691073 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.887203 (+196130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97647,"raft.queue_time_us":95365,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":10091,"spinlock_wait_cycles":73472,"thread_start_us":342305,"threads_started":1,"wal-append.queue_time_us":480405}
I20260709 13:03:15.616760 11953 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:15.995636  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.941679 11848 consensus_queue.cc:1059] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:03:15.999853  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:16.001340  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:15.922663  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:15.903705 11977 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:15.931847  8681 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:16.003935  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:16.004313  8677 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:16.004657  8679 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 4.
I20260709 13:03:15.962260  8678 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:15.615717 11718 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.305s
I20260709 13:03:15.925302 11751 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.006705 11751 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:16.007616  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:16.008391  8674 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 4.
I20260709 13:03:16.009389  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:16.009979  8674 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:03:15.946557  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:15.991025 12001 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:15.916677 11917 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.841049 12014 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:16.047358 12014 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:16.048331 12014 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:16.049738 12014 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:16.051189 11751 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:16.052752  9384 raft_consensus.cc:2368] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9.
I20260709 13:03:16.058640 11751 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:15.920742 11749 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.946466  8575 catalog_manager.cc:5697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:16.052820  8793 connection.cc:256] RPC call timeout handler was delayed by -3.6932026509999787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
I20260709 13:03:15.733968 11936 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:15.965672  8749 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:16.063040 11749 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:15.786027 11226 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.148s
I20260709 13:03:15.902086 11823 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:03:15.634915  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:16.072589  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:16.073483  8679 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:16.074316  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:16.076203  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:16.077761  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:15.616091 12028 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:15.979570 11996 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:15.916178 11982 consensus_queue.cc:1059] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:03:16.080554 11996 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:16.080947 11982 raft_consensus.cc:3055] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:16.081562 11982 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:15.920615 11948 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:03:15.894572 11950 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:15.942240 11276 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:16.083729 11276 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.084838 11276 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:16.086024 12070 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:03:15.921470  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 319693us
I20260709 13:03:16.087682  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:16.089198  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:15.951056  8753 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:16.011734  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:16.092157  8677 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:16.093374  8676 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:03:16.094295 11687 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:16.021067  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.003463 11908 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:03:16.035929 12077 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:16.037031 11914 raft_consensus.cc:3055] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:16.046449 11917 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.048758  8681 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:16.094118 12028 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:16.130478  8674 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:03:16.065443 11936 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:03:16.094951  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.095095  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:16.095319 11792 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:15.989604 11828 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:16.096283 11982 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:16.095696  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:16.093426  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:16.142405  8675 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:03:16.142933  8675 pending_rounds.cc:85] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 6
I20260709 13:03:16.143184  8675 pending_rounds.cc:107] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:03:16.143260  8677 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:16.143451  8675 raft_consensus.cc:2889] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:16.017383  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 131751us
W20260709 13:03:16.144539  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 54024.5us
I20260709 13:03:16.109719 11908 consensus_queue.cc:1243] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:03:16.064121  8793 connection.cc:256] RPC call timeout handler was delayed by -3.5960026509999352s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
W20260709 13:03:16.147397  8793 connection.cc:256] RPC call timeout handler was delayed by -3.5789026510000213s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
W20260709 13:03:16.147874  8793 connection.cc:256] RPC call timeout handler was delayed by -3.5663026510000009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1695s for a response
I20260709 13:03:16.121345  8749 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
W20260709 13:03:16.148188  8793 connection.cc:256] RPC call timeout handler was delayed by -3.2279026510000222s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20710000000000003s for a response
W20260709 13:03:16.148540  8793 connection.cc:256] RPC call timeout handler was delayed by -3.15860265099991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21480000000000002s for a response
I20260709 13:03:16.121399  8828 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:03:16.148851  8793 connection.cc:256] RPC call timeout handler was delayed by -3.0974026509999248s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22160000000000002s for a response
W20260709 13:03:16.149185  8793 connection.cc:256] RPC call timeout handler was delayed by -2.8994026509999458s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2436s for a response
W20260709 13:03:16.126888  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 482228us
I20260709 13:03:16.135869  8796 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:16.064298  8575 catalog_manager.cc:5697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 2 to 7, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 7 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:16.149466  8793 connection.cc:256] RPC call timeout handler was delayed by -2.7086026509999783s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26480000000000004s for a response
I20260709 13:03:16.150913  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:16.111964  8822 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:16.151811  8575 catalog_manager.cc:5697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 6 to 7, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 7 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:16.152714  8672 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:16.074442  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:16.128436 11914 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:16.157784 12078 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:16.158471 12078 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:16.160063 12078 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:16.160153  8575 catalog_manager.cc:5697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:16.161319 12005 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:16.162158 12072 log.cc:927] Time spent T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:03:16.164510  8681 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:16.134152 11936 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:16.166776  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 569) took 1378 ms. Trace:
I20260709 13:03:16.167234  8830 rpcz_store.cc:276] 0709 13:03:14.788798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.901638 (+112840us) service_pool.cc:224] Handling call
0709 13:03:14.902163 (+   525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.902725 (+   562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.902757 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:14.903852 (+  1095us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.486246 (+582394us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b1800335620. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:03:15.846915 (+360669us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:15.846965 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.847046 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:16.166396 (+319350us) raft_consensus.cc:1685] finished
0709 13:03:16.166420 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.166705 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1976,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":1399,"spinlock_wait_cycles":2759936,"thread_start_us":360435,"threads_started":1,"wal-append.queue_time_us":1240868}
I20260709 13:03:16.166250  8681 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:16.162227  8575 catalog_manager.cc:5697] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 2 to 5, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 5 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:16.171200 11226 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.172029 11666 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:16.171921 11226 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.172525 11666 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:16.172063  8575 catalog_manager.cc:5697] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 3 to 6, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 6 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:16.173478 11666 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.130885 11687 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:16.146454 11828 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:03:16.151032  8793 connection.cc:256] RPC call timeout handler was delayed by -2.2370026510000116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31720000000000004s for a response
I20260709 13:03:16.268721  8793 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:16.270478  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:16.060539  8680 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:16.151072 12028 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:16.150175 11996 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:16.171284 12057 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:16.259416  8575 catalog_manager.cc:5697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 2 to 5, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 5 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
W20260709 13:03:16.263734  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.263852  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 128179us
I20260709 13:03:16.260646  8672 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:16.278713  8796 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:16.282176  8796 leader_election.cc:400] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11.
I20260709 13:03:16.293824  8793 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:16.294759  8796 leader_election.cc:403] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:16.296809  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.297960  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 495412us
W20260709 13:03:16.286420 11993 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.645s	user 0.013s	sys 0.005s
I20260709 13:03:16.297490  8575 catalog_manager.cc:5697] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:03:16.311972  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:16.328449  9384 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:16.286370 11848 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:16.304427  8679 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:16.316144  9352 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:16.291708  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.267231 11953 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:16.289649 12078 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:16.292733 11982 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:16.336850  8575 catalog_manager.cc:5697] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 4 to 5, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 5 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:16.331669  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:16.275951 12057 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:16.362361 12057 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:16.363615 12057 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:16.366838 11687 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.367964  8756 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:16.291606 11996 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:16.371086 11996 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:16.372371 11917 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:16.373116 11917 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:16.326879 12067 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:16.373821 11917 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11.
I20260709 13:03:16.374063 12067 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.376847 12067 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:16.276450 11792 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:16.387642  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.344688 11953 consensus_queue.cc:1059] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:16.344547  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:16.389035  8749 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:16.350762 11914 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.354663 11828 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:16.355295  8755 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:16.341595  9352 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:16.361207 11823 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:03:16.328598 12002 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.704s	user 0.007s	sys 0.007s
I20260709 13:03:16.335813  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:16.441087  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.443655  9381 raft_consensus.cc:2368] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9.
I20260709 13:03:16.336328 12077 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:16.384243  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 452570us
W20260709 13:03:16.385879  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.385461  8575 catalog_manager.cc:5697] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 1 to 5, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 5 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:16.333161 11848 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:16.338214 12081 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:16.445520  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 105374us
W20260709 13:03:16.451056  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 484053us
W20260709 13:03:16.446715  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.462126  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:16.384323 11996 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.382824  9353 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:16.479694  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:16.334820 11993 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.465229  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:16.450415  8749 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:16.467701  8830 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:16.480772  8828 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:16.485550  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.485984  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 103307us
I20260709 13:03:16.388751 11917 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:16.448346 11823 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:16.412616 11933 consensus_meta.cc:287] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.004s
I20260709 13:03:16.448823 12082 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:16.495828  9385 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.009s	sys 0.026s
W20260709 13:03:16.503975  9385 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.009s	sys 0.026s
I20260709 13:03:16.505349  8828 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:16.384120 11751 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 13:03:16.461123 11848 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:16.500188  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.445782 11276 raft_consensus.cc:3055] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:16.486980  9355 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:16.340292 11950 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.497617  8575 catalog_manager.cc:5697] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 4 to 8, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 8 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:16.451297 12002 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:16.502144 11823 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:16.385084 11749 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.463229 11958 raft_consensus.cc:697] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:16.512668  8822 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:16.485667  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:16.516594  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.512320 12067 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:03:16.538416  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 387589us
I20260709 13:03:16.539044  8575 catalog_manager.cc:5697] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:03:16.539767  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:16.490906 11996 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.490487 11931 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:16.490130 11971 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:16.519650  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.520730  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.515354 11917 consensus_queue.cc:1059] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:03:16.527753 12084 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:16.463552 11914 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:16.485313 11953 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:03:16.509088 12015 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:16.412005 11792 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:16.536574  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 49679.8us
I20260709 13:03:16.543536  8750 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
W20260709 13:03:16.537297  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 51751.1us
I20260709 13:03:16.568923  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:16.535353  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.570873  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.537472  8678 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:03:16.533627  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 251601us
I20260709 13:03:16.541343  8719 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:16.523990 11276 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:16.537295  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 243590us
I20260709 13:03:16.577075  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:16.511245 11936 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.520056 11934 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
W20260709 13:03:16.523722  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.574919  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:16.559198 11711 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:16.545285 11931 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:03:16.589958  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 48737.2us
I20260709 13:03:16.591096  8830 raft_consensus.cc:2368] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9.
I20260709 13:03:16.591838  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:16.703744  8823 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:16.655283  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.534098 12082 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
W20260709 13:03:16.576596 12001 consensus_meta.cc:287] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.005s
I20260709 13:03:16.589550 12079 consensus_queue.cc:1059] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.102s
I20260709 13:03:16.674522  8829 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:16.568559 12084 raft_consensus.cc:697] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:16.680783  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:16.683858  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:16.682601  8747 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:16.553256 11933 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:16.692773  8756 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:16.673498  8575 catalog_manager.cc:5697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
W20260709 13:03:16.743919  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 166856us
I20260709 13:03:16.696679  8821 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:16.696507  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:16.744299  8752 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:16.746150  8755 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:16.698077  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 121110us
I20260709 13:03:16.747458  8749 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:16.747816  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:16.698876  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 130051us
I20260709 13:03:16.746686  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:16.698977  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 125219us
I20260709 13:03:16.544445 12067 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.716542  8830 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:03:16.725373  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 238126us
I20260709 13:03:16.769411  8575 catalog_manager.cc:5697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 1 to 5, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 5 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:16.770655  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:16.771617  8824 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:03:16.562294 11953 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:16.737294  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:16.727325 11931 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:16.693235 11971 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:16.786943  8575 catalog_manager.cc:5697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 3 to 4, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 4 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:16.558284 12085 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:16.789153  8575 catalog_manager.cc:5697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 5 to 8, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 8 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:16.762646 11687 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:16.767359 11934 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:16.735916  8828 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:03:16.798743  8647 leader_election.cc:400] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9.
I20260709 13:03:16.799243  8647 leader_election.cc:403] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:16.694208 12089 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.187s
I20260709 13:03:16.775151 11936 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:16.776288 11948 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:16.768673  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:16.780144  8829 raft_consensus.cc:2410] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:16.807891  8749 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:16.780646  8830 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:16.735431 11917 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:16.700219 11711 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:03:16.794119  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.795781  8824 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:16.814365 12097 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:16.749411 11993 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:16.691847  8751 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:16.809439  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 61776.9us
I20260709 13:03:16.836988  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:16.838568  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:16.839932  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:16.739957 12001 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:16.779625 11953 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:16.811129  8575 catalog_manager.cc:5697] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 4 to 5, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 5 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:16.748756  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:16.792476 11971 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:16.820351 12088 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:03:16.780670 11933 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.856045 12088 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:16.856940 11933 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:16.852875 11971 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:16.863848 12084 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:16.864892 12001 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:16.865671 12001 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.847472 11936 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:16.799698 11276 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:16.867352 11953 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:16.849864  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 101743us
I20260709 13:03:16.923411  8755 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
W20260709 13:03:16.837759 12087 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 13:03:16.751698 11958 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:16.874269  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 75716.2us
I20260709 13:03:16.876437  8823 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
W20260709 13:03:16.932523  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 184080us
W20260709 13:03:16.880616  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 106402us
I20260709 13:03:16.868340 12096 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:16.934669 12096 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:16.882115  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.850061  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 84893.9us
I20260709 13:03:16.936532  8796 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:16.824460 12097 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:16.826082 12080 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:16.871001 12088 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.806170 11934 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:16.938390 12014 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.888s	user 0.012s	sys 0.002s
I20260709 13:03:16.954134  8677 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:03:16.931195  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 149644us
I20260709 13:03:16.846079 12002 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:16.845552 11749 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:16.848623 11917 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:16.870069 12092 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:16.925419  8575 catalog_manager.cc:5697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 4 to 5, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 5 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:16.951483  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.866263 12001 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.952476  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:16.960947  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 243502us
I20260709 13:03:16.963214  8679 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:03:16.963636  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 78604us
I20260709 13:03:16.965219  8793 leader_election.cc:400] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10.
I20260709 13:03:16.958984  8672 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:16.966843  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:16.952936 11953 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:16.952684 12088 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:16.949273 11950 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:16.983786 11666 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:16.984328 12086 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
I20260709 13:03:16.985723  9385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 699) took 1375 ms. Trace:
I20260709 13:03:16.953167 12095 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:03:16.986320  9385 rpcz_store.cc:276] 0709 13:03:15.610220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.637318 (+ 27098us) service_pool.cc:224] Handling call
0709 13:03:15.637676 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:15.638104 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.638138 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:15.677233 (+ 39095us) log.cc:844] Serialized 26 byte log entry
0709 13:03:16.504536 (+827303us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:16.504600 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.504692 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:16.985351 (+480659us) raft_consensus.cc:1685] finished
0709 13:03:16.985385 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.985614 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":943,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"thread_start_us":710606,"threads_started":1,"wal-append.queue_time_us":784689}
I20260709 13:03:16.989094 12028 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.989737 12028 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:16.992861  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 527) took 1353 ms. Trace:
I20260709 13:03:16.993294  8673 rpcz_store.cc:276] 0709 13:03:15.639651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.146999 (+507348us) service_pool.cc:224] Handling call
0709 13:03:16.170519 (+ 23520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.990888 (+820369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.990924 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.990968 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:16.991010 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.992596 (+  1586us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b54001ff9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:16.992644 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.992789 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1886190848}
I20260709 13:03:17.012723  8679 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:16.962682 11936 raft_consensus.cc:697] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:16.962597 11792 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:16.965548 12103 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:16.995733 12093 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:03:17.023095 11950 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:17.014871 12057 consensus_meta.cc:287] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.651s	user 0.006s	sys 0.007s
I20260709 13:03:17.035157 11934 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:17.036655  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.015110 11982 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.640s	user 0.007s	sys 0.005s
I20260709 13:03:16.958341 12105 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
W20260709 13:03:17.069350  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 132880us
I20260709 13:03:17.021800 11917 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:17.073701  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 110010us
W20260709 13:03:17.067279  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 100530us
I20260709 13:03:17.079988  8796 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:16.958487 11711 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:17.087075  8796 leader_election.cc:400] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9.
I20260709 13:03:17.087476  8796 leader_election.cc:403] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:17.081753 11792 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.063206 12115 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:17.096546  9355 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:17.098367  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:17.039230 11751 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:03:17.073442 12110 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:17.109663 11982 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:17.110349 11982 raft_consensus.cc:697] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:03:17.111123 11848 consensus_meta.cc:287] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.585s	user 0.012s	sys 0.000s
I20260709 13:03:17.062896 12106 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9.
I20260709 13:03:16.974701 12097 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.091997 11908 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:17.089927 12092 raft_consensus.cc:697] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:17.135704  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.117538 11848 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.087522 11917 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:17.198480  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.126981 11961 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.178s	user 0.014s	sys 0.001s
W20260709 13:03:17.242837  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.073853 12103 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:17.248713  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.107430 11792 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:17.266110 12114 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:03:17.089767 12116 consensus_queue.cc:1059] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:03:17.267211 11792 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:17.268090  8821 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:17.268296 11848 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:17.269905 12103 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:17.121606 12014 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.124s
I20260709 13:03:17.129530 12121 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:03:17.244077 12093 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:03:17.241333 12097 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:17.111521 11982 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:17.291199  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 358221us
I20260709 13:03:17.246945 12105 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.303668 12105 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.244889 12095 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.306984 12095 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:17.306373 12105 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:17.311842 12130 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:17.312721 12103 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:17.313197 12130 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:17.314215 12130 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:17.315553 12130 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.316778 12103 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:17.317396 12130 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:17.318588  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:17.319840 11792 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:17.320901 11792 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:17.321908 11792 consensus_queue.cc:1243] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:17.322496 11792 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:17.323446 11792 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:17.323788  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.324867 11792 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.259914 11908 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.253608 12096 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:17.325341  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:17.241250 12117 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9.
I20260709 13:03:17.134876 12115 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.199s
W20260709 13:03:17.350467  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 389509us
W20260709 13:03:17.351109  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.355549  8647 leader_election.cc:400] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9.
I20260709 13:03:17.292619 12120 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:03:17.334225 12078 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.999s	user 0.015s	sys 0.000s
I20260709 13:03:17.118326 12110 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:17.363300  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:17.356670  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:17.336954 12028 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:17.285188 11971 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:17.383541  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:17.384758  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:17.384953 11971 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:17.384991  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:17.385830  8829 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
W20260709 13:03:17.358322 12108 log.cc:927] Time spent T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.329s	user 0.002s	sys 0.000s
I20260709 13:03:17.381724  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:17.212129 11936 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.335881 11908 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.383708  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:17.392437  8823 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:17.382537  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:17.393092  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:17.393321  8828 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:17.383013  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:17.355170 12096 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:17.389523  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:17.391484  8673 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:17.405252  9381 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:17.357838 11792 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:17.388198 11934 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:17.404294  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:03:17.414551  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 316269us
W20260709 13:03:17.410698 12094 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.598s	user 0.001s	sys 0.003s
I20260709 13:03:17.419598  8826 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:17.422556  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:17.423468  9384 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:17.424193 12110 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:17.428786 11931 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.625s	user 0.010s	sys 0.001s
I20260709 13:03:17.429345 11931 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:17.438767  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:17.445569  9384 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:03:17.449973  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 92961.6us
I20260709 13:03:17.442764  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:17.419121 12096 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:17.455961  9382 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:17.457892  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:17.463833  8793 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:17.467412 12105 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:03:17.468221 11823 consensus_meta.cc:287] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.927s	user 0.014s	sys 0.000s
I20260709 13:03:17.468374  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:17.469091  8673 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:17.468768 11823 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:17.361502 12092 consensus_queue.cc:237] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.472642  9385 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:03:17.436532 11934 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:17.476719  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.477267  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.479005  8673 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:17.454149  9384 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:17.480165  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:17.487628  8677 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:17.540560  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:17.478426 12128 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:17.482439  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:17.544107  8673 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:17.455272 12096 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.542335  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:17.544656  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 62285.6us
I20260709 13:03:17.552656  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:17.555756  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:17.456277 12014 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:17.486300 11914 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.922s	user 0.012s	sys 0.003s
W20260709 13:03:17.548710 12085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.018s	sys 0.004s
I20260709 13:03:17.479838 12135 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:17.442127 12129 consensus_queue.cc:1059] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:03:17.486640 12119 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:03:17.567739 12119 raft_consensus.cc:3055] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:17.568148 12119 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:17.568997 12119 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:17.550944 11917 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:17.561053 11958 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:17.559289  8647 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:17.559288 12057 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:03:17.561209 12085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.018s	sys 0.005s
I20260709 13:03:17.559366  8793 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:17.579824 11666 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.595s	user 0.012s	sys 0.000s
I20260709 13:03:17.589412 12110 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.558449 12096 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:17.570430 12119 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:17.595248  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 107940us
W20260709 13:03:17.570555 11953 consensus_meta.cc:287] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.000s
I20260709 13:03:17.598737  8793 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:17.551690 12128 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:17.590739 12088 consensus_meta.cc:287] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.574s	user 0.008s	sys 0.006s
I20260709 13:03:17.596572  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:17.603181  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:17.540814 12105 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:03:17.598594 12110 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.600497 11914 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:17.575389 11823 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:17.608355 12110 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:17.608493 11934 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.612437  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1597) took 2028 ms. Trace:
I20260709 13:03:17.606256  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:17.589762 12129 raft_consensus.cc:3055] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:17.579722 11931 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:17.591076 12028 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:17.612995  8826 rpcz_store.cc:276] 0709 13:03:15.583991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.588696 (+  4705us) service_pool.cc:224] Handling call
0709 13:03:15.632659 (+ 43963us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.420226 (+1787567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.612090 (+191864us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54002189d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:17.612349 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4108583168}
I20260709 13:03:17.616117  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:17.628261  8680 raft_consensus.cc:2410] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:17.615844  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:17.629087  8672 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:17.620752  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:17.560760 11936 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:17.624078  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:17.590023 12085 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:17.613277 12128 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:17.630440  8674 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:17.633806  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.563098 12014 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:17.577010 12095 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:17.637069  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:17.613212 11917 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:17.643424  8672 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:17.618265 12129 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:17.650759  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:17.651284  8677 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:03:17.631940 11792 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.601310 11958 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:03:17.625124 11848 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:03:17.652031  8674 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:17.626897 12028 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:17.657112  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 530) took 2153 ms. Trace:
I20260709 13:03:17.644372 11936 raft_consensus.cc:697] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:17.658957  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:17.645224 12105 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.660977  8674 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:17.657552  8750 rpcz_store.cc:276] 0709 13:03:15.503587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.503779 (+   192us) service_pool.cc:224] Handling call
0709 13:03:15.556529 (+ 52750us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.564200 (+1007671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.656855 (+1092655us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b5400102a54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:17.657044 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2269482880}
I20260709 13:03:17.671830  8673 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:17.653496 11917 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:17.657359 12028 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:17.615480 12088 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.658013 11828 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:17.679418 12123 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:17.679719 12102 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:17.680527 11917 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.681491 11996 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:17.683590 12014 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:17.684057 12132 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:17.656644 11953 consensus_queue.cc:1059] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:03:17.655440 11792 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:17.712311  8826 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
W20260709 13:03:17.716293  8645 leader_election.cc:336] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.608s (SENT)
I20260709 13:03:17.716854  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:17.718040  8679 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
W20260709 13:03:17.735086  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 801080us
I20260709 13:03:17.675230 12105 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:17.703431 12102 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:17.772848  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 535) took 1424 ms. Trace:
I20260709 13:03:17.720036 11996 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:17.772060 12101 raft_consensus.cc:3055] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:17.779458 12098 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 13:03:17.780777 12002 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:03:17.782060  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 1.04043e+06us
W20260709 13:03:17.782498  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.731971 12132 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:17.727286 11687 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:17.774483  8753 rpcz_store.cc:276] 0709 13:03:16.348290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.348509 (+   219us) service_pool.cc:224] Handling call
0709 13:03:16.349038 (+   529us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:16.349559 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.349599 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:16.350432 (+   833us) log.cc:844] Serialized 52 byte log entry
0709 13:03:16.556913 (+206481us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:16.740148 (+183235us) log.cc:844] Serialized 14 byte log entry
0709 13:03:17.684493 (+944345us) spinlock_profiling.cc:243] Waited 8.87 ms on lock 0x7b1800400420. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:03:17.766596 (+ 82103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.766772 (+   176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:17.766866 (+    94us) raft_consensus.cc:1685] finished
0709 13:03:17.772463 (+  5597us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:17.772735 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32718,"prepare.queue_time_us":234000,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":60,"raft.queue_time_us":1096404,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":28755,"spinlock_wait_cycles":20412160,"thread_start_us":401876,"threads_started":3,"wal-append.queue_time_us":402153}
I20260709 13:03:17.790566  8722 leader_election.cc:400] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9.
I20260709 13:03:17.769254 12077 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:17.794190  8722 leader_election.cc:403] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:03:17.708444 12090 consensus_queue.cc:1059] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:03:17.784339 12101 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:17.784991 12078 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.788136 11226 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:17.674245 12129 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:17.676820 12085 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:17.792083 12132 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:03:17.807412  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.808818 12147 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.795163 11993 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.733289 11936 consensus_queue.cc:237] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.815689 12092 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:17.816519 12142 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:03:17.816401 12092 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.818851 12113 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:03:17.820883 12088 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:17.823760  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:17.827739  8674 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:17.829289  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:17.830029  8677 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
W20260709 13:03:17.839454  8645 leader_election.cc:336] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.732s (SENT)
I20260709 13:03:17.841652  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:17.842973 11792 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:17.843132 12005 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:17.843523 12105 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:17.843801 12090 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:17.845655 12090 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:17.846551 12081 raft_consensus.cc:3055] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:17.847082 12081 raft_consensus.cc:740] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:17.828171 11933 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:17.848590 11933 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:17.849257  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:17.855384  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:17.869415  9384 raft_consensus.cc:2410] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:03:17.807435 12078 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.877686  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:17.878587  8749 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:17.882829  8751 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:17.884686  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:17.885250  8749 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:17.885625  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:17.886287 12146 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:17.860893 12130 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:17.814008 11993 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:17.861212 12089 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9.
I20260709 13:03:17.889643 11993 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:17.891033  8747 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
W20260709 13:03:17.893652  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.893635  8826 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:17.885058  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:17.895707  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.907651  8674 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:17.910729  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:17.911482  8752 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:17.913142  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 122794us
I20260709 13:03:17.913164  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:17.914808  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:17.928568  8752 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:17.932269  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:17.945287  8748 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:17.933496  8749 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:17.935782  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:17.939564  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:17.943616  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:17.948674  8756 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:17.901616 12092 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:17.952529  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:17.953543  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:17.954512  8824 raft_consensus.cc:2410] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:03:17.955304  9353 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:17.960523  8825 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:17.963862  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:17.968429  8722 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:17.949223 12117 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.972198  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:17.883937 11687 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.978653  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:17.982472  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:17.985313  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:17.814528 12147 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.012210  8756 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:17.956290 11872 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:03:18.014082  8751 raft_consensus.cc:2410] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:03:17.963614 12053 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:03:17.846805 12090 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.848151 12081 consensus_queue.cc:260] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.015127  8676 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:03:18.015360 12081 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:18.015549 12090 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:17.982903 12089 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:17.886992 12146 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.019068 12112 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.019740 12112 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:18.020898 12146 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:17.821759 12102 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:17.951246 11933 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.023167 11933 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:17.975967 11977 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:17.981012 11993 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:17.819479 12101 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:17.981600 12014 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:18.031801 12014 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.033558 12014 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:17.974226 12155 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:18.038729  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.030471 12101 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:17.974941 12148 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:18.058641  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:18.100644  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 301879us
I20260709 13:03:17.978693 12117 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:18.026592 12147 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:18.061776 11828 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:18.051400 12002 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.084498 12089 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:18.029560 12112 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:18.088063 11823 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.111774  8719 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:18.119858  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 134658us
I20260709 13:03:18.105067 11934 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.121572  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:18.121692 12002 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:18.134548  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1784) took 1396 ms (client timeout 1713 ms). Trace:
I20260709 13:03:18.133339  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:18.129760  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:18.130638  9384 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:18.105124 12156 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:18.125499 12119 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.524s	user 0.006s	sys 0.007s
W20260709 13:03:18.135383  8821 rpcz_store.cc:269] 0709 13:03:16.739191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.746332 (+  7141us) service_pool.cc:224] Handling call
0709 13:03:18.134213 (+1387881us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b540002b7d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:18.134447 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1156939648}
I20260709 13:03:18.142856  8719 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:18.166512  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:18.167073  8719 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:18.122324 11823 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:18.130564 11828 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:18.141458 11687 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:18.199414 12157 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:03:18.200457 11993 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.201128 11993 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:18.134795 12166 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:18.245759  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 79316.6us
W20260709 13:03:18.246662  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 79720.7us
W20260709 13:03:18.213094  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.146420 12078 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:18.129209 11934 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:18.256455  8722 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:18.180976 12163 raft_consensus.cc:2764] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:03:18.185657 12112 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.199519 12146 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:18.273041  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:18.275202  8826 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:18.248474 12147 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:18.252415 12119 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:18.186218 12148 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 13:03:18.259650 11828 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:18.271450 12078 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:03:18.277292 12096 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.664s	user 0.008s	sys 0.007s
W20260709 13:03:18.332742  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 377546us
I20260709 13:03:18.282495 11993 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:18.299577 11687 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:18.301988 11934 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:18.303217 11823 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:18.330133 12147 raft_consensus.cc:697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:18.331538 12119 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.286469 12112 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:18.282745 12014 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:18.356455 12167 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:03:18.339841 12104 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:03:18.330946 12163 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:03:18.341558 12096 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.374899 12147 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:18.411549  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 226610us
I20260709 13:03:18.424211  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.424242  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:18.425444  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.426229  8750 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:18.426818  8755 raft_consensus.cc:2368] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9.
I20260709 13:03:18.428861  8749 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:18.428710  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:18.290272  8722 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:18.431509  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 141221us
I20260709 13:03:18.435878  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:18.437480  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:18.439015  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:18.442427  8678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2176) took 2399 ms (client timeout 2896 ms). Trace:
W20260709 13:03:18.443163  8678 rpcz_store.cc:269] 0709 13:03:16.043340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.087387 (+ 44047us) service_pool.cc:224] Handling call
0709 13:03:18.441770 (+2354383us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54003395d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:18.442112 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1032233344}
I20260709 13:03:18.444540  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.447497 11666 raft_consensus.cc:3055] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:18.448272 11666 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:18.448807  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:18.450402 11666 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:18.349974 11687 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:18.453341  8755 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:18.455271 11687 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.459331 11934 consensus_queue.cc:1048] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:18.463646 11914 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:18.464465 11914 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:18.465958 11936 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.466923 11792 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.467320 11936 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.467828 11792 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:18.471556 11934 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:18.447926  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:18.475888  8821 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
W20260709 13:03:18.489315  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.496183 12078 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:18.497965 12078 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:18.498679 12001 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:18.500298 12078 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.502537 12079 raft_consensus.cc:3055] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:18.503271 12079 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:18.504537  8673 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:18.505192 12079 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.502563  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:18.360926 12117 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:18.529497 12081 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.529757  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:18.530851 12081 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.340191 12102 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
W20260709 13:03:18.539748  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 98480.9us
I20260709 13:03:18.517217 11934 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.542299  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.543454 11792 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:18.543942  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:18.544883  8755 raft_consensus.cc:2376] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6. Re-sending same reply.
I20260709 13:03:18.546104  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.546219 12096 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:18.557523  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:18.466322 11914 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:18.533769 12130 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.533715 12105 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:18.548280 12079 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:18.548319 12067 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:18.579830  8753 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.019s	sys 0.003s
I20260709 13:03:18.580703 11934 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.583587 11953 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:03:18.584889 12101 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.543s	user 0.011s	sys 0.004s
I20260709 13:03:18.585789  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:18.586668  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:18.587598 12014 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:18.590950  8821 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
W20260709 13:03:18.590317 12128 consensus_meta.cc:287] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.946s	user 0.014s	sys 0.000s
I20260709 13:03:18.599948 12179 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:03:18.600560 12081 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:18.600911  8674 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:03:18.601742  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 70646.6us
I20260709 13:03:18.602721  8677 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:18.603701  8681 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:18.604198  8680 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:18.604389  8676 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:18.603801 12105 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:18.604925 12015 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:18.605298  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:18.606212  8679 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 8: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:18.606456  8751 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:18.607725 12112 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:18.608597 11936 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:18.608968  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:18.613036  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.613734  8748 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:18.584896  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:03:18.614805  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.616300  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:18.616789  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:18.632058 11953 consensus_queue.cc:1243] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:03:18.632756  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 103977us
I20260709 13:03:18.633216  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:18.643803  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:18.650902  8672 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:18.651677  8676 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:18.654064  8750 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:18.656066  8675 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:18.656736  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:18.659061  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.660701  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.661334  9352 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:18.655898  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.652347  8673 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:18.664131  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.673962  9382 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:03:18.675987 12119 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:18.678870  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:18.679881  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.681285 12130 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:18.683528  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 50103.8us
I20260709 13:03:18.683557 11936 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.683862  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:18.669564  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:18.684753  9355 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:18.684705 11936 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.685436  8677 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:18.684871  8748 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:03:18.682859 12081 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.694749  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:18.695297 12081 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.696826  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:18.698149  8749 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:18.701063 11936 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:18.702780  8750 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:18.703490 12119 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.704284  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.701234 12112 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:18.707406  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.707926  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:18.687294  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:18.715206 12147 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.716310 12147 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.716800  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:18.719839  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.720429  8750 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:18.721861 12147 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:18.722633  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.724251 11934 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:18.725112  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.726367  8755 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:03:18.727041 12028 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.040s	user 0.016s	sys 0.012s
I20260709 13:03:18.727305 11980 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:18.727546  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.729313 12105 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:18.729092  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:18.730127  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.730058 11276 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:18.730381  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:18.663231 12128 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:18.724352  8675 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:03:18.738255  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47206us
I20260709 13:03:18.745559  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:18.746155  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.756541  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:18.757817 12130 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:18.758984 12096 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:18.759527 12096 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:18.759581  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.760212 12096 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:18.760643  8751 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:18.761947  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:18.762512 12128 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:18.762732  8751 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:18.766459  8676 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:18.767203  8679 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:18.768735  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:18.769667  8679 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:03:18.704290 12119 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.773518 12119 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:18.773643 12002 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:18.775197  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:18.727625 12028 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:18.777499  8680 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:03:18.779121 12028 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:18.776777  8676 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:18.780725  8749 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:18.784230  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 101659us
I20260709 13:03:18.794449  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.787266 11980 raft_consensus.cc:697] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:18.800704  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:18.809233  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 57851.8us
I20260709 13:03:18.800061 12105 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:18.811653  9355 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:18.812408 12105 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:18.812108 11980 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:18.812742  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:18.813769 12105 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:18.800259 12085 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:18.814956  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:18.818971  8677 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:18.800558 12112 raft_consensus.cc:697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:18.819917  8749 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:18.818050  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.819454  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.826658  8754 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:18.843780  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.852057  8680 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:18.855826  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.857010  8755 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
W20260709 13:03:18.857546  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 254149us
I20260709 13:03:18.858382 12102 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:18.858958 12102 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.859441 12102 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:18.859510 12085 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:18.861807  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.863271  8750 raft_consensus.cc:2393] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:18.859546  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:18.869479  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.891224  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:18.880538  8722 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:18.863859 12112 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:18.810245 12081 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:18.915195  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 54797us
I20260709 13:03:18.923784  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:18.925002  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 94064.9us
I20260709 13:03:18.923751  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:18.927305  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 117041us
W20260709 13:03:18.936084  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.937258  8748 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:18.937074  8673 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:18.933105  8647 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:18.931736 11687 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:18.953531  8747 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:18.949267  8794 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:18.955821  8681 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:18.956936  8677 raft_consensus.cc:2410] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:03:18.959537  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.965167  8748 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:18.964787 12080 consensus_queue.cc:1059] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:03:18.966193 12080 raft_consensus.cc:3055] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:18.964967  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:18.966682 12080 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:18.967095  9382 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:18.968101 12080 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:19.041625  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 53116.4us
W20260709 13:03:18.979560  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 116572us
I20260709 13:03:19.038826  8676 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:03:19.006510  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 71981.2us
I20260709 13:03:19.017529  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:19.009127  9382 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:03:18.970031 12080 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:18.982237  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:18.984447  8749 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:19.017238  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:19.015563  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:19.084069  8679 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:19.004845  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:19.007032 11276 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:19.082351  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 153862us
I20260709 13:03:19.010059  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:19.018275  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:19.082571  9382 pending_rounds.cc:85] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting all ops after (but not including) 6
I20260709 13:03:18.986923  8830 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:19.013523 11914 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:19.001696 12089 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.882s	user 0.016s	sys 0.006s
I20260709 13:03:18.953608 11687 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.101514  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:19.003599 12109 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:19.097851 12028 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:18.980334 12113 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.182s
W20260709 13:03:18.994817 12190 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:19.171607 12109 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:19.172991  8673 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:19.005268 11958 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:19.174944  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 73234.1us
W20260709 13:03:19.135614  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 125625us
I20260709 13:03:18.976104 12101 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.187s
I20260709 13:03:19.132249  8677 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:19.166791  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:19.107317  8751 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:19.165242  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:19.169679 11276 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:18.975606 12176 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.188s
I20260709 13:03:19.199414  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:19.107457  9382 pending_rounds.cc:107] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:03:19.200493  8673 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:19.197558  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:19.200963  9382 raft_consensus.cc:2889] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:19.121783  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:19.023087 12129 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:18.977953 12096 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:19.162907 11687 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:19.187628  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 182888us
I20260709 13:03:19.166222 12028 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.214531  9353 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:19.222594  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 135799us
I20260709 13:03:19.172958 12089 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:19.223917  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:19.229425 11953 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:19.204355 12147 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:19.233634  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:19.230978 11276 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:19.230175  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.230070  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:19.236588  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.107930 12002 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.108s
I20260709 13:03:19.233227  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:19.230098  8676 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:19.298180  8677 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:19.229977 11848 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.234890 12028 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:19.313069  8676 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:19.233503 12176 consensus_queue.cc:1243] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:03:19.296583 12079 consensus_meta.cc:287] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.668s	user 0.015s	sys 0.000s
I20260709 13:03:19.310588  8681 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:19.287659 12130 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.289113 11953 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:19.332430  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 103872us
I20260709 13:03:19.231359 12177 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:19.331459  8756 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:19.339100  8754 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
W20260709 13:03:19.323781  8645 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.576s (SENT)
W20260709 13:03:19.341630  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 207625us
I20260709 13:03:19.230360 12005 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.132s
I20260709 13:03:19.342777  8673 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:19.344302 12081 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:19.346673  8645 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:19.346870  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 168478us
I20260709 13:03:19.215474 11958 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:19.342139 12177 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.349761  8679 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:19.338467 11982 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:19.349036  9352 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:19.366580  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:19.355770  8793 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:19.342542 11953 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:19.352852 12177 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:19.370210  8794 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:19.331240 11980 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
W20260709 13:03:19.385211  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50704.9us
I20260709 13:03:19.370018  9353 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:19.355701 12081 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:19.341404 12130 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:19.365403 11982 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:19.426604 12198 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:03:19.416512 12101 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:19.429333 12101 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:19.430859 11996 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:03:19.432147 12193 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.235s	user 0.002s	sys 0.001s
W20260709 13:03:19.433327  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 76736.4us
I20260709 13:03:19.339804 12147 raft_consensus.cc:3055] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:19.374979 12109 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:19.416008 12126 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:03:19.456073  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.457211 11931 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:19.456732 12081 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:19.432619 12101 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:19.438123 11982 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:19.462872  8679 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:19.463016  8673 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:03:19.353654 12102 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:19.421181 12079 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:19.469043  8824 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.029s	sys 0.105s
W20260709 13:03:19.481640  8824 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.029s	sys 0.105s
W20260709 13:03:19.461370  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 97985.2us
I20260709 13:03:19.477866  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:19.486285  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:19.369199 11823 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.153s
I20260709 13:03:19.465652 12126 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:19.497963  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 44069.2us
I20260709 13:03:19.505527  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:19.482813 12079 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.508749  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:19.511310  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:19.467340 12147 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:19.520090  8756 raft_consensus.cc:2410] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:03:19.511937  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:19.499816 12126 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:19.332754 11848 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:19.523342  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:19.500519 12096 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:19.525647  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 225934us
I20260709 13:03:19.513155  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:19.530341  8681 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:19.536084 12053 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:03:19.504771 12123 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.541656 12109 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:19.543869  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:19.528383 12132 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:19.549175 12195 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
W20260709 13:03:19.570474 12083 consensus_queue.cc:1175] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" responder_term: 4 status { last_received { term: 4 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 4 index: 6 } }
I20260709 13:03:19.570866 12067 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:19.554911 12169 log.cc:927] Time spent T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 13:03:19.511231 12079 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:19.536103 12051 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.129s
I20260709 13:03:19.529235 12119 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:19.545140  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:19.571950 12106 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.566167  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:19.569020  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:19.580067  9382 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:19.568017  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:19.575728  8647 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.854s (SENT) [suppressed 27 similar messages]
W20260709 13:03:19.560335  8645 leader_election.cc:336] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.586s (SENT)
I20260709 13:03:19.584369  8645 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:19.575201 12119 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.587074  8673 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:19.562031 12130 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:19.564514 12102 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:19.570261 12132 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:19.591048 12119 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:19.598667  8796 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:19.604099  8756 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:19.563524 11958 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:19.555343 12147 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:19.609253  8679 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:19.554519 12028 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:19.627460  8674 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:19.571550 12067 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:19.628661 12028 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:19.631274 12067 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:19.623157  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 62916us
I20260709 13:03:19.624214 12103 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.636574 11982 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:19.637141 12103 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:19.637327 12177 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:19.639593 12103 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:19.641917 12001 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:19.555044 12123 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:19.644846  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:19.592026 12106 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:19.646052 12096 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:19.648306 12106 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:19.624577 11996 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.650310 11996 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:19.651058 12149 raft_consensus.cc:3055] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:19.651501 12149 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:19.652896 11996 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:19.653548 11996 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:19.655864 11666 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:19.657932  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 544) took 1027 ms. Trace:
I20260709 13:03:19.658644 11996 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.658447  8678 rpcz_store.cc:276] 0709 13:03:18.636476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.658281 (+ 21805us) service_pool.cc:224] Handling call
0709 13:03:18.663724 (+  5443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.744405 (+ 80681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.744455 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.819001 (+ 74546us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.921366 (+102365us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:18.921596 (+   230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.657390 (+735794us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b5400175054. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:19.657455 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.657509 (+    54us) raft_consensus.cc:1685] finished
0709 13:03:19.657525 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.657827 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":173,"spinlock_wait_cycles":62197120,"thread_start_us":101885,"threads_started":1,"wal-append.queue_time_us":386236}
I20260709 13:03:19.660128 11276 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:19.661967 11276 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.639623 11982 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:19.659343  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 67647us
I20260709 13:03:19.662323 11276 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:19.652519 12149 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:19.649042 12106 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:19.661182 11996 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:19.676000 12149 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:19.603117 11917 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:03:19.664659 12101 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.698864 12177 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:19.671257 12147 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:19.726445  8645 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:19.723531 11276 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:19.728546  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 523530us
W20260709 13:03:19.729113  8721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.004s	sys 0.006s
W20260709 13:03:19.748411  8721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.005s	sys 0.006s
I20260709 13:03:19.641922 11687 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:19.791517  8678 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:19.798519  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:19.804548  9383 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:19.762782  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.761741  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.787338 11276 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:19.739542 12105 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.000s	sys 0.011s
I20260709 13:03:19.786963  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:19.812603  8752 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:19.813102  8673 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:03:19.814544  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 126623us
I20260709 13:03:19.821205 11996 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:19.808246 12129 consensus_meta.cc:287] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.584s	user 0.013s	sys 0.002s
W20260709 13:03:19.823844  8827 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.003s	sys 0.006s
W20260709 13:03:19.836797  8827 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.004s	sys 0.006s
I20260709 13:03:19.742775 12130 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:19.826858  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:03:19.836306  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 487421us
I20260709 13:03:19.843479  8673 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:19.841567  9385 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:19.734004 12101 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:19.859526  8645 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:19.759421 12126 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:19.863659  9381 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 13:03:19.872287  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 309362us
I20260709 13:03:19.855279 12123 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:19.877002  9382 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:19.877002 11996 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:19.869134 12109 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:19.924410  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:19.969549  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:19.970773  8673 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:03:19.975006  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 620744us
W20260709 13:03:20.002808  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:20.003647  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.004082 12078 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.004792 12078 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:20.007218  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 167575us
W20260709 13:03:20.008268 12194 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.811s	user 0.004s	sys 0.008s
I20260709 13:03:20.008337  8645 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:20.010106  8645 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:19.903677 11917 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
W20260709 13:03:20.007440 12205 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
W20260709 13:03:20.014775  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 103431us
I20260709 13:03:20.014995  8794 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:03:20.014384 11953 consensus_meta.cc:287] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.593s	user 0.014s	sys 0.000s
I20260709 13:03:20.029471  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 555) took 1251 ms. Trace:
I20260709 13:03:20.004797 11908 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:19.928323 12119 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:20.016469 12177 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:20.065588  9352 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:20.039028  8672 rpcz_store.cc:276] 0709 13:03:18.768981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.770277 (+  1296us) service_pool.cc:224] Handling call
0709 13:03:18.770770 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.771265 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.771297 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.772063 (+   766us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.790264 (+ 18201us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:18.790311 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.791426 (+  1115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.007494 (+1216068us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:20.011718 (+  4224us) raft_consensus.cc:1685] finished
0709 13:03:20.011742 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.020071 (+  8329us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1596,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":166,"spinlock_wait_cycles":33774464,"thread_start_us":17660,"threads_started":1,"wal-append.queue_time_us":161163}
I20260709 13:03:20.079567 11971 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:20.080430 11971 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.082974 12081 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:20.083940  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 558) took 1466 ms. Trace:
I20260709 13:03:20.085117  8793 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:20.085712 12213 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:03:20.087100 12109 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:20.093232 12208 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:20.094100 12208 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.059942 11996 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:20.084167 12156 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:20.095305 12225 raft_consensus.cc:3055] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:20.100124  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 126136us
W20260709 13:03:20.101126  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 124932us
I20260709 13:03:20.102996  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:20.103884  8756 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:20.192545  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.067529 11687 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:20.084797  8755 rpcz_store.cc:276] 0709 13:03:18.617807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.858045 (+240238us) service_pool.cc:224] Handling call
0709 13:03:18.858590 (+   545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.054907 (+196317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.054976 (+    69us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.120176 (+ 65200us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.674962 (+554786us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:19.675022 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.675118 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.083576 (+408458us) raft_consensus.cc:1685] finished
0709 13:03:20.083610 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.083814 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13999,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":8833,"thread_start_us":468401,"threads_started":1,"wal-append.queue_time_us":644055}
I20260709 13:03:20.083762 12218 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:20.083892 12081 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:20.261615  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:20.284579  8747 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:20.048354 11953 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:20.208653 12079 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.634s	user 0.012s	sys 0.000s
I20260709 13:03:20.266417  9383 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
W20260709 13:03:20.267855  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55170.6us
I20260709 13:03:20.076905 12123 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:20.286176 11996 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:20.097127 12177 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.341027 11996 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.342355  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:20.343247  8677 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:20.344168 11996 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:20.345916  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 248714us
I20260709 13:03:20.287801 12078 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:20.100260 12225 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:20.286051 12210 log.cc:927] Time spent T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.396s	user 0.000s	sys 0.002s
I20260709 13:03:20.236943 12218 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:20.328896  8647 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.000s	sys 0.020s
W20260709 13:03:20.370245  8647 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.001s	sys 0.021s
W20260709 13:03:20.370664  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 794955us
W20260709 13:03:20.350374  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 48596.3us
I20260709 13:03:20.352905  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:20.353140  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:20.208490 12156 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:20.304061 12140 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:20.344796 11958 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.296936 12101 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:20.360443  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:20.364807 12085 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:20.305513 11687 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:20.372062  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:20.365242  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:20.298111 12126 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:20.373591 12218 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:20.336347 11666 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:20.377323  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 606) took 1515 ms. Trace:
I20260709 13:03:20.307371 12123 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.390389  8828 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:20.313884 11908 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:20.386328  8824 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:20.367925 12078 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:20.381001 12080 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.268s	user 0.013s	sys 0.002s
W20260709 13:03:20.293138  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 73672.8us
I20260709 13:03:20.275831 11276 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:20.447297 11276 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:20.397801  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:20.401556  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 48500.8us
W20260709 13:03:20.368676 11914 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.206s	user 0.008s	sys 0.008s
W20260709 13:03:20.391968  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 370229us
I20260709 13:03:20.424101  8672 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 11 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 13:03:20.419967 12123 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:20.375597 11971 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:20.391076 12128 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:20.400609 12140 raft_consensus.cc:697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:20.311973 12079 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.395578  8827 rpcz_store.cc:276] 0709 13:03:18.855525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.858743 (+  3218us) service_pool.cc:224] Handling call
0709 13:03:18.921623 (+ 62880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.937668 (+ 16045us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:19.837332 (+899664us) log.cc:844] Serialized 14 byte log entry
0709 13:03:20.370099 (+532767us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:20.370155 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:20.370206 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.370277 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.370431 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":598979,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":3791,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":9703,"thread_start_us":1254722,"threads_started":2}
I20260709 13:03:20.492395  8823 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:20.398887 12101 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:20.521342  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:20.528050  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 156095us
I20260709 13:03:20.527352 12225 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.527693 12101 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.493640 11666 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:20.535878 12234 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:03:20.552145 12234 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:20.553642 12234 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:20.553869 12168 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 2.217s	user 0.005s	sys 0.009s
W20260709 13:03:20.556102 12227 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:20.557876  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1768) took 1109 ms. Trace:
I20260709 13:03:20.558429  8676 rpcz_store.cc:276] 0709 13:03:19.448272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.448451 (+   179us) service_pool.cc:224] Handling call
0709 13:03:19.468224 (+ 19773us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.496096 (+ 27872us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.496148 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.542278 (+ 46130us) log.cc:844] Serialized 26 byte log entry
0709 13:03:20.002299 (+460021us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:20.002355 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.002448 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.557561 (+555113us) raft_consensus.cc:1685] finished
0709 13:03:20.557590 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.557791 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31216,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":133,"thread_start_us":430224,"threads_started":1,"wal-append.queue_time_us":776533}
I20260709 13:03:20.426646 11958 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:20.561455  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1766) took 1118 ms. Trace:
I20260709 13:03:20.560806 12081 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:20.561872  8674 rpcz_store.cc:276] 0709 13:03:19.442802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.442930 (+   128us) service_pool.cc:224] Handling call
0709 13:03:19.463812 (+ 20882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.627893 (+164081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.561157 (+933264us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54001f9854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:20.561382 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375862528}
I20260709 13:03:20.452323 11908 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:20.564846 11908 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:20.569650 11276 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:20.570758 12206 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.662s	user 0.003s	sys 0.000s
I20260709 13:03:20.571167 12080 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:20.574249 12119 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:20.575693 11953 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:20.576311  8647 leader_election.cc:336] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.713s (SENT)
W20260709 13:03:20.552296  8830 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1788) took 3172 ms (client timeout 1854 ms). Trace:
W20260709 13:03:20.576990  8830 rpcz_store.cc:269] 0709 13:03:17.379846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.382713 (+  2867us) service_pool.cc:224] Handling call
0709 13:03:20.551961 (+3169248us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b54002689d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:20.552164 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3662385408}
I20260709 13:03:20.577837 12079 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:20.528484 11917 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:20.578630 11917 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.580222 11848 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:20.580996 11917 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:20.582288 12221 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 13:03:20.583920  8748 raft_consensus.cc:2368] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
W20260709 13:03:20.584106  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 56391.5us
I20260709 13:03:20.584663 12207 raft_consensus.cc:3055] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:20.585125 12207 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:20.585373  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1887) took 1085 ms. Trace:
I20260709 13:03:20.543237 11977 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:20.577404 11958 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:20.586592 11977 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:20.593540 11977 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:20.544324 12225 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:20.596475 12208 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:20.401844 11687 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.599257 11687 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:20.544389  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 155103us
I20260709 13:03:20.600971  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:20.601504 12156 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:20.546319 12101 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:20.602900 12217 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.267s	user 0.002s	sys 0.000s
I20260709 13:03:20.441325 12078 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:20.603461 12001 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:20.604156 12001 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.604650 12001 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:20.605477 12078 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:20.544952 12220 log.cc:927] Time spent T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
W20260709 13:03:20.548965 12184 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 1.609s	user 0.008s	sys 0.013s
I20260709 13:03:20.609346  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1883) took 1131 ms. Trace:
I20260709 13:03:20.610584  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:20.586691  8754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1879) took 1809 ms (client timeout 2399 ms). Trace:
W20260709 13:03:20.611425  8754 rpcz_store.cc:269] 0709 13:03:18.777591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.843379 (+ 65788us) service_pool.cc:224] Handling call
0709 13:03:20.586500 (+1743121us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b540034a154. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:20.586629 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136798464}
I20260709 13:03:20.614080  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:20.588634  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 559) took 1053 ms. Trace:
I20260709 13:03:20.543646  8829 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:20.609874  8751 rpcz_store.cc:276] 0709 13:03:19.477651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.477789 (+   138us) service_pool.cc:224] Handling call
0709 13:03:19.490103 (+ 12314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.558928 (+ 68825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.558960 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.559482 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.858519 (+299037us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:19.858582 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.862917 (+  4335us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.609049 (+746132us) raft_consensus.cc:1685] finished
0709 13:03:20.609083 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.609263 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18510,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":157,"thread_start_us":298521,"threads_started":1,"wal-append.queue_time_us":503904}
I20260709 13:03:20.599983  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1886) took 1101 ms. Trace:
I20260709 13:03:20.608302  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 560) took 1069 ms. Trace:
I20260709 13:03:20.586808 12128 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.586967 11953 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:20.618562 12128 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:20.619107 11953 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.619545 12140 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.621117 12128 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:20.622054 11953 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:20.585800  8748 rpcz_store.cc:276] 0709 13:03:19.500223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.500393 (+   170us) service_pool.cc:224] Handling call
0709 13:03:20.585137 (+1084744us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400157cd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:20.585286 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2477694848}
I20260709 13:03:20.617128  8752 rpcz_store.cc:276] 0709 13:03:19.499808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.506560 (+  6752us) service_pool.cc:224] Handling call
0709 13:03:20.599727 (+1093167us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b540034ba54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:20.599900 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":527828224}
I20260709 13:03:20.617782  8675 rpcz_store.cc:276] 0709 13:03:19.538673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.572405 (+ 33732us) service_pool.cc:224] Handling call
0709 13:03:19.586127 (+ 13722us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.632647 (+ 46520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.632685 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.633318 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.870635 (+237317us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:19.870676 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.588168 (+717492us) spinlock_profiling.cc:243] Waited 46 ms on lock 0x7b54002576d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:20.588217 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.601903 (+ 13686us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:20.608065 (+  6162us) raft_consensus.cc:1685] finished
0709 13:03:20.608091 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.608233 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8025,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":109,"spinlock_wait_cycles":182943744,"thread_start_us":236801,"threads_started":1,"wal-append.queue_time_us":434514}
I20260709 13:03:20.623944 12177 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:20.548810 11666 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:20.538568  8821 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:20.625861 12105 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.625964 11977 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:20.627280  8796 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:20.590378  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:20.629377 12132 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.008s	user 0.013s	sys 0.000s
I20260709 13:03:20.630421  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:20.631464 12028 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:20.632949  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:20.633998 12028 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:20.634339  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:20.634562 12028 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.635759 12028 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:20.615384  8681 rpcz_store.cc:276] 0709 13:03:19.535374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.541206 (+  5832us) service_pool.cc:224] Handling call
0709 13:03:19.578170 (+ 36964us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.624992 (+ 46822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.625026 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:19.625662 (+   636us) log.cc:844] Serialized 26 byte log entry
0709 13:03:19.911091 (+285429us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:19.911156 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.911248 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.588358 (+677110us) raft_consensus.cc:1685] finished
0709 13:03:20.588383 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.588564 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21279,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":5416,"thread_start_us":284011,"threads_started":1,"wal-append.queue_time_us":737721}
I20260709 13:03:20.503160 12219 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.637023  8645 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:20.637079 12219 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:20.626430 12105 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:20.641700  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 137556us
I20260709 13:03:20.644927 12177 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:20.592183  8756 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
W20260709 13:03:20.642489  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 102846us
I20260709 13:03:20.629686  8796 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:20.673157  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 43811.6us
I20260709 13:03:20.629727  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:20.667304  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 536162us
I20260709 13:03:20.669582  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:20.676491  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:20.629961 11666 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:20.688711  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 614) took 1378 ms. Trace:
I20260709 13:03:20.664562 11953 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:20.722600  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:03:20.722890  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 93267.4us
I20260709 13:03:20.723778  8821 rpcz_store.cc:276] 0709 13:03:19.310874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.317196 (+  6322us) service_pool.cc:224] Handling call
0709 13:03:19.362158 (+ 44962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.625518 (+1263360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.688326 (+ 62808us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54002292d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:20.688613 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2604244224}
I20260709 13:03:20.731117  9385 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:20.735635  8719 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:20.738055  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:20.680243 11996 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:20.747843  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 110953us
I20260709 13:03:20.714111 12165 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:20.706298 11666 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:20.688123 12177 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:20.760679  9384 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:03:20.755328  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:20.774379  8720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.007s	sys 0.005s
I20260709 13:03:20.774544  8829 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:20.705646 12128 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:03:20.763695  9355 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.019s	sys 0.031s
I20260709 13:03:20.761245  8747 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:20.727527 11953 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:20.746953  8679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.002s	sys 0.007s
I20260709 13:03:20.766772  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:20.775008  8720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.008s	sys 0.005s
W20260709 13:03:20.775534  9355 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.019s	sys 0.031s
I20260709 13:03:20.741605 11917 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:20.784027  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:20.783362  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.554423 12234 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:20.781003  8679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.003s	sys 0.007s
I20260709 13:03:20.777225  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:20.770723 11823 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:03:20.793917 12095 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:03:20.586145 12207 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.795477 12207 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:20.796373  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:20.796923 12147 consensus_meta.cc:287] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.063s	user 0.007s	sys 0.010s
I20260709 13:03:20.797703  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:20.730383 12101 consensus_queue.cc:1059] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:03:20.805189 12101 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:20.755532 12113 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:03:20.808457 12113 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:20.809058 12113 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:20.769747 11666 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:20.742674 12078 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:03:20.777307 12141 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.816244 12141 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.783839 11908 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:20.823290 11908 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:20.823823 11908 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:20.789022  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:20.830816 12149 consensus_meta.cc:287] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.135s	user 0.016s	sys 0.000s
I20260709 13:03:20.747608 11687 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:20.775640 12128 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:20.832166  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1884) took 1354 ms. Trace:
I20260709 13:03:20.833016  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:20.834318  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:20.835660  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:20.836850  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:20.832648  8749 rpcz_store.cc:276] 0709 13:03:19.478001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.483051 (+  5050us) service_pool.cc:224] Handling call
0709 13:03:19.502276 (+ 19225us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:19.545491 (+ 43215us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:19.908926 (+363435us) log.cc:844] Serialized 14 byte log entry
0709 13:03:20.404413 (+495487us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:20.497394 (+ 92981us) log.cc:844] Serialized 26 byte log entry
0709 13:03:20.497553 (+   159us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:20.503920 (+  6367us) log.cc:844] Serialized 14 byte log entry
0709 13:03:20.510101 (+  6181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.510185 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.831906 (+321721us) raft_consensus.cc:1685] finished
0709 13:03:20.831931 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.832100 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14703,"prepare.queue_time_us":780073,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":4372,"raft.queue_time_us":59717,"raft.run_cpu_time_us":381,"raft.run_wall_time_us":26812,"thread_start_us":736468,"threads_started":2,"wal-append.queue_time_us":865007}
I20260709 13:03:20.837172 11971 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:20.837661  8827 raft_consensus.cc:2393] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:20.775831 11953 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:20.838706  9388 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.013s	sys 0.016s
W20260709 13:03:20.839157  9388 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.013s	sys 0.016s
I20260709 13:03:20.839735  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:20.840461  9383 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:20.841790  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:20.842825  8676 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 6: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:20.843988  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:20.844206  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:20.844497  8821 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:20.844842  8828 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:20.845708  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:20.846185  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 78896.3us
I20260709 13:03:20.788938  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:20.847044  8823 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:20.847981 12219 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:20.848196  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:20.832959 12128 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.849092  8823 raft_consensus.cc:2410] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:20.850078  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:20.836705 12083 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:20.852161  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:20.853157  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:20.832381  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 72869.1us
I20260709 13:03:20.853830  8829 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:03:20.853941  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 64915.4us
I20260709 13:03:20.847050  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:20.762049 12165 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:20.845852  8674 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:20.845631  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:20.861375  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:20.862131  8823 raft_consensus.cc:2410] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:03:20.862167  8829 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:20.766188 12112 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.239s	user 0.012s	sys 0.022s
I20260709 13:03:20.855195  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:20.868716 11996 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:20.888878  8824 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
W20260709 13:03:20.896006  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.896114  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:20.899082  9384 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 581) took 2362 ms (client timeout 1621 ms). Trace:
W20260709 13:03:20.892434  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 56373.9us
W20260709 13:03:20.899693  9384 rpcz_store.cc:269] 0709 13:03:18.531300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.538417 (+  7117us) service_pool.cc:224] Handling call
0709 13:03:20.893132 (+2354715us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b54006275d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:20.893317 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4554770176}
W20260709 13:03:20.893548 12112 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.379s	user 0.013s	sys 0.022s
I20260709 13:03:20.864724 12165 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:20.853329 12218 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:20.908680  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:20.915997  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:20.918454  9353 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:20.918241  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 63126.5us
I20260709 13:03:20.918531  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:20.898563 12001 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:20.938994  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:20.939179  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:20.938652  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:20.942929  8827 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:20.943562  8823 raft_consensus.cc:2410] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:20.911131 12156 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:20.917079 12247 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:20.916311 12002 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:20.810427 12113 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:20.962239  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.977506  9353 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:20.966171  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:20.966288  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:20.956629 12112 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:20.983445 12156 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:20.931346 12129 consensus_queue.cc:1059] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
W20260709 13:03:21.001394  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 73479.4us
W20260709 13:03:21.003559  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 64105us
I20260709 13:03:20.938233 12095 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:03:20.990543  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:21.008858  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:20.962733 12240 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:21.012239  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:21.018028  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:21.018824  9383 raft_consensus.cc:2376] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5. Re-sending same reply.
I20260709 13:03:21.020308  8748 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:03:21.029791  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:21.031977  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:21.032811  8825 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:20.916644 12177 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:21.039047  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.041932  8750 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:20.997017 11687 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.005753 12147 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.003582 11934 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:21.062076  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 135554us
I20260709 13:03:21.062435  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:21.063838  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:21.005223 12078 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.948853  9384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:03:21.065634  9384 raft_consensus.cc:606] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:21.010103 12111 consensus_queue.cc:1059] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 13:03:21.055591 12251 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 13:03:21.056473 12228 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:03:21.058811 12248 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.059885  8748 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:03:21.060105  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 82707.7us
I20260709 13:03:21.060493 11711 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:21.013939 12240 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.061339 12179 raft_consensus.cc:3055] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:21.042580  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:21.067607  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:21.073647  8827 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:21.022774 12112 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:21.076607  8674 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:21.159807  8823 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:21.032003 12113 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:20.995451 12002 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.040544 12156 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:21.134752  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 63283us
I20260709 13:03:21.089758  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.075786 11934 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.056537 12234 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:20.977715 12247 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.074503 12149 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.165555  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:21.171792 12217 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:03:21.075297 12147 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:21.146773 12179 raft_consensus.cc:740] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:21.151857 12240 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:21.200335 12246 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:03:21.201048  8826 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:21.176107 11711 raft_consensus.cc:697] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:21.201256  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.141961 12248 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:21.090137 11687 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:21.080147 12078 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:21.205106  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.180348 12225 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:21.221057  8748 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:21.213637 12105 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:21.224536 12234 raft_consensus.cc:3055] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:03:21.221685  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 157922us
W20260709 13:03:21.219352 12123 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.699s	user 0.016s	sys 0.000s
I20260709 13:03:21.207259 12218 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:21.259130  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.217160 12001 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:21.261300  8753 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:21.180883 11934 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:21.226380 12096 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:21.240109 12002 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:21.265298  8645 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:21.271871  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 269681us
I20260709 13:03:21.285204  9382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "142891c077a34ba289adf90a99aee612"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:03:21.286829  9382 raft_consensus.cc:606] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:21.287241  9382 raft_consensus.cc:614] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:21.188786 12149 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:21.236091 12234 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:21.302838  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:21.303917  8826 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:21.308524  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:21.267006 12105 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:21.319564  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.323556 12105 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.324008 12234 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.324605  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.325553 12234 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:21.319772 11276 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:21.326930 11276 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:21.315032  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:21.328951 11977 consensus_meta.cc:287] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.702s	user 0.011s	sys 0.004s
I20260709 13:03:21.330322  8647 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:21.314554 12002 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.331305 12002 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:21.332422  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:21.333810 12002 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:21.262285 12238 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 13:03:21.337651  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:21.288363 12253 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 13:03:21.335551  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.326133  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.333897  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:21.344416  8748 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:03:21.285128 12250 log.cc:927] Time spent T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
I20260709 13:03:21.242874 12225 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.354588  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:21.356057  8826 raft_consensus.cc:2410] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:03:21.260993 12123 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:21.326202 12028 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.690s	user 0.006s	sys 0.006s
W20260709 13:03:21.336805 12207 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.541s	user 0.013s	sys 0.000s
I20260709 13:03:21.319603 12218 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:21.318748 12267 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:21.372059  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:21.378227  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.379266  8748 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:21.352485 12111 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:21.240830 12112 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.263520 12179 consensus_queue.cc:260] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.409557  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:21.410341  8827 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:03:21.417404 12252 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 13:03:21.417784  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:21.418859  8827 raft_consensus.cc:2410] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:03:21.419014 12256 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:03:21.419920  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:21.420248 12255 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:03:21.421677  8828 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:21.407652  8722 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:21.425038  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:21.395877 12254 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 13:03:21.357865 11977 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.378360 12259 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:21.404852  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.371529 12262 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:21.382786  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:21.436851  8826 raft_consensus.cc:2410] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:03:21.436874  8829 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.541s	user 0.008s	sys 0.004s
I20260709 13:03:21.432086  8756 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:21.369132 12028 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:21.437647  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:21.436738  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:21.441207  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:21.441817  8748 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:21.431995  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 98179.3us
I20260709 13:03:21.437683  8829 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:21.384418 12218 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.439845  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:21.443547  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:21.452826  8675 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:21.443585  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:21.438486  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:21.441857  8826 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:21.445870 12273 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:21.443040 11977 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.448664 12147 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:21.346249 11711 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.451151 12001 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:21.429924 12267 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:21.454305  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 46730.2us
I20260709 13:03:21.455581  8673 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:21.458204  8827 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:21.510685 12053 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:21.510484  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:21.430439 12269 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:21.501818 12247 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:21.450194 12240 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:21.512326  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 66948.2us
I20260709 13:03:21.503453  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:21.511580 11908 consensus_meta.cc:287] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.687s	user 0.015s	sys 0.000s
I20260709 13:03:21.553793  8826 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:21.430382 12111 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.564913 12149 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:21.568996 11908 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:21.571527  8830 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:21.574141  8828 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:21.512584 12141 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:21.505231  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:21.504388  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:21.500880 12273 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:21.511641  9382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:03:21.520802  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:21.460732  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:21.541213  8754 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:21.532219 12259 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.543395 12269 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:21.546064  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:21.547307 11996 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.636s	user 0.010s	sys 0.003s
I20260709 13:03:21.547628  8645 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:21.548583  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:21.437642 12262 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.556392 12248 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:21.545902 11687 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:21.557428  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:21.558861 12240 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:21.559304  8827 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:21.559604  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:21.514961 12274 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.633181  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:21.634649 12274 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.564864 12085 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:21.635839 12085 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:21.545873 12053 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.637060 12274 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:21.637754 12265 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:03:21.639245 12053 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:21.637101 12085 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.629467  9382 raft_consensus.cc:606] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:21.624596  8675 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:03:21.595069 11953 consensus_meta.cc:287] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.733s	user 0.008s	sys 0.004s
I20260709 13:03:21.644747 11953 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.645690 12147 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.646343 12147 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:21.624470 11996 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:21.595157 12269 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:21.632720  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 85235.1us
I20260709 13:03:21.633123 12225 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:21.633500 12276 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:21.596272 12262 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:21.634019 12267 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.643460 12259 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:21.648361  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 95874.9us
I20260709 13:03:21.649178  8754 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:21.633451  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:21.664566  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:21.660475  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:21.668000  8645 leader_election.cc:400] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
I20260709 13:03:21.668708  8645 leader_election.cc:403] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:21.663050  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:21.665472  8676 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:21.670955  8678 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:21.653143 12085 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:21.680150  8675 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:21.675416  8828 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:21.677512  8827 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:03:21.552838 12126 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 13:03:21.651889 12078 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:21.656911 12276 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.701148 12225 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:21.701375 12276 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:21.701675 12225 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.702064 12225 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:21.682966 11908 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:21.703069 11908 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.703593 11908 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:21.703842  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:21.682857 12267 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:21.705224  8681 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:21.705689 12274 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:21.706252 12274 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:21.707249 12274 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:21.684115 11977 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:21.679534 11996 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:21.708997  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.711148  8755 raft_consensus.cc:2410] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:03:21.712319  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:21.713269  8755 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:21.735409  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:21.736227  8827 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:21.737174  9386 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:21.737391  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:21.738196  8827 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:21.716231 12165 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.781s	user 0.010s	sys 0.003s
I20260709 13:03:21.739439 12165 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.740623  9384 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:21.741304  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.743577 12128 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:21.685019 12259 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:21.685509  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:21.686697  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:21.751355  8754 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:03:21.751406  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 66024.5us
W20260709 13:03:21.700433  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 392002us
I20260709 13:03:21.742281  8752 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:21.752401  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:21.753443  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.753659  8645 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:21.724104  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:21.755405  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:21.727658  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:21.729967  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.725540  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:21.733372  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:21.717537  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:21.722887  9353 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:21.759248  8752 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:21.733042  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:21.744465 12165 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:21.731949  9382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "293a9dab5be3479a90d5803585682a00"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:33196
W20260709 13:03:21.748991  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.760201  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.733974  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:21.810509  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:21.811862  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 56564.8us
W20260709 13:03:21.820482  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.821105  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.820977  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 46287.6us
I20260709 13:03:21.822439  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:21.762813  8750 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:21.824115  9382 raft_consensus.cc:606] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:21.822301  8752 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
W20260709 13:03:21.744617 12257 log.cc:927] Time spent T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.551s	user 0.004s	sys 0.000s
W20260709 13:03:21.825037  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 71458.7us
W20260709 13:03:21.719278 12113 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.003s
I20260709 13:03:21.733101 12218 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:21.827767  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:21.725690 11666 consensus_meta.cc:287] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.915s	user 0.006s	sys 0.007s
W20260709 13:03:21.825392  8672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.035s	sys 0.038s
W20260709 13:03:21.836683  8672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.035s	sys 0.038s
I20260709 13:03:21.731719 12111 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:21.762285 12219 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:21.834448  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.838835  8755 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:21.716831 11953 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:21.733841 12028 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:21.811249 12147 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:21.762979 12001 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:21.759341 12096 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:21.853166  8645 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:21.731009 11908 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:21.732401 12267 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:21.884620  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 569) took 1044 ms. Trace:
I20260709 13:03:21.831138 12259 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:21.892385  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:21.759311 11711 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:21.911823  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:21.927843  8827 raft_consensus.cc:2376] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10. Re-sending same reply.
I20260709 13:03:21.895540 12219 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:21.831717 12014 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:21.893059  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.738965 12081 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 13:03:21.918236  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.831631 12277 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:03:21.919401  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 66367.1us
I20260709 13:03:21.718026 12123 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:21.924176  8755 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:21.920316  8672 rpcz_store.cc:276] 0709 13:03:20.840119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.848261 (+  8142us) service_pool.cc:224] Handling call
0709 13:03:20.848670 (+   409us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:20.849148 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:20.849179 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:20.849916 (+   737us) log.cc:844] Serialized 52 byte log entry
0709 13:03:21.022687 (+172771us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:21.127112 (+104425us) log.cc:844] Serialized 14 byte log entry
0709 13:03:21.450789 (+323677us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b48001002d8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:03:21.837404 (+386615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.837531 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:21.837605 (+    74us) raft_consensus.cc:1685] finished
0709 13:03:21.884291 (+ 46686us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.884521 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40519,"prepare.queue_time_us":33552,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":195973,"raft.run_cpu_time_us":2505,"raft.run_wall_time_us":116261,"spinlock_wait_cycles":11771136,"thread_start_us":888437,"threads_started":3,"wal-append.queue_time_us":392897}
I20260709 13:03:21.717092 11934 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.854795 12282 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 13:03:21.910949 12218 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:21.930308  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:21.831013 12273 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.911475 12001 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.943024  8754 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
W20260709 13:03:21.944245 12234 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.618s	user 0.015s	sys 0.001s
I20260709 13:03:21.945394  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:21.854059 11666 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:21.884519 12111 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.941021 12219 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:21.930269 12267 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:21.968523  8749 raft_consensus.cc:2410] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:21.951776  8827 raft_consensus.cc:2410] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:21.920337 12028 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:03:21.971045  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.973065  8680 raft_consensus.cc:2368] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8.
I20260709 13:03:21.939745 12014 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:21.972781  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.975354  8751 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
I20260709 13:03:21.975494  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.972133  8750 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:21.940444 11953 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:03:21.979542  8821 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
W20260709 13:03:21.977713  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 199554us
I20260709 13:03:21.964277 12001 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:21.963886 12218 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:21.993903  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.968873 12267 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.991402  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:21.978500 12014 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:22.008080  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:22.008350  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:22.008919  8673 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:22.009582  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:22.008080  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:22.009721  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:22.011086  8675 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:22.012167  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:21.965711 12111 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:22.013401  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:22.014000  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 562) took 2378 ms. Trace:
I20260709 13:03:22.000869  8827 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:03:21.994234 12277 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:21.988862 12275 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.020754  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1614) took 1130 ms. Trace:
I20260709 13:03:22.003010 12267 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:22.023285  8826 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:03:22.024183  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 570) took 1183 ms. Trace:
I20260709 13:03:22.024904  8674 raft_consensus.cc:1217] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[4.6-5.7]   Dedup: 4.6->[5.7-5.7]
I20260709 13:03:22.030681  8674 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: [4.6-5.7]
I20260709 13:03:21.992769  8645 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:22.028249  8827 rpcz_store.cc:276] 0709 13:03:20.890686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.987972 (+1097286us) service_pool.cc:224] Handling call
0709 13:03:22.000356 (+ 12384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.020468 (+ 20112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.020686 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:22.020992  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 274164us
I20260709 13:03:22.028997  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:22.033495  8747 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:22.027009  8755 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:22.030332  8826 pending_rounds.cc:85] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 4
I20260709 13:03:22.038060  8826 pending_rounds.cc:107] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:03:22.038313  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:22.018046  8679 rpcz_store.cc:276] 0709 13:03:19.625687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.625895 (+   208us) service_pool.cc:224] Handling call
0709 13:03:19.626353 (+   458us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:19.869845 (+243492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:19.869889 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:19.913844 (+ 43955us) log.cc:844] Serialized 52 byte log entry
0709 13:03:20.856748 (+942904us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:21.087921 (+231173us) log.cc:844] Serialized 14 byte log entry
0709 13:03:21.348022 (+260101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.995191 (+647169us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b54002a8fd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:21.995276 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:21.995348 (+    72us) raft_consensus.cc:1685] finished
0709 13:03:22.003365 (+  8017us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.003633 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10492,"prepare.queue_time_us":45243,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":7166,"raft.queue_time_us":234013,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":23266,"spinlock_wait_cycles":47362432,"thread_start_us":1455093,"threads_started":4,"wal-append.queue_time_us":887641}
I20260709 13:03:22.039681  8751 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:22.038416  8826 raft_consensus.cc:2889] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:22.035174  8794 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:22.021826 11996 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
W20260709 13:03:22.036989  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 44406us
I20260709 13:03:22.027087 12277 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.048220  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:22.016036 12262 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.049846  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:22.052565  8754 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:22.053731  8645 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:22.053968  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:22.047079  8673 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
W20260709 13:03:22.050153  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 213642us
I20260709 13:03:22.050971  8827 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:22.051994  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:22.030405  8677 rpcz_store.cc:276] 0709 13:03:20.840436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.850486 (+ 10050us) service_pool.cc:224] Handling call
0709 13:03:20.855401 (+  4915us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:20.869335 (+ 13934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:20.869366 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:20.870039 (+   673us) log.cc:844] Serialized 52 byte log entry
0709 13:03:20.985892 (+115853us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:21.018290 (+ 32398us) log.cc:844] Serialized 14 byte log entry
0709 13:03:21.046847 (+ 28557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.046940 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:21.743835 (+696895us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:22.014437 (+270602us) raft_consensus.cc:1685] finished
0709 13:03:22.023891 (+  9454us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.024104 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1285,"prepare.queue_time_us":64023,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":213687,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":13984,"spinlock_wait_cycles":63798272,"thread_start_us":143619,"threads_started":2,"wal-append.queue_time_us":209275}
I20260709 13:03:22.063149 11996 raft_consensus.cc:2764] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:03:22.064478 11996 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:22.067178  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:22.033250 12275 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.068161  8747 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:22.068588  8752 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:22.039219 12274 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:22.081729 12274 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.049800 12096 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:22.133358  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:22.134500  8754 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:22.055950 12277 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:22.119884 12081 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.146692  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1608) took 1316 ms. Trace:
I20260709 13:03:22.104945 12096 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:22.152638  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:22.153551  8754 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:22.147223  8823 rpcz_store.cc:276] 0709 13:03:20.830330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.957770 (+127440us) service_pool.cc:224] Handling call
0709 13:03:21.148108 (+190338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.160370 (+ 12262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.146393 (+986023us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b540032e950. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:22.146579 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":382528896}
I20260709 13:03:22.155871  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:22.159032  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 465813us
W20260709 13:03:22.168439  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.171353  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:22.173552  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:22.173904  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:22.173467  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:22.156262 12292 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:22.177263  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 125343us
I20260709 13:03:22.144362 11934 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:22.183358  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:22.120568 12218 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.175628 12128 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:22.213900  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:22.207193  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:22.214874  8826 raft_consensus.cc:2393] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:22.169129 12291 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:22.241741  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2280) took 1067 ms. Trace:
W20260709 13:03:22.242427  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 57173.4us
I20260709 13:03:22.242954  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:22.244477  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:22.245610  8794 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:22.242235  8753 rpcz_store.cc:276] 0709 13:03:21.174400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.180651 (+  6251us) service_pool.cc:224] Handling call
0709 13:03:22.241522 (+1060871us) spinlock_profiling.cc:243] Waited 39.3 ms on lock 0x7b54001022d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:22.241659 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90385152}
I20260709 13:03:22.248803  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:22.202414 11666 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:22.180824 12123 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:22.216195  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:22.220654 12085 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.536s	user 0.006s	sys 0.006s
I20260709 13:03:22.220885 11908 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:22.221489  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:22.222858 12096 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.205631  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1806) took 1360 ms. Trace:
W20260709 13:03:22.226680 12288 log.cc:927] Time spent T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:03:22.230479  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:03:22.230443 12225 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.528s	user 0.014s	sys 0.000s
I20260709 13:03:22.155372 12081 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.156051 12014 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:22.236910 12285 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:03:22.243711  8826 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:22.258654  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:22.263564  8823 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:22.225535 12302 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:22.274343  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:22.215068 12128 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:22.310483 12128 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:22.288201  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:22.312157  8754 raft_consensus.cc:2368] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11.
I20260709 13:03:22.312258 12081 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:22.312963 12081 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:22.267976 12269 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.594s	user 0.007s	sys 0.007s
I20260709 13:03:22.315057 12269 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.315689 12305 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:22.316310 12305 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:22.255941 12111 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:22.306595  8827 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:22.314383  8754 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:03:22.189769 12292 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.271848 11908 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:22.294114 12274 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:22.248991 12291 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:22.329721  8823 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:03:22.334980  8753 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:03:22.334446  8754 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:03:22.335716  8748 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:22.336876  8754 pending_rounds.cc:85] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 2
I20260709 13:03:22.293867 12225 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.337236  8754 pending_rounds.cc:107] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:03:22.343561  8754 raft_consensus.cc:2889] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:22.345075  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1813) took 1432 ms (client timeout 1781 ms). Trace:
I20260709 13:03:22.292265  8825 rpcz_store.cc:276] 0709 13:03:20.844870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.712459 (+867589us) service_pool.cc:224] Handling call
0709 13:03:21.712802 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.713086 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:21.956422 (+243336us) log.cc:844] Serialized 14 byte log entry
0709 13:03:22.198893 (+242471us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:22.199117 (+   224us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:22.199331 (+   214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.199704 (+   373us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.205535 (+  5831us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":396566,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":2339,"thread_start_us":452400,"threads_started":3}
W20260709 13:03:22.345582  8821 rpcz_store.cc:269] 0709 13:03:20.902727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.904476 (+  1749us) service_pool.cc:224] Handling call
0709 13:03:22.335059 (+1430583us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54003dead4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:22.335180 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2434602880}
I20260709 13:03:22.335661  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2278) took 1250 ms. Trace:
I20260709 13:03:22.351223  8753 rpcz_store.cc:276] 0709 13:03:21.085430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.321677 (+1236247us) service_pool.cc:224] Handling call
0709 13:03:22.334575 (+ 12898us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.335387 (+   812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.335599 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:22.353461  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 104755us
I20260709 13:03:22.322888 12123 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:22.354472  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:22.323079 12262 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:22.355691  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 299211us
I20260709 13:03:22.355993  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:22.356917  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 303291us
W20260709 13:03:22.331507  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 138890us
W20260709 13:03:22.362738 12287 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:03:22.359092  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:22.373687  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:22.316872 12294 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:03:22.299866 12301 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:03:22.382663  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1799) took 1540 ms. Trace:
W20260709 13:03:22.366839 12261 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.160s	user 0.005s	sys 0.000s
I20260709 13:03:22.376783  8753 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:22.384989  8645 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:22.399091  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2284) took 1034 ms. Trace:
I20260709 13:03:22.399734  8753 rpcz_store.cc:276] 0709 13:03:21.364458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.352386 (+987928us) service_pool.cc:224] Handling call
0709 13:03:22.376422 (+ 24036us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.398319 (+ 21897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.398991 (+   672us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.383164  8828 rpcz_store.cc:276] 0709 13:03:20.842439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.681451 (+839012us) service_pool.cc:224] Handling call
0709 13:03:21.706683 (+ 25232us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.707172 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.707211 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.707832 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 13:03:21.973407 (+265575us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:21.973507 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.973624 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.382321 (+408697us) raft_consensus.cc:1685] finished
0709 13:03:22.382352 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.382561 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2286,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":258005,"threads_started":1,"wal-append.queue_time_us":356659}
I20260709 13:03:22.379345 12274 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:22.382418 12269 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:22.401990 12262 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:22.464956  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 103414us
I20260709 13:03:22.473049  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:22.402323 11934 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:22.474892  8753 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:22.434056 12302 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:22.439332 12274 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.426867 12297 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:22.492954  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 108049us
I20260709 13:03:22.497058  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:22.492293  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2288) took 1072 ms. Trace:
I20260709 13:03:22.503175  8753 rpcz_store.cc:276] 0709 13:03:21.419848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.400853 (+981005us) service_pool.cc:224] Handling call
0709 13:03:22.451692 (+ 50839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.491871 (+ 40179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.492195 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:22.506037  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 150122us
I20260709 13:03:22.419776 11908 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:22.496245  8793 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:22.490391 12001 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.502449 12306 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:22.533507 12276 consensus_meta.cc:287] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.832s	user 0.015s	sys 0.001s
I20260709 13:03:22.514137 12275 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:22.503026 12274 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:22.428316 12305 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.492141 11934 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:22.541141 12001 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:22.601154 12291 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:22.501860 11711 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:03:22.605155 11711 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:22.605748 11711 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:22.518250 12014 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:22.611143 12292 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:22.612093 12277 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.614959 12225 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:22.615754 12277 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:22.616240 11711 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:22.617805 11711 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:22.515503 12299 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:22.621481 12299 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.623670  8752 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:22.625545  8752 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:22.629724  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2210) took 1207 ms. Trace:
I20260709 13:03:22.623445 12299 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:22.630270  8674 rpcz_store.cc:276] 0709 13:03:21.422664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.423062 (+   398us) service_pool.cc:224] Handling call
0709 13:03:22.024663 (+601601us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:22.031203 (+  6540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.629454 (+598251us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b54003a25d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:22.629618 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1382609280}
I20260709 13:03:22.633054  8752 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:22.634756 11934 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:22.590415 12276 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:22.641036 12259 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.715s	user 0.013s	sys 0.002s
I20260709 13:03:22.660429  8752 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:22.655678 12259 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:22.663978 12267 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.626s	user 0.013s	sys 0.001s
I20260709 13:03:22.667919 12128 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.668406 12299 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:22.669559 12268 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 1.166s	user 0.006s	sys 0.000s
W20260709 13:03:22.659878 12219 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.684s	user 0.015s	sys 0.000s
I20260709 13:03:22.671351 12219 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:22.670552 12128 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:22.675426 12219 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:22.678929 12081 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:22.681871 12274 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:22.664532 12267 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.684651 12276 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:22.688331 12274 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:22.689628 12267 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:22.718565 11276 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:22.858944 12105 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:22.903178  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1606) took 2073 ms. Trace:
I20260709 13:03:22.904992  8824 rpcz_store.cc:276] 0709 13:03:20.829634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.957554 (+127920us) service_pool.cc:224] Handling call
0709 13:03:20.958093 (+   539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.004656 (+ 46563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.004716 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.030006 (+ 25290us) log.cc:844] Serialized 26 byte log entry
0709 13:03:21.338504 (+308498us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:21.338548 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.338651 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.660512 (+1321861us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:22.902461 (+241949us) raft_consensus.cc:1685] finished
0709 13:03:22.902500 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.903065 (+   565us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7897,"raft.queue_time_us":21798,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":8271,"spinlock_wait_cycles":42772736,"thread_start_us":188473,"threads_started":1,"wal-append.queue_time_us":348168}
I20260709 13:03:22.908653  8824 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:22.909597  8753 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:22.911386  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2289) took 1491 ms. Trace:
I20260709 13:03:22.911832  8753 rpcz_store.cc:276] 0709 13:03:21.420199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.533247 (+1113048us) service_pool.cc:224] Handling call
0709 13:03:22.909418 (+376171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.911122 (+  1704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.911321 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.912698  8821 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:22.913477  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 581) took 1090 ms. Trace:
I20260709 13:03:22.914114  8753 rpcz_store.cc:276] 0709 13:03:21.822821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.912660 (+1089839us) service_pool.cc:224] Handling call
0709 13:03:22.912989 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.913183 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.913207 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:22.913249 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:22.913280 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.913327 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.913424 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.916810  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 582) took 1093 ms. Trace:
I20260709 13:03:22.917198  8753 rpcz_store.cc:276] 0709 13:03:21.823065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.915239 (+1092174us) service_pool.cc:224] Handling call
0709 13:03:22.915824 (+   585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.916033 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.916073 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:22.916118 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:22.916159 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.916208 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.916739 (+   531us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.919377  8753 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:22.920091  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 585) took 1094 ms. Trace:
I20260709 13:03:22.920540  8753 rpcz_store.cc:276] 0709 13:03:21.825516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.918356 (+1092840us) service_pool.cc:224] Handling call
0709 13:03:22.918900 (+   544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.919793 (+   893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.920015 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.922292  8753 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:03:22.923283  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2306) took 1092 ms. Trace:
I20260709 13:03:22.923696  8753 rpcz_store.cc:276] 0709 13:03:21.830606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.921458 (+1090852us) service_pool.cc:224] Handling call
0709 13:03:22.921879 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.922911 (+  1032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.923198 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:22.938791  9389 consensus_meta.cc:287] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.652s	user 0.012s	sys 0.004s
I20260709 13:03:22.939460  9389 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
W20260709 13:03:22.944015 12293 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.580s	user 0.000s	sys 0.005s
W20260709 13:03:22.947669  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.948446 12269 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:22.949065 12269 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:22.949664  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1903) took 1605 ms. Trace:
I20260709 13:03:22.950073  8748 rpcz_store.cc:276] 0709 13:03:21.344131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.336774 (+992643us) service_pool.cc:224] Handling call
0709 13:03:22.337159 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:22.359601 (+ 22442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.359634 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:22.371174 (+ 11540us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.502074 (+130900us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:22.502123 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.502200 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.949408 (+447208us) raft_consensus.cc:1685] finished
0709 13:03:22.949430 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.949599 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10208,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":5815,"thread_start_us":65878,"threads_started":1,"wal-append.queue_time_us":215116}
I20260709 13:03:22.951447 12269 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:22.951995 12269 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:22.952493 12269 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:22.954962 12314 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.955576 12314 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:22.957085 12298 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.957656 12298 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:22.958542 12177 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.003s	sys 0.001s
I20260709 13:03:22.959975 12298 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:22.962370 12314 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:22.963194 12269 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:22.978019 12299 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:22.981827 12300 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.506s	user 0.003s	sys 0.001s
W20260709 13:03:22.991385  8830 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1794) took 2249 ms (client timeout 1886 ms). Trace:
W20260709 13:03:22.991952  8830 rpcz_store.cc:269] 0709 13:03:20.742268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.754918 (+ 12650us) service_pool.cc:224] Handling call
0709 13:03:22.991114 (+2236196us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b54003dd6d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:22.991284 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1708222848}
I20260709 13:03:22.997056  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:22.998639 12177 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:22.999428 12266 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:23.001108 12177 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:23.001541 12177 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:23.002516 12177 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:23.009428  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:23.017388 12179 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:23.024578  8722 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:23.030910 12273 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:23.031955  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.031157  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:23.033185  8676 raft_consensus.cc:2410] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:23.035095  8675 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:23.031239 12269 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.031839  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.038300  8674 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:23.038946  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:23.031497 12273 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:23.040501 12273 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:23.040587  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.041509  8673 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:23.040597  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.043937  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.042514  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.033581 12299 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:23.048094 12299 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.051674  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.052596  8675 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:23.045665 12275 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:23.042013 12269 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.054636  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.055462  8674 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:23.058086  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:23.057844  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.059597  8722 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:23.061728 12275 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:23.062312 12275 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:23.062762 12275 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:22.536682  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.067323  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.068251  8675 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:23.070672  8827 pending_rounds.cc:85] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 6
I20260709 13:03:23.071108  8827 pending_rounds.cc:107] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:03:23.071442  8827 raft_consensus.cc:2889] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:23.071831  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:23.066521  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.072484  8823 raft_consensus.cc:2376] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10. Re-sending same reply.
I20260709 13:03:23.075855  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:23.076890  8675 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:23.079082  8830 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:03:23.079510  8830 pending_rounds.cc:85] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 6
I20260709 13:03:23.079798  8830 pending_rounds.cc:107] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:03:23.079965  8830 raft_consensus.cc:2889] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:23.075045 12126 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:03:23.084266  8680 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:23.092013 12269 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:23.076470 12299 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:23.101554  9389 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
W20260709 13:03:23.105327  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 45803.1us
I20260709 13:03:23.106318  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:23.112969  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.113914  8673 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:23.118577  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.119910  8673 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:23.132891  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.134298  8676 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:23.116663 12126 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
W20260709 13:03:23.156073  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 49712.8us
I20260709 13:03:23.158128  8722 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:23.169065  8675 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
W20260709 13:03:23.174167  8644 connection.cc:256] RPC call timeout handler was delayed by -1.2235769870000013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:03:23.161387 12323 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:23.175871 12323 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:23.176304 12323 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.176622 12323 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:23.178553 12299 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:23.179245 12299 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.179791  8678 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:23.181102  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.181955  8678 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:23.184604  8681 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:03:23.184628  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.187834  8673 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 5.
I20260709 13:03:23.193948  8675 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:23.209510 12304 log.cc:927] Time spent T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.808s	user 0.004s	sys 0.000s
I20260709 13:03:23.210536  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1807) took 2365 ms. Trace:
I20260709 13:03:23.211655 12273 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:23.211174  8829 rpcz_store.cc:276] 0709 13:03:20.845172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.985524 (+1140352us) service_pool.cc:224] Handling call
0709 13:03:21.997099 (+ 11575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.008042 (+ 10943us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:22.152240 (+144198us) log.cc:844] Serialized 14 byte log entry
0709 13:03:22.271702 (+119462us) log.cc:844] Serialized 14 byte log entry
0709 13:03:23.027411 (+755709us) spinlock_profiling.cc:243] Waited 7.62 ms on lock 0x7b18003cb440. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:03:23.177452 (+150041us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b18003cb440. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:03:23.185638 (+  8186us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:23.210019 (+ 24381us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:23.210098 (+    79us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:23.210174 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.210272 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.210462 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31982,"prepare.queue_time_us":133400,"prepare.run_cpu_time_us":276,"prepare.run_wall_time_us":8802,"raft.queue_time_us":179528,"raft.run_cpu_time_us":375,"raft.run_wall_time_us":20444,"spinlock_wait_cycles":395090304,"thread_start_us":239108,"threads_started":4,"wal-append.queue_time_us":145353}
I20260709 13:03:23.237190 12273 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:23.238771 12299 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:23.253997 12280 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:23.254717 12280 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:23.247699 12323 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:23.256065 12280 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:23.257396 12323 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:23.257642 12280 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:23.267463  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 109525us
I20260709 13:03:23.279232 12323 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:23.280043  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:03:23.280220 12262 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.002s	sys 0.010s
I20260709 13:03:23.280748  9383 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:23.280797 12262 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.002s	sys 0.010s
I20260709 13:03:23.288022  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1902) took 1944 ms. Trace:
I20260709 13:03:23.242254 12275 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.285789 12323 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.288755  8754 rpcz_store.cc:276] 0709 13:03:21.343851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.336383 (+992532us) service_pool.cc:224] Handling call
0709 13:03:22.336688 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:22.351954 (+ 15266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.351990 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:22.352468 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.492810 (+140342us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:23.287157 (+794347us) log.cc:844] Serialized 14 byte log entry
0709 13:03:23.287540 (+   383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.287654 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:23.287719 (+    65us) raft_consensus.cc:1685] finished
0709 13:03:23.287738 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.287928 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"thread_start_us":155206,"threads_started":2,"wal-append.queue_time_us":188462}
I20260709 13:03:23.311822  9383 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:23.281435 12262 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:23.312237 12323 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:23.313886 12262 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:23.318357  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.319010  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 52450.6us
I20260709 13:03:23.321624  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.322487 12299 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:23.323237 12269 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:23.323289 12299 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.324174 12269 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.324779 12332 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:23.325138 12177 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:23.325444 12332 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.326475 12269 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:23.320111 12323 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:23.328516  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.326381 12299 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:23.332857  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 625) took 1140 ms. Trace:
I20260709 13:03:23.333370  8822 rpcz_store.cc:276] 0709 13:03:22.192589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.287386 (+ 94797us) service_pool.cc:224] Handling call
0709 13:03:22.319082 (+ 31696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.319315 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:22.330205 (+ 10890us) log.cc:844] Serialized 14 byte log entry
0709 13:03:22.532664 (+202459us) log.cc:844] Serialized 14 byte log entry
0709 13:03:23.332430 (+799766us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:23.332502 (+    72us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:23.332554 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.332635 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.332795 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":803393,"prepare.queue_time_us":19955,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":99,"raft.queue_time_us":808784,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":46114,"thread_start_us":161967,"threads_started":1,"wal-append.queue_time_us":266298}
I20260709 13:03:23.327150  8645 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:23.335778  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:23.336073 12337 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:23.336553  9381 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:23.338879 12307 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:23.339383 12307 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.339780 12307 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:23.340147  8722 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:23.341006 12323 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:23.341383 12323 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.341512  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:23.341940 12323 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:23.342070 12337 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:22.519203  8755 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:03:23.343698  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2297) took 1623 ms. Trace:
I20260709 13:03:23.344215  8719 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:23.344412 12123 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.344004  8755 rpcz_store.cc:276] 0709 13:03:21.719961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.518420 (+798459us) service_pool.cc:224] Handling call
0709 13:03:22.518837 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.343412 (+824575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.343628 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:23.345021 12123 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.344249  8796 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:23.346818 12301 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:23.347369 12301 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:23.351218  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:23.335505  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 839304us
I20260709 13:03:23.364682  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:23.365887  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:23.367868  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:23.368530  9381 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:23.371078  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:23.374351  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:23.376065  9389 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:23.381557  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:23.405709  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.360809 12266 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:23.440817  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:23.351935 12325 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:23.468003  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:23.470738  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 113742us
W20260709 13:03:23.472290  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.473332  8749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1896) took 2106 ms (client timeout 1957 ms). Trace:
W20260709 13:03:23.475267  8749 rpcz_store.cc:269] 0709 13:03:21.341365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.557089 (+215724us) service_pool.cc:224] Handling call
0709 13:03:23.447866 (+1890777us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400139054. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:23.448040 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774150528}
I20260709 13:03:23.471760  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:23.478348  8722 leader_election.cc:336] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.804s (SENT)
I20260709 13:03:23.367285 12273 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.104s
W20260709 13:03:23.479614  8722 leader_election.cc:336] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.865s (SENT)
I20260709 13:03:23.348294 12301 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.489039  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:23.496342  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1911) took 1514 ms. Trace:
I20260709 13:03:23.491828  9382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:33196 (request call id 4) took 1761 ms. Trace:
I20260709 13:03:23.501268  9382 rpcz_store.cc:276] 0709 13:03:21.721311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.731767 (+ 10456us) service_pool.cc:224] Handling call
0709 13:03:23.482265 (+1750498us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54006e08a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2476520 00007f32e24f3652 00007f32e74c160a 00007f32e0a477d4 00007f32e0a47762 00007f32e0a47691 00007f32e0a4760c 00007f32e0a468c2 00007f32e083448c 00007f32e08338c6
0709 13:03:23.482587 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233086848}
I20260709 13:03:23.497009  8753 rpcz_store.cc:276] 0709 13:03:21.982114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.924757 (+942643us) service_pool.cc:224] Handling call
0709 13:03:22.925311 (+   554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.925550 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:03:23.311308 (+385758us) log.cc:844] Serialized 14 byte log entry
0709 13:03:23.495841 (+184533us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:23.495918 (+    77us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:23.495996 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.496090 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.496267 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":376267,"prepare.queue_time_us":521875,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"thread_start_us":199495,"threads_started":3}
I20260709 13:03:23.507164  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.508297  8672 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:22.507472  8794 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:23.516348  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 48461.6us
W20260709 13:03:23.516340  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 165191us
I20260709 13:03:23.518536  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:23.520375  8719 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:23.522130  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50480.8us
I20260709 13:03:23.517354  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:23.531831  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:23.532610  8673 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:23.536206  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 1.02879e+06us
I20260709 13:03:23.495963 12266 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:23.537222  8794 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:23.538875  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 580) took 1716 ms. Trace:
I20260709 13:03:23.483801 11666 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:03:23.547657  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:23.544418  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:23.547830  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.550716  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:23.551684  9382 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
W20260709 13:03:23.553779  8794 connection.cc:256] RPC call timeout handler was delayed by -1.0986160580000615s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 13:03:23.557088  8793 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:23.546099  8752 rpcz_store.cc:276] 0709 13:03:21.822540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.662442 (+839902us) service_pool.cc:224] Handling call
0709 13:03:22.662779 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.662994 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:23.312303 (+649309us) log.cc:844] Serialized 14 byte log entry
0709 13:03:23.538447 (+226144us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:23.538508 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:23.538562 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.538656 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:23.538809 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":599891,"prepare.queue_time_us":706818,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":64810,"thread_start_us":253030,"threads_started":3}
I20260709 13:03:23.561353  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:23.526911 12267 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:23.580332  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:23.551043  8673 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:03:23.583293  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:23.585006  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:23.586000  8824 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:23.590652  8826 raft_consensus.cc:2393] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 5: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:23.592315  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:23.540302 11953 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:23.599382  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:23.555420 12291 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:23.610214 12291 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.557164 12337 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.610775 12291 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:23.611142 12337 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.600546  8793 leader_election.cc:400] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8.
I20260709 13:03:23.613720 12337 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:23.554713 12129 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.581127 12267 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:23.605508 12305 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:23.606220 12347 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:23.637269  8678 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:23.637049  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.607183 12314 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:23.607431  8796 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:23.563086 12306 raft_consensus.cc:2764] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:03:23.547780 12123 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:23.562201 12269 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:23.587095 12225 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:23.642931  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.587422 12299 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.560916 12128 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:23.638718  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:23.642278  8676 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:23.642124  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.650815  8672 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:23.643077  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.643543  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.619042  8794 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:23.598955 11953 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.636302 12267 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:23.637102 12266 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:23.648366 12105 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:23.644779 12273 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:23.660874  8680 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:23.661947  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.662887  8681 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:23.642393 12314 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:23.664328  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:23.665658  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.668256  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:23.670198  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:23.610500 12323 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:23.672503  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 53619us
I20260709 13:03:23.672644  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:23.676070  8794 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:23.677975  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:23.678083  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:23.676757  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 156450us
I20260709 13:03:23.649641 12305 raft_consensus.cc:697] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:23.636983 12129 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.655385 11953 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:23.703421  8751 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:23.687693  8824 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:23.652004 12269 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:23.689599  8794 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:23.689920  8756 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:23.690922  8826 raft_consensus.cc:2393] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:23.692627  8673 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:23.703421  8749 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:23.705694  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:23.641958 12095 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:03:23.719590  8750 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:23.712909  8794 leader_election.cc:400] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11.
I20260709 13:03:23.714253  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:23.661475 12105 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.721340  8794 leader_election.cc:403] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:03:23.722184  8824 raft_consensus.cc:2410] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:23.675536 12314 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:23.677532 12349 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.652537 12225 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:23.725454  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:23.733480  8754 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:23.727352  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:23.733397  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 253815us
I20260709 13:03:23.655318 12299 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.707899 12123 raft_consensus.cc:2764] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:03:23.680850 12306 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:23.725767 12273 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.729629  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:23.744339  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:23.745009  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:23.745082  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 107645us
W20260709 13:03:23.741596  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 197289us
I20260709 13:03:23.733141 12128 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:23.739445  8722 leader_election.cc:336] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.049s (SENT)
I20260709 13:03:23.717609 12081 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:23.725602 12105 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:23.742228  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.735395  8680 raft_consensus.cc:2410] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:03:23.758406  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:23.763581  8824 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:23.767388  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:23.769950  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.735375 12349 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.771165  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:23.771274  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:23.785152  8673 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:23.771924 11908 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:23.752194 12273 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.773144  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:23.773872  8722 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:23.758224 12346 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:03:23.776546  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.791611  8755 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:23.791782  8681 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
W20260709 13:03:23.778743 12280 consensus_meta.cc:287] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.009s
I20260709 13:03:23.742933 12306 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:23.788887 11996 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:23.801595 12306 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:23.803323  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 90661.7us
W20260709 13:03:23.803876  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 120832us
I20260709 13:03:23.798010 12301 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:23.808079  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:23.822125  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:23.823019  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.823732  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:23.825265  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:23.808746 12352 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:23.827248 12352 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:23.828279 12352 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:23.801764 11996 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:23.829471 12352 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:23.830866 12081 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:23.831146  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 91746.8us
I20260709 13:03:23.831818 12081 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:23.832904 12269 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:23.833976 12177 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:23.823832  8680 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:23.826514  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:23.822904 12129 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:23.838577  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:23.839401  8680 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:23.836076  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:23.831290 11996 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:23.869616  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:23.877827  8680 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:23.878847  8673 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
W20260709 13:03:23.886507  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.909097  8754 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:23.910601  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:23.911877  8824 raft_consensus.cc:2393] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:23.912386  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:23.915493  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 110083us
W20260709 13:03:23.919589  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 173715us
I20260709 13:03:23.921212  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:23.922112  8747 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:23.917559 12353 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:23.927430 11908 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:23.928296  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:23.928408 12371 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:23.929164 11908 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:23.929590 11908 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:23.930015  8675 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:03:23.932156 12364 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:23.936033  8680 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:03:23.937126  8675 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 13:03:23.939023  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 146678us
W20260709 13:03:23.960122 12351 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 13:03:23.834729 12305 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:23.961835  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:23.974363  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 100604us
I20260709 13:03:23.949425  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:23.962874  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 103857us
I20260709 13:03:23.940565  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:23.975804  8680 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:23.984134  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:23.970793 12357 log.cc:927] Time spent T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 13:03:23.998132  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 61216.4us
W20260709 13:03:24.001959  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 61520.5us
I20260709 13:03:24.005308  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:24.007843  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:24.006021  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:24.010569  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 91314us
I20260709 13:03:24.007812  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:23.971246 12303 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:23.941169 11996 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:03:24.000464 12323 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:24.021253  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:24.011808 12350 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:03:24.044327  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:24.049579  8828 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:03:24.060612 12349 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:24.061388 12360 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:03:24.062153 12334 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.006428 11908 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.063330 11908 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:24.062822 12334 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:24.066004 12323 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:24.071388 12332 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:24.078439  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.079191  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 58237.9us
I20260709 13:03:24.035225 12370 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:24.045553  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:24.089823  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:24.096568 12363 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:24.107314 12363 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.108882 12363 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:24.119571  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:24.120229 12314 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:24.121970  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:24.097225  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 113151us
W20260709 13:03:24.123065  8828 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 636) took 1693 ms (client timeout 1641 ms). Trace:
W20260709 13:03:24.124670  8828 rpcz_store.cc:269] 0709 13:03:22.430225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.475751 (+ 45526us) service_pool.cc:224] Handling call
0709 13:03:24.122850 (+1647099us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54003ab6d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:24.122988 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3287297408}
I20260709 13:03:24.126259  8748 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:24.126565  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:24.127951  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:24.128016  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:24.128803  8748 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:24.129724  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:24.129633  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:24.130352  8752 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:24.132681 12275 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:24.135751  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:24.136861  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:24.137554  8748 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:24.140858  8747 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:24.143573  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:24.142349  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:24.144469  8751 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:24.147801  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:24.149230  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:24.147934  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:24.150450  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:24.150578  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:24.151342  8751 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:24.154770  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:24.151942  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:24.146396  8752 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:24.156798  8754 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:24.158064  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:24.159222  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:24.161947 12347 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.514s	user 0.004s	sys 0.008s
I20260709 13:03:24.162715 12347 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:24.093358 12384 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:24.161104  8748 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:24.165755  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:24.169485  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:24.167695 12384 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:24.172322 12384 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:24.173496 12384 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:24.164237  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:24.175967  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:24.176501 12366 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:24.177381 12373 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:24.177062 12384 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:24.180853 12376 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:24.181391 12376 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.181838 12376 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:24.182540  8748 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
W20260709 13:03:24.184573  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.182976 12370 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:24.185973 12312 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:24.079080 12361 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:03:24.187289 12361 raft_consensus.cc:3055] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:24.187822 12361 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:24.187892  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:24.186122  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:24.189651 12361 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:24.225827  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:24.227587  8747 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:24.254431  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 162582us
I20260709 13:03:24.257230  8824 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:24.259428  8751 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:24.265446  8754 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:24.255481  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:24.286893  8754 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
W20260709 13:03:24.289865  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.290978  8678 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 13:03:24.290401  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 281251us
I20260709 13:03:24.293314  8748 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:24.299578  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:24.300559  8824 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
W20260709 13:03:24.309448  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.319351  8747 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:24.256563 12354 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.304776 12366 consensus_queue.cc:1243] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:03:24.325402  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 285546us
I20260709 13:03:24.325896  9384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:03:24.331125  9384 raft_consensus.cc:606] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:24.327039  8824 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: [8.7-8.7]
W20260709 13:03:24.332566  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 126492us
I20260709 13:03:24.334084  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:24.306420 11711 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:24.337831  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:24.343276  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.339659  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:24.345814  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:24.350368  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:24.351711  8747 raft_consensus.cc:2410] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:24.305492 12275 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:03:24.301465 12347 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:03:24.299063 12325 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:03:24.386571  9353 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:24.341961 12366 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:24.340781 12299 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:24.397847  8748 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:24.352430 12352 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:24.411722  8678 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:03:24.412513  8678 pending_rounds.cc:85] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 6
I20260709 13:03:24.412837  8678 pending_rounds.cc:107] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:03:24.413185  8678 raft_consensus.cc:2889] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:24.332003 12354 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:24.402779  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:24.417074 12352 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:24.387640 12353 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:03:24.436573  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 421014us
I20260709 13:03:24.437196  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:24.438129  8748 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:24.402746 12366 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:24.450500  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 103469us
I20260709 13:03:24.451849  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:24.422191 12314 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:24.468142  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 81725.6us
I20260709 13:03:24.434049 12352 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:24.472828  8747 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:24.474037  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:24.480458  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:24.494065  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:24.489414  8722 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:24.478758 12383 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.477402 12325 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:24.505151  8722 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:24.481781 12275 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:24.473548 12393 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:24.513506  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.473856 12362 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.509511  8722 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:24.501150  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 155614us
I20260709 13:03:24.488163 12347 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:24.520020 12389 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:24.522028 12401 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:24.510931 12383 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.505906 12372 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:24.524130 12389 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:24.527071  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:24.486876 12381 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:24.515852  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 64885.1us
I20260709 13:03:24.541189  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:24.543104 11711 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:24.546145 12407 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:24.508610 12400 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:24.554200  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:24.571949  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:24.573240  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:24.581632  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:24.583285  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:24.517022 12397 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:24.587096  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 70511.2us
I20260709 13:03:24.550891 12275 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:24.588495  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:24.575085  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:24.548131 12392 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:24.537302 12362 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:24.586040 11908 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.522s	user 0.013s	sys 0.004s
W20260709 13:03:24.590137  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 62627.7us
I20260709 13:03:24.578928 12325 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:24.552235 12389 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.597198  8645 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:24.574728 12373 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:24.571564 12400 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:24.578101 12269 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:24.569916 12372 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:24.544613 11711 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:24.658299 12372 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:24.660665 12363 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.548s	user 0.013s	sys 0.000s
I20260709 13:03:24.661278 12363 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:24.602191 12347 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:24.599053 12356 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:24.663553 12356 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:24.607596 12388 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
W20260709 13:03:24.608461 12385 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:03:24.611030  9384 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:24.612571 12362 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:24.551550 12402 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:03:24.615633  8828 raft_consensus.cc:2410] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:24.621629  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 631) took 2320 ms. Trace:
I20260709 13:03:24.672688  8821 rpcz_store.cc:276] 0709 13:03:22.298445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.362960 (+ 64515us) service_pool.cc:224] Handling call
0709 13:03:22.363368 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.913383 (+550015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.617327 (+1703944us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54002689d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:24.617600 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1262452480}
I20260709 13:03:24.504405 12366 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.623538 12395 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:24.623612 12376 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.578598 12314 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:03:24.550504 12381 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:24.625877 11908 consensus_queue.cc:1059] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:03:24.514158 12353 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.528116 12393 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:24.653774  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 81357.3us
I20260709 13:03:24.595319 12398 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 13:03:24.604197  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:24.703393  8830 raft_consensus.cc:2376] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12. Re-sending same reply.
W20260709 13:03:24.666500  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 128386us
I20260709 13:03:24.704689 12405 raft_consensus.cc:3055] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:24.705175 12405 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:24.707212  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:24.654260  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 57125.4us
I20260709 13:03:24.708796  8645 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:24.708902  8719 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:24.669770  9385 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:24.654795 12337 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:24.641644 12369 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:03:24.669251 12363 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:24.656335 12400 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:24.665668 12373 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:24.685216 12362 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:24.685514 12356 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.030325 12225 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:24.689006 12376 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:24.668824 12372 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:24.677022 12354 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:24.702188 12381 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:24.690847 12383 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:24.744782 12412 raft_consensus.cc:3055] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:24.757241 12373 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:24.758152 12412 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:24.759647 12412 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.734447 12382 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.727355  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:24.689972 12403 consensus_queue.cc:1175] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" responder_term: 11 status { last_received { term: 8 index: 5 } last_committed_idx: 5 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:24.728909  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:24.761885  8719 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:24.736502  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:24.761245 12382 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.731962 12352 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:24.725661 12347 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:24.760470 12373 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:24.654491 12389 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:24.764499  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 59984.5us
I20260709 13:03:24.765668  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2251) took 1062 ms. Trace:
W20260709 13:03:24.733860 12323 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.007s	sys 0.003s
I20260709 13:03:24.764163  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:24.769784  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:24.732920 12314 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:24.728036 12275 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:24.771203  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 176040us
W20260709 13:03:24.739598  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 123137us
I20260709 13:03:24.735630 12392 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:24.770084 12372 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:24.777261 12352 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:24.725914 12397 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:03:24.729451 12299 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:03:24.778084 12378 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:24.779302  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 67072.3us
I20260709 13:03:24.735296 12269 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.070s
W20260709 13:03:24.748888 11953 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.021s	user 0.016s	sys 0.000s
I20260709 13:03:24.786826  8675 rpcz_store.cc:276] 0709 13:03:23.703160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.938015 (+234855us) service_pool.cc:224] Handling call
0709 13:03:23.938307 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.938546 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.938583 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:23.974525 (+ 35942us) log.cc:844] Serialized 26 byte log entry
0709 13:03:24.208989 (+234464us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:24.209053 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.209183 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:24.765409 (+556226us) raft_consensus.cc:1685] finished
0709 13:03:24.765435 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:24.765594 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":191191,"threads_started":1,"wal-append.queue_time_us":401774}
I20260709 13:03:24.846101  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:24.739291 12402 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:24.799355 12356 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:03:24.796064 12323 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.008s	sys 0.004s
I20260709 13:03:24.833562  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:24.832685  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 68620.1us
I20260709 13:03:24.780675  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:24.842293  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:24.842505  8794 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:24.856652 12305 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:24.857462 12305 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:24.857990 12305 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:24.860513 12305 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:24.804785 12314 raft_consensus.cc:697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:24.838649 12299 consensus_queue.cc:1243] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:03:24.848238  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:24.804117 12275 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.805003 12412 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:24.795451 11908 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.802528 12337 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:24.856689  8794 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:24.843183 12378 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.801631 12376 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:03:24.848307 12225 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
W20260709 13:03:24.872288  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.873507 12314 consensus_queue.cc:237] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.875548  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:24.875463 12411 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:03:24.876492  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:24.878120 12382 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:24.795815 12405 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.879915 12392 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.881094  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:24.881572 12299 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:24.881963 12420 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:24.885223  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:24.862830 12269 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:03:24.848904 12402 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:24.906177 12305 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:24.894763 12393 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:24.895284  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:24.901867 12275 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:24.902788  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:24.935803 12418 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
W20260709 13:03:24.935809 12266 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.008s	sys 0.019s
W20260709 13:03:24.939085 12266 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.008s	sys 0.020s
I20260709 13:03:24.907280  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:24.871495  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:24.852707 12373 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:03:24.850749 11953 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.837533 12400 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:24.919662  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:24.941843  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:24.906461 12378 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:24.923964  8754 raft_consensus.cc:2410] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:24.883414 12354 raft_consensus.cc:3055] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:24.906051 12356 raft_consensus.cc:697] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:24.928014  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.956498  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.929164  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.956743 12356 consensus_queue.cc:237] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.935451 12402 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.957347 12337 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.959336  8822 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:24.961227  9383 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
W20260709 13:03:24.902979  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 46371.4us
I20260709 13:03:24.942559  8681 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:24.943012  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:24.942183  8753 raft_consensus.cc:2376] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9. Re-sending same reply.
W20260709 13:03:24.943390 12404 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:03:24.963608  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:24.964447  8753 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:24.965528  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:24.967259  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50622.5us
I20260709 13:03:24.966303  8753 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:24.906663 11908 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:24.975298  8753 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:24.982506  8753 pending_rounds.cc:85] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 3
I20260709 13:03:24.985735  8645 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:24.919126 12299 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:24.958436 12402 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:24.968262 12305 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:24.997433  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:25.013193  8677 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:25.018337  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:25.020841  8747 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:03:25.003638  8753 pending_rounds.cc:107] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:03:24.946817 12354 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:25.023542 12354 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.025017 12354 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:25.025288 12275 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.005156  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:24.992223  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:25.027158  8754 raft_consensus.cc:2376] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8. Re-sending same reply.
I20260709 13:03:25.027702 12275 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:25.021998  8747 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:25.033573 12378 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.034534 12299 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:25.036571  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:25.022468  8753 raft_consensus.cc:2889] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:25.042769  8681 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:25.043497  8645 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:25.045914  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:25.046051  8754 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:25.052582  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:25.057291 12422 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:25.065331 12417 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.065415  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:25.066090 12417 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:25.066848  9410 debug-util.cc:398] Leaking SignalData structure 0x7b08002e9b40 after lost signal to thread 7529
W20260709 13:03:25.068198  9410 debug-util.cc:398] Leaking SignalData structure 0x7b08002e98c0 after lost signal to thread 8613
I20260709 13:03:25.070112 12417 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:25.071353  9353 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.945s (SENT) [suppressed 36 similar messages]
I20260709 13:03:25.071650  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:25.073047  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:25.075927 12378 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:24.045202 12291 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.075634  8674 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:25.079152  8827 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:25.079509 12354 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:25.080399  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:25.080484 12128 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:25.049854 12299 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:25.081961  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:25.082156  8674 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:25.082329 12299 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:25.082571  8749 raft_consensus.cc:2376] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11. Re-sending same reply.
I20260709 13:03:25.082907 12299 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:25.084194  8749 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 13:03:25.085562  8749 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:03:25.049659 12420 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:25.086030 12420 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.086411 12420 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:25.081856 12400 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:24.027761 12303 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:25.069746  9410 debug-util.cc:398] Leaking SignalData structure 0x7b08002e9f00 after lost signal to thread 8705
I20260709 13:03:25.053428 12418 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:25.105983 12400 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:25.107348 12400 raft_consensus.cc:3055] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:25.107729 12400 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:25.108533  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1635) took 1440 ms. Trace:
I20260709 13:03:25.109524 12400 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:25.111821 12400 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:25.109184  8828 rpcz_store.cc:276] 0709 13:03:23.668492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.175564 (+507072us) service_pool.cc:224] Handling call
0709 13:03:25.108211 (+932647us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b540016ba54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:25.108437 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1005008768}
I20260709 13:03:24.047513 11996 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.115335  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:25.115412 11996 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.116757  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:25.117910  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:25.118101 11996 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:25.119923  9353 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:25.123329  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 77522.6us
I20260709 13:03:25.124217  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:25.124449  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:25.125646  8754 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:25.061859 12389 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:25.063995 12422 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:25.081940 12128 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:25.081247 12354 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:25.133714  8828 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:25.139195  8674 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
W20260709 13:03:25.163712  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.154793  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:25.176136  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:25.056155 12423 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:03:25.132869 12225 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:25.129127 12417 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:25.179522  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:25.188938  8674 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:25.192299 11908 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:25.132498 12418 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:25.195406 12420 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:25.195677 12418 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.198024  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:25.199999  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:25.200978  8754 raft_consensus.cc:2368] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11.
I20260709 13:03:25.155695 12371 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.188267 12225 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:25.203142 12371 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.203665 12225 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:25.183043 12354 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:25.207542  8681 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:25.210315 12299 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:25.211390 12325 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.002s
I20260709 13:03:25.211948 12325 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:25.197996 12420 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:25.213565 12378 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:25.185077 12305 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:25.209723 12354 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:25.202167  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:25.194446 12417 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.215817  8754 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:25.214525 12305 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:25.216110  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.187286  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:25.230440  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:25.233187  8828 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:25.178725 12376 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:25.219676  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:25.223925  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 102975us
I20260709 13:03:25.242687  9389 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:25.236557  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:25.213675 12299 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:25.221541  8645 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:25.238206  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.206413 12371 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:25.234503  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:25.247833  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.246141  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:25.250798  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:25.250959  8719 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:25.230824 12305 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:25.225731 12417 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:25.234395 12378 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:25.255735  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:25.264689  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.266289  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:25.259361 12427 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:25.252811 12420 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:25.254495 12325 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:25.287328  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:25.257496 12376 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:25.288704  8674 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
W20260709 13:03:25.291953  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 48641us
I20260709 13:03:25.244935 12407 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:03:25.272022 12275 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:25.309545  8754 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:25.312454  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:25.313225  8674 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:25.314316  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:25.314947  8674 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:25.319116  8719 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:25.311851 12407 consensus_queue.cc:1243] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:03:25.270956 12422 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.320561 12303 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:25.268921 12393 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:03:25.323637 12393 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:25.324431  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:25.280802 12420 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.329506  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:25.330479  8674 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:25.324054 12393 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:25.334470  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:25.345906  8747 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:25.335975 12323 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:25.340790  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:25.254776 12392 consensus_queue.cc:1059] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:03:24.057413 12262 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:25.356739 12262 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:25.362802 12376 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.272779 12371 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:03:25.366163 12366 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:24.048156 12177 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.373023 12177 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.350479 12433 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:25.351378 12323 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.377736 12323 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:25.379307  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 121370us
I20260709 13:03:25.304719 12325 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:25.311524 12275 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.390376  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:25.385232  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:25.395156 12432 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:03:25.380264  8794 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:03:25.396121 12429 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:03:25.362860  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:25.398715 12275 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:25.399688 12393 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.400907 12352 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.346405 12420 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:25.401937 12430 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:03:25.404923  8674 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:25.413434  8676 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:25.414561  8747 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:25.386755 12433 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:25.430176  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.448768  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.433490 12128 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.411079 12356 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.437424 12427 raft_consensus.cc:3055] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:25.438912 12393 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:25.346174 12225 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:25.442840 12371 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:25.448093  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:25.413661 12402 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:25.449915  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 157056us
I20260709 13:03:25.477205  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:25.455210 12410 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.495479 12402 raft_consensus.cc:697] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:25.450798 12352 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:25.469224 12356 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:25.720062  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:25.721086  8676 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:25.474027 12371 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.721868  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:25.509092 12105 consensus_meta.cc:287] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.736s	user 0.017s	sys 0.012s
W20260709 13:03:25.590891  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 148536us
W20260709 13:03:25.616364  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.729357  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 252281us
I20260709 13:03:25.499898 12325 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:25.737125  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.771322  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.771876  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.466045 12427 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:25.757478  9353 leader_election.cc:400] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11.
I20260709 13:03:25.774540  9353 leader_election.cc:403] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:03:25.730270  8747 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:25.776795  8747 pending_rounds.cc:85] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 7
I20260709 13:03:25.777153  8747 pending_rounds.cc:107] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:03:25.777426  8747 raft_consensus.cc:2889] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:25.498093 12128 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:25.491654 12280 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:25.760676  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:25.773193  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:25.781828  8748 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:25.667337 12412 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.684305 12303 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.670060 12177 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:25.785130  8676 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:25.808158  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:25.792050  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:25.739359 12410 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:25.764750 12371 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:25.765352 12376 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:25.850044  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:25.851228 12307 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 2.511s	user 0.017s	sys 0.003s
W20260709 13:03:25.851068  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 93760.3us
W20260709 13:03:25.859047  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.821519 12280 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:25.831364 12437 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 13:03:25.914222  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 106145us
I20260709 13:03:25.765234 12105 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:25.840950 12128 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:25.929461  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 208749us
W20260709 13:03:25.932730  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.907864 12177 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.935449  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:25.824625 12303 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:25.924926 12366 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.005s
I20260709 13:03:25.724349 12433 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.834652 12412 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.938287  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 589) took 2261 ms. Trace:
I20260709 13:03:25.965987  8754 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:25.965492  8755 rpcz_store.cc:276] 0709 13:03:23.676839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.707567 (+ 30728us) service_pool.cc:224] Handling call
0709 13:03:23.708054 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.792136 (+ 84082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.937866 (+2145730us) spinlock_profiling.cc:243] Waited 83.1 ms on lock 0x7b540018ced4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:25.938165 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191237888}
I20260709 13:03:25.966485  8754 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:25.952311 12177 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:25.910835 12280 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:25.950527 12443 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 13:03:25.971149  9410 debug-util.cc:398] Leaking SignalData structure 0x7b0800339c60 after lost signal to thread 8780
W20260709 13:03:25.978269  9410 debug-util.cc:398] Leaking SignalData structure 0x7b08002f09e0 after lost signal to thread 8854
W20260709 13:03:25.978698  9410 debug-util.cc:398] Leaking SignalData structure 0x7b08003d14a0 after lost signal to thread 8855
W20260709 13:03:25.982476  9410 debug-util.cc:398] Leaking SignalData structure 0x7b08003b42c0 after lost signal to thread 9413
I20260709 13:03:25.912909 12402 consensus_queue.cc:237] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.911697 12307 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.970547  8754 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:25.951310 12126 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:25.999472  8754 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:25.985558 12225 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:26.012969  8824 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 13:03:26.014376  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.992281 12450 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.999264 12352 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:26.028504  8754 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:25.939937 12427 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:26.028085 12450 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:26.053062 12450 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:26.023180 12382 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:26.036527  8754 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:03:26.063482  8754 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:26.064464  8754 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:26.068022  8754 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:26.029333 12356 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:26.030539 12455 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:26.076138  8754 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:26.077610  9385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 604) took 1506 ms. Trace:
W20260709 13:03:26.066434 12306 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 2.231s	user 0.014s	sys 0.005s
I20260709 13:03:26.086135  9385 rpcz_store.cc:276] 0709 13:03:24.570958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.571195 (+   237us) service_pool.cc:224] Handling call
0709 13:03:24.587468 (+ 16273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.725492 (+138024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.077214 (+1351722us) spinlock_profiling.cc:243] Waited 77.5 ms on lock 0x7b54006275d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:26.077471 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178318592}
I20260709 13:03:26.066679 12427 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:26.094276 12275 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.648s	user 0.015s	sys 0.000s
I20260709 13:03:26.037693 12392 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:03:26.123935  9383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 537) took 1241 ms. Trace:
I20260709 13:03:26.124662  9383 rpcz_store.cc:276] 0709 13:03:24.870444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.870687 (+   243us) service_pool.cc:224] Handling call
0709 13:03:26.111719 (+1241032us) spinlock_profiling.cc:243] Waited 46.3 ms on lock 0x7b54001ccd54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:26.111871 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106553216}
I20260709 13:03:26.083220 12378 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:26.128710 12378 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:26.079386 12410 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:26.109826 12460 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.131387 12303 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:26.131790 12460 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:26.132488  8826 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1839) took 2536 ms (client timeout 1724 ms). Trace:
W20260709 13:03:26.132959  8826 rpcz_store.cc:269] 0709 13:03:23.596016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.638218 (+ 42202us) service_pool.cc:224] Handling call
0709 13:03:26.132194 (+2493976us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b54001937d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:26.132401 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59954944}
I20260709 13:03:26.074198 12225 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:03:26.111840 12392 consensus_queue.cc:1243] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:03:26.134986 12447 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.175s	user 0.000s	sys 0.002s
I20260709 13:03:26.135319 12307 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:26.135407  8672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2241) took 2512 ms (client timeout 1976 ms). Trace:
W20260709 13:03:26.135915  8672 rpcz_store.cc:269] 0709 13:03:23.622538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.654449 (+ 31911us) service_pool.cc:224] Handling call
0709 13:03:26.135174 (+2480725us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b5400257bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:26.135325 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1525788544}
I20260709 13:03:26.099184 12306 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:26.128132 12452 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:03:26.109041 12275 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:26.129940 12262 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:26.141899  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.134348  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:26.139983  8755 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:26.153990  8826 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:26.066831 12382 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:26.148270  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.138552  8672 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: [7.5-8.6]
I20260709 13:03:26.157648 12303 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.143689  8675 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: []
W20260709 13:03:26.159258  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.160867  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1805) took 1163 ms. Trace:
I20260709 13:03:26.161489  8672 rpcz_store.cc:276] 0709 13:03:24.997220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.137046 (+1139826us) service_pool.cc:224] Handling call
0709 13:03:26.138270 (+  1224us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:26.160421 (+ 22151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.160782 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:26.158434 12303 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:26.142309 12306 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:26.155079 12305 consensus_meta.cc:287] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.887s	user 0.014s	sys 0.000s
I20260709 13:03:26.168866 12305 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:26.139894 12456 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:03:26.172789 12305 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:26.141500 12275 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:26.174412 12378 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:26.176029 12460 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:26.170066 12303 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:26.180222 12412 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:26.181233 12105 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:26.123121 12393 consensus_meta.cc:287] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.649s	user 0.008s	sys 0.004s
I20260709 13:03:26.186540  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:26.134358 12301 consensus_meta.cc:287] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 2.303s	user 0.012s	sys 0.003s
I20260709 13:03:26.196745  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1652) took 1339 ms. Trace:
I20260709 13:03:26.210341  8826 rpcz_store.cc:276] 0709 13:03:24.857682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.186146 (+1328464us) service_pool.cc:224] Handling call
0709 13:03:26.196569 (+ 10423us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:26.183722 12266 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.212843 12266 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:26.217563 12266 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:26.221918  8672 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:26.225968  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:26.226982  8826 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:26.227596  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1661) took 1363 ms. Trace:
I20260709 13:03:26.227963  8826 rpcz_store.cc:276] 0709 13:03:24.864210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.225539 (+1361329us) service_pool.cc:224] Handling call
0709 13:03:26.227521 (+  1982us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:26.229103  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:26.229806  8826 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
W20260709 13:03:26.229890  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.230929  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1806) took 1233 ms. Trace:
I20260709 13:03:26.231441  8672 rpcz_store.cc:276] 0709 13:03:24.997498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.162831 (+1165333us) service_pool.cc:224] Handling call
0709 13:03:26.221637 (+ 58806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.222787 (+  1150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.230850 (+  8063us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:26.237689  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1654) took 1379 ms. Trace:
W20260709 13:03:26.238283  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.238225  8826 rpcz_store.cc:276] 0709 13:03:24.858309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.228759 (+1370450us) service_pool.cc:224] Handling call
0709 13:03:26.237595 (+  8836us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:26.239637  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:26.243062 12417 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.972s	user 0.005s	sys 0.009s
I20260709 13:03:26.243688 12417 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:26.244513 12323 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.866s	user 0.005s	sys 0.008s
I20260709 13:03:26.244994 12323 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:26.245332 12409 raft_consensus.cc:3055] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:26.245837 12409 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:26.246235 12417 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:26.225193 12301 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11.
I20260709 13:03:26.249063 12382 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:26.249413 12409 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:26.209513 12420 consensus_meta.cc:287] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.754s	user 0.011s	sys 0.004s
W20260709 13:03:26.252041  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 250790us
I20260709 13:03:26.225106 12262 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:26.278280  8672 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:03:26.236445 12266 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:26.273062 12446 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.343s	user 0.001s	sys 0.000s
W20260709 13:03:26.270229  9384 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 601) took 1721 ms (client timeout 2050 ms). Trace:
W20260709 13:03:26.279412  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.256403 12126 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:26.279256  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1807) took 1281 ms. Trace:
W20260709 13:03:26.281965  9384 rpcz_store.cc:269] 0709 13:03:24.548908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.570147 (+ 21239us) service_pool.cc:224] Handling call
0709 13:03:26.270007 (+1699860us) spinlock_profiling.cc:243] Waited 36.8 ms on lock 0x7b540046a5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:26.270153 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84692224}
I20260709 13:03:26.291232 11953 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:26.289825  8672 rpcz_store.cc:276] 0709 13:03:24.997727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.254459 (+1256732us) service_pool.cc:224] Handling call
0709 13:03:26.277902 (+ 23443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.278982 (+  1080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.279194 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:26.304196  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.303941  8754 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:03:26.305634  8754 pending_rounds.cc:85] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 5
I20260709 13:03:26.306038  8754 pending_rounds.cc:107] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:03:26.307642 12409 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:26.280777 12420 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:26.326303  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.306411  8754 raft_consensus.cc:2889] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:26.313508 12455 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11.
I20260709 13:03:26.287987 12266 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:26.345911 12353 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:26.347385 12353 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:26.347885 12353 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:26.350942 12353 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:26.353020 12353 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:26.331601 12323 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:26.360039 12301 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.377107 12266 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:26.385555 12273 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:26.415738  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.419981  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:26.388912 12301 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:26.450995  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:26.452428  9383 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:03:26.398978 12395 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.455310  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:26.361925 12126 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:26.410418 12273 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:26.470796  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:26.471657  9385 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:26.718736  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.518429  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.679533  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2264) took 1704 ms. Trace:
W20260709 13:03:26.846771  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.863783  8676 rpcz_store.cc:276] 0709 13:03:24.974830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.964854 (+990024us) service_pool.cc:224] Handling call
0709 13:03:25.968515 (+  3661us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.968913 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.968948 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.969516 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.457633 (+488117us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:26.457802 (+   169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.457880 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:26.679117 (+221237us) raft_consensus.cc:1685] finished
0709 13:03:26.679155 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.679416 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1314,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":3896,"thread_start_us":487659,"threads_started":1,"wal-append.queue_time_us":664459}
I20260709 13:03:26.865566  9385 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
W20260709 13:03:26.520650  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.625976  8752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1926) took 2480 ms (client timeout 2116 ms). Trace:
W20260709 13:03:26.869709  8752 rpcz_store.cc:269] 0709 13:03:24.145746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.157510 (+ 11764us) service_pool.cc:224] Handling call
0709 13:03:26.625612 (+2468102us) spinlock_profiling.cc:243] Waited 882 ms on lock 0x7b540038b3d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:26.625821 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2027720064}
W20260709 13:03:26.509603  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.871327  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.845168  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.579880  8826 raft_consensus.cc:2410] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
W20260709 13:03:26.551545  8722 leader_election.cc:336] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.002s (SENT)
I20260709 13:03:26.872855  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:26.618767  8749 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.000s	sys 0.018s
W20260709 13:03:26.873869  8749 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.007s	sys 0.031s
I20260709 13:03:26.578225 12408 raft_consensus.cc:993] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: : Instructing follower 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:03:26.746577 12378 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.876062 12312 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:26.542232 12349 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.436s	user 0.037s	sys 0.023s
I20260709 13:03:26.644312 12390 raft_consensus.cc:993] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: : Instructing follower 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:03:26.603715 12470 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.479355 12420 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:26.464982 12395 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:26.498524 12393 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.523568 12325 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.504033 12301 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:26.629489 12370 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:26.730101 12371 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.815s	user 0.012s	sys 0.003s
I20260709 13:03:26.523001 12403 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:26.858222 12397 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:26.858562 12334 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:26.643535  8681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 592) took 1643 ms (client timeout 1713 ms). Trace:
I20260709 13:03:26.711489 12266 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.560948 12464 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.342s
I20260709 13:03:26.711844 12323 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:26.728472 12382 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:27.069648  8829 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.007s	sys 0.046s
I20260709 13:03:26.476900 12273 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:26.889567 12364 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:26.909382 12349 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.803s	user 0.052s	sys 0.037s
W20260709 13:03:26.980729  8681 rpcz_store.cc:269] 0709 13:03:25.000470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.052085 (+ 51615us) service_pool.cc:224] Handling call
0709 13:03:26.643197 (+1591112us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400430dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:26.643402 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353158400}
W20260709 13:03:27.044615 12280 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.014s	sys 0.030s
I20260709 13:03:26.950830 12470 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:27.070204  8829 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.008s	sys 0.047s
I20260709 13:03:26.912300 12362 raft_consensus.cc:1081] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Signalling peer 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
W20260709 13:03:27.106791  8674 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.830s	user 0.026s	sys 0.061s
W20260709 13:03:27.149179  8674 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.026s	sys 0.062s
I20260709 13:03:26.973628 12393 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:27.007536 12371 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:26.916695 12363 raft_consensus.cc:1081] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Signalling peer 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
W20260709 13:03:27.116178  8672 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.017s	sys 0.009s
W20260709 13:03:27.078172 12409 consensus_meta.cc:287] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.738s	user 0.008s	sys 0.004s
W20260709 13:03:27.007788 12262 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.687s	user 0.011s	sys 0.004s
I20260709 13:03:26.972157 12427 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:27.192791  8672 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.018s	sys 0.009s
W20260709 13:03:27.198894  8825 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.023s	sys 0.053s
W20260709 13:03:27.206911  8825 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.023s	sys 0.053s
I20260709 13:03:26.909404 12378 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:27.135982 12402 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.014s	sys 0.039s
W20260709 13:03:27.180673 12410 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.034s	sys 0.041s
I20260709 13:03:26.912182 12312 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.100438 12323 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:27.148788 12280 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.980s	user 0.014s	sys 0.035s
W20260709 13:03:27.202064 12438 log.cc:927] Time spent T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:03:27.181311 12383 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:27.052831 12403 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:26.982652 12370 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.192142 12371 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:27.070353 12325 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:27.074661 12266 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.070065 12397 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.271160 12395 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:27.275156 12403 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:27.276157 12383 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.277858 12370 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:27.278209  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1947) took 2360 ms. Trace:
W20260709 13:03:27.231156 12410 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.034s	sys 0.042s
W20260709 13:03:27.289021  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.305812  8753 rpcz_store.cc:276] 0709 13:03:24.917583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.981511 (+ 63928us) service_pool.cc:224] Handling call
0709 13:03:24.982079 (+   568us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:03:25.059063 (+ 76984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.059093 (+    30us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:03:25.063132 (+  4039us) log.cc:844] Serialized 104 byte log entry
0709 13:03:25.256331 (+193199us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:25.282263 (+ 25932us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.303215 (+ 20952us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.305701 (+  2486us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.305890 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.305975 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.368266 (+ 62291us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b1800373a20. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f
0709 13:03:25.369462 (+  1196us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b48001e03d8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:03:26.138603 (+769141us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b1800373a20. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:03:26.648298 (+509695us) spinlock_profiling.cc:243] Waited 4.4 ms on lock 0x7b1800373a20. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1
0709 13:03:26.650266 (+  1968us) spinlock_profiling.cc:243] Waited 34 ms on lock 0x7b1800373a20. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:03:27.277915 (+627649us) raft_consensus.cc:1685] finished
0709 13:03:27.277936 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.278141 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24507,"prepare.queue_time_us":81003,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":2290,"raft.queue_time_us":6126,"raft.run_cpu_time_us":3735,"raft.run_wall_time_us":96463,"spinlock_wait_cycles":248923520,"thread_start_us":92867,"threads_started":1,"wal-append.queue_time_us":179179}
I20260709 13:03:27.122756 12364 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:27.315402 12364 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:27.107617 12382 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.214259 12427 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:27.224033 12402 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.066s	user 0.015s	sys 0.040s
W20260709 13:03:27.306325 12353 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.920s	user 0.011s	sys 0.000s
W20260709 13:03:27.329533 12479 log.cc:927] Time spent T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.533s	user 0.000s	sys 0.002s
I20260709 13:03:27.340226 12470 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:27.383227 12378 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:27.384426 12378 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.385210 12378 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.376420 12353 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:27.376940 12397 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:27.389971 12418 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.028s	sys 0.045s
W20260709 13:03:27.404021  8644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.012s	sys 0.022s
W20260709 13:03:27.405025  8722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.022s	sys 0.011s
W20260709 13:03:27.423784  8722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.022s	sys 0.011s
I20260709 13:03:27.423844 12427 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:27.410518  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.017s	sys 0.019s
W20260709 13:03:27.425532  8677 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.115s	sys 0.168s
W20260709 13:03:27.425334  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.017s	sys 0.019s
W20260709 13:03:27.419858  8721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.004s	sys 0.032s
W20260709 13:03:27.426993  8721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.004s	sys 0.032s
W20260709 13:03:27.427179  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.006s	sys 0.027s
W20260709 13:03:27.427561  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.006s	sys 0.027s
I20260709 13:03:27.430339  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:27.432528  8752 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
W20260709 13:03:27.437100  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.439210  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:27.441000  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:27.407467 12418 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.028s	sys 0.045s
W20260709 13:03:27.421818  8794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.014s	sys 0.022s
W20260709 13:03:27.413877  8796 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.001s	sys 0.023s
W20260709 13:03:27.452309  8794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.015s	sys 0.022s
W20260709 13:03:27.452509  8796 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.001s	sys 0.023s
W20260709 13:03:27.424830  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 873314us
W20260709 13:03:27.414439  8719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.026s	sys 0.024s
W20260709 13:03:27.426016  8677 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.115s	sys 0.168s
W20260709 13:03:27.415526  8795 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.000s	sys 0.024s
W20260709 13:03:27.454130  8795 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.000s	sys 0.024s
W20260709 13:03:27.454245  8719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.026s	sys 0.024s
W20260709 13:03:27.454888  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.409752 12353 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:27.455698  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:27.444119  8644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.013s	sys 0.022s
W20260709 13:03:27.459317  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:27.460372  8722 leader_election.cc:336] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.051s (SENT)
I20260709 13:03:27.460875  8722 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:27.464258  8796 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:27.466387  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:27.467363  8756 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:27.443226 12280 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.475250  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:27.438076 12397 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:27.444087 12402 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.437731 12427 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:27.492251  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1810) took 2493 ms. Trace:
I20260709 13:03:27.476102  8752 pending_rounds.cc:85] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 2
I20260709 13:03:27.504804  8752 pending_rounds.cc:107] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:03:27.502949  8674 rpcz_store.cc:276] 0709 13:03:24.998454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.275330 (+1276876us) service_pool.cc:224] Handling call
0709 13:03:26.275768 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.276015 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:27.149769 (+873754us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.473577 (+323808us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.491863 (+ 18286us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:27.491914 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.491997 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.492177 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8068,"prepare.queue_time_us":1103405,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":30836,"thread_start_us":1188537,"threads_started":2}
I20260709 13:03:27.509713  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:27.468454 12081 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:27.513068  8752 raft_consensus.cc:2889] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:27.512334  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:27.462977 12403 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.514302  8752 pending_rounds.cc:107] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:03:27.514669  8752 raft_consensus.cc:2889] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:27.514788  9384 raft_consensus.cc:2410] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:03:27.463238 12395 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.516319  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:27.516748  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:27.517068  8754 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:27.463117 12370 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:03:27.513684  8748 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:03:27.468755 12409 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.503684 12397 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.535786 12409 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.504021 12402 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:27.537057 12353 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:27.537719 12353 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:27.538144 12409 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:27.547943 12409 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.522145 12378 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:27.522497  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:27.542385 12393 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:27.516487 12266 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:27.523348 12497 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:03:27.523806 12488 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:03:27.523986 12499 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:27.476532 12312 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:27.524540 12177 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:27.577149  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:27.458436 12433 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
W20260709 13:03:27.525764 12481 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:03:27.525967  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:27.508391 12427 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.526502  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 655) took 3124 ms. Trace:
I20260709 13:03:27.528205  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2256) took 2608 ms. Trace:
W20260709 13:03:27.529459  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1644) took 3858 ms (client timeout 2121 ms). Trace:
I20260709 13:03:27.499619 12325 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:27.530927 12273 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:27.532006 12426 log.cc:927] Time spent T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 2.474s	user 0.011s	sys 0.000s
I20260709 13:03:27.549558 12398 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:27.532392 12382 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:27.533629 12081 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:27.504504 12422 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:27.534724  9410 debug-util.cc:398] Leaking SignalData structure 0x7b0800162680 after lost signal to thread 11666
I20260709 13:03:27.538743 12389 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:27.578038 12497 raft_consensus.cc:697] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:27.520512 12395 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:27.534499 12403 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:27.566469 12378 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:27.496939 12280 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.547025  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:27.548080  8827 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1827) took 3968 ms (client timeout 2074 ms). Trace:
I20260709 13:03:27.521690  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:27.549892  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:27.551347  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:27.548015  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 630) took 5256 ms. Trace:
I20260709 13:03:27.469574 12323 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:27.554672  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:27.553015  8749 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.000s	sys 0.034s
I20260709 13:03:27.549250 12297 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:27.559551  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:27.560323  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:27.560527  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1932) took 3278 ms. Trace:
I20260709 13:03:27.562842  9389 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:27.563387 12409 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.565256  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:27.565965  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:27.568027  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:27.612404  8821 rpcz_store.cc:276] 0709 13:03:24.402294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.708231 (+305937us) service_pool.cc:224] Handling call
0709 13:03:24.738373 (+ 30142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.763379 (+ 25006us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.763416 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:24.763470 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:24.763516 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.526188 (+2762672us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b54002292d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:27.526245 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.526409 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56773504}
W20260709 13:03:27.524844 12487 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:27.538244 12353 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:27.539547  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 647) took 4375 ms. Trace:
I20260709 13:03:27.612524  8675 rpcz_store.cc:276] 0709 13:03:24.920015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.920153 (+   138us) service_pool.cc:224] Handling call
0709 13:03:26.143394 (+1223241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.159388 (+ 15994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.527986 (+1368598us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400257bd0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:27.528140 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2812463360}
I20260709 13:03:27.625516  8747 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:27.558408  8756 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:27.624837  8748 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:27.629758  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:27.623484  8679 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:03:27.618259  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 644) took 4511 ms. Trace:
I20260709 13:03:27.623544  8754 raft_consensus.cc:2410] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:03:27.623957  9388 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:27.628489  9386 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:27.624104  8825 rpcz_store.cc:276] 0709 13:03:22.291879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.346460 (+ 54581us) service_pool.cc:224] Handling call
0709 13:03:22.370360 (+ 23900us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.370642 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:22.427838 (+ 57196us) log.cc:844] Serialized 14 byte log entry
0709 13:03:23.212910 (+785072us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b1800413920. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:03:26.591906 (+3378996us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.547347 (+955441us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.547545 (+   198us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.547597 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:27.547647 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.547736 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.547952 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4623423,"prepare.queue_time_us":3205355,"prepare.run_cpu_time_us":316,"prepare.run_wall_time_us":6726,"raft.queue_time_us":157909,"raft.run_cpu_time_us":441,"raft.run_wall_time_us":20523,"spinlock_wait_cycles":25252992,"thread_start_us":1025464,"threads_started":3,"wal-append.queue_time_us":1728738,"wal-append.run_cpu_time_us":1152,"wal-append.run_wall_time_us":1002685}
I20260709 13:03:27.612246  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:27.714957  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:27.715696  8748 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:27.614583 12081 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:27.717573  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1655) took 2859 ms (client timeout 2072 ms). Trace:
W20260709 13:03:27.625039  8749 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.001s	sys 0.035s
W20260709 13:03:27.623188  8827 rpcz_store.cc:269] 0709 13:03:23.579484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.580005 (+   521us) service_pool.cc:224] Handling call
0709 13:03:27.547854 (+3967849us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54002283d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:27.548004 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3444488448}
I20260709 13:03:27.626803  8751 rpcz_store.cc:276] 0709 13:03:24.281721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.281892 (+   171us) service_pool.cc:224] Handling call
0709 13:03:24.282306 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.399491 (+1117185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.399527 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.400145 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 13:03:25.427697 (+ 27552us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:25.427769 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.272273 (+844504us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:27.560074 (+1287801us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540031ae54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:27.560146 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.560195 (+    49us) raft_consensus.cc:1685] finished
0709 13:03:27.560205 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.560360 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24856,"raft.queue_time_us":26089,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"spinlock_wait_cycles":2522139904,"thread_start_us":27051,"threads_started":1,"wal-append.queue_time_us":385429,"wal-append.run_cpu_time_us":5873,"wal-append.run_wall_time_us":1547751}
W20260709 13:03:27.612701  8822 rpcz_store.cc:269] 0709 13:03:23.670807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.077163 (+406356us) service_pool.cc:224] Handling call
0709 13:03:27.529233 (+3452070us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b54002297d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:27.529368 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1979456256}
I20260709 13:03:27.657166  8829 rpcz_store.cc:276] 0709 13:03:23.164123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.213891 (+ 49768us) service_pool.cc:224] Handling call
0709 13:03:23.214657 (+   766us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.370620 (+155963us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:26.242635 (+2872015us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b180009c940. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e5ef13c7 00007f32e25039a7 00007f32e25130bf 00007f32e2513038
0709 13:03:26.243034 (+   399us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.149121 (+906087us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.149819 (+   698us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.150048 (+   229us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:27.150103 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.150236 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.539172 (+388936us) raft_consensus.cc:1685] finished
0709 13:03:27.539291 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.539456 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1467205,"prepare.queue_time_us":2816139,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":189698,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":28661,"spinlock_wait_cycles":4170624,"thread_start_us":905508,"threads_started":1,"wal-append.queue_time_us":1249949}
I20260709 13:03:27.658344  9387 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:27.720422  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2331) took 1951 ms. Trace:
W20260709 13:03:27.717995  8821 rpcz_store.cc:269] 0709 13:03:24.858554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.717344 (+2858790us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:27.717492 (+   148us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:27.719264  8827 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1854) took 2568 ms (client timeout 1986 ms). Trace:
I20260709 13:03:27.715751  9383 raft_consensus.cc:2393] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:03:27.721695  8827 rpcz_store.cc:269] 0709 13:03:25.150876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.719039 (+2568163us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:27.719162 (+   123us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:27.719323  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:27.722568  8825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1859) took 2496 ms (client timeout 1777 ms). Trace:
W20260709 13:03:27.723089  8827 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1858) took 2497 ms (client timeout 1964 ms). Trace:
I20260709 13:03:27.715783  9389 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:27.626650  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:27.723315  8825 rpcz_store.cc:269] 0709 13:03:25.226003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.722362 (+2496359us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:27.722491 (+   129us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:27.723469  8827 rpcz_store.cc:269] 0709 13:03:25.225748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.722888 (+2497140us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:27.723001 (+   113us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:27.632328  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:27.724320  8825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 659) took 2654 ms (client timeout 2228 ms). Trace:
W20260709 13:03:27.724897  8825 rpcz_store.cc:269] 0709 13:03:25.069654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.724156 (+2654502us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:27.724260 (+   104us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:27.716598  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:27.728058  8748 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:27.734874  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:27.725816  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:27.736002  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:27.736138  8675 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:27.750972 12353 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:27.721217  8749 rpcz_store.cc:276] 0709 13:03:25.769935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.945751 (+175816us) service_pool.cc:224] Handling call
0709 13:03:25.946164 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.946429 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:26.874348 (+927919us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.719954 (+845606us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.720020 (+    66us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:27.720075 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.720177 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.720341 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":80906,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":2875,"thread_start_us":1724229,"threads_started":2}
I20260709 13:03:27.620985 12395 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:27.719393 12081 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:27.739488  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1666) took 2570 ms (client timeout 2547 ms). Trace:
W20260709 13:03:27.770089  8822 rpcz_store.cc:269] 0709 13:03:25.169538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.739281 (+2569743us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:27.739417 (+   136us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:27.771684  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:27.772842  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1669) took 1298 ms (client timeout 1524 ms). Trace:
I20260709 13:03:27.614892 12422 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.739737  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:27.622895 12280 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:27.610390 12418 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:27.618995 12389 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:27.740319  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:27.781816  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 194943us
I20260709 13:03:27.782289  8829 raft_consensus.cc:2410] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:27.619810 12497 consensus_queue.cc:237] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.713424  8823 rpcz_store.cc:276] 0709 13:03:23.107006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.107200 (+   194us) service_pool.cc:224] Handling call
0709 13:03:23.111607 (+  4407us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:23.367084 (+255477us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:23.440877 (+ 73793us) log.cc:844] Serialized 14 byte log entry
0709 13:03:24.119775 (+678898us) spinlock_profiling.cc:243] Waited 65.6 ms on lock 0x7b48000cd158. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:03:26.256668 (+2136893us) log.cc:844] Serialized 14 byte log entry
0709 13:03:26.619653 (+362985us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:26.620594 (+   941us) log.cc:844] Serialized 52 byte log entry
0709 13:03:26.620768 (+   174us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:27.542896 (+922128us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.543169 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.543351 (+   182us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.617920 (+ 74569us) raft_consensus.cc:1685] finished
0709 13:03:27.617952 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.618168 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6553017,"prepare.queue_time_us":4831430,"prepare.run_cpu_time_us":154,"prepare.run_wall_time_us":2846,"raft.queue_time_us":261403,"raft.run_cpu_time_us":5079,"raft.run_wall_time_us":823636,"spinlock_wait_cycles":150931328,"thread_start_us":601348,"threads_started":2,"wal-append.queue_time_us":1368579,"wal-append.run_cpu_time_us":3092,"wal-append.run_wall_time_us":1319552}
I20260709 13:03:27.737969 12334 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.749651  8750 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:27.629659 12409 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:27.751500  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:27.737841  8827 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1656) took 2864 ms (client timeout 2825 ms). Trace:
W20260709 13:03:27.802933  8827 rpcz_store.cc:269] 0709 13:03:24.859466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.724201 (+2864735us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:27.724276 (+    75us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:27.620999 12403 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:27.614049 12427 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:27.773306  8822 rpcz_store.cc:269] 0709 13:03:26.474117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.771235 (+1297118us) service_pool.cc:224] Handling call
0709 13:03:27.772756 (+  1521us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:27.775431  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:27.777954  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:27.784655  8829 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1664) took 2755 ms (client timeout 2923 ms). Trace:
I20260709 13:03:27.614089 12398 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:27.811303  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:27.812297  8823 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:27.753522  8756 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
W20260709 13:03:27.815280  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 352816us
I20260709 13:03:27.805768  8748 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:03:27.807874  8829 rpcz_store.cc:269] 0709 13:03:25.028825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.740035 (+2711210us) service_pool.cc:224] Handling call
0709 13:03:27.784555 (+ 44520us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:27.808101  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:27.804675  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.818286  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:27.825289  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:27.826500  8674 raft_consensus.cc:2368] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11.
I20260709 13:03:27.819164  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1816) took 2819 ms. Trace:
I20260709 13:03:27.822819  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:27.828776  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:27.828090  8647 connection.cc:256] RPC call timeout handler was delayed by -2.7346154169999863s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16410000000000002s for a response
W20260709 13:03:27.829869  8647 connection.cc:256] RPC call timeout handler was delayed by -2.6599154169999792s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 13:03:27.830231  8647 connection.cc:256] RPC call timeout handler was delayed by -2.6320154169999341s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1755s for a response
I20260709 13:03:27.830104  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:27.828575  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1668) took 1354 ms. Trace:
I20260709 13:03:27.821110  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:27.831010  8829 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:27.831593  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1670) took 1357 ms. Trace:
I20260709 13:03:27.832101  8827 rpcz_store.cc:276] 0709 13:03:26.474371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.803958 (+1329587us) service_pool.cc:224] Handling call
0709 13:03:27.831513 (+ 27555us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:27.832226  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:27.830994  8823 rpcz_store.cc:276] 0709 13:03:26.473820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.783380 (+1309560us) service_pool.cc:224] Handling call
0709 13:03:27.828502 (+ 45122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:27.834225  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:27.830657  8677 rpcz_store.cc:276] 0709 13:03:24.999983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.627260 (+1627277us) service_pool.cc:224] Handling call
0709 13:03:26.627751 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.628051 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:27.458716 (+830665us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.818746 (+360030us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.818807 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:27.818865 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.818934 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.819095 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11181,"prepare.queue_time_us":775436,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":2444,"raft.queue_time_us":12361,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":13112,"thread_start_us":1173273,"threads_started":2}
W20260709 13:03:27.933674  8719 leader_election.cc:336] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.208s (SENT)
I20260709 13:03:27.934427  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:27.842689  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.835114  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:27.925709  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:27.938566  8823 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:27.835080  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 660) took 1359 ms. Trace:
I20260709 13:03:27.940412  8822 rpcz_store.cc:276] 0709 13:03:26.475385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.806938 (+1331553us) service_pool.cc:224] Handling call
0709 13:03:27.835015 (+ 28077us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:27.940930  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:27.941653  8675 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:27.891851  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:27.875283  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.942598  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:27.942978  8827 raft_consensus.cc:2393] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:27.801036 12334 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:27.999541  8722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.006s	sys 0.020s
W20260709 13:03:27.840880  8647 connection.cc:256] RPC call timeout handler was delayed by -2.615077862000021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2828s for a response
I20260709 13:03:28.001166  8756 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:28.001010  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.002136  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:27.878539 12502 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:03:27.999624 12353 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.003834 12492 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:28.031039  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:27.953905 12389 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:28.004681  8754 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:03:28.008620  8722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.007s	sys 0.021s
W20260709 13:03:28.220842  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 760493us
W20260709 13:03:28.008862  8647 connection.cc:256] RPC call timeout handler was delayed by -2.4421154169999681s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1966s for a response
W20260709 13:03:28.222446  8647 connection.cc:256] RPC call timeout handler was delayed by -2.3197154169999976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
W20260709 13:03:28.222863  8647 connection.cc:256] RPC call timeout handler was delayed by -2.3071154169999772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2116s for a response
I20260709 13:03:28.222985  8750 raft_consensus.cc:2410] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:03:28.223259  8647 connection.cc:256] RPC call timeout handler was delayed by -2.3053154169999743s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2118s for a response
W20260709 13:03:28.223786  8647 connection.cc:256] RPC call timeout handler was delayed by -2.2819154169999365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21440000000000003s for a response
W20260709 13:03:28.224124  8647 connection.cc:256] RPC call timeout handler was delayed by -2.2819154169999365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21440000000000003s for a response
W20260709 13:03:28.224443  8647 connection.cc:256] RPC call timeout handler was delayed by -2.2675154170000269s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21600000000000003s for a response
W20260709 13:03:28.224753  8647 connection.cc:256] RPC call timeout handler was delayed by -2.23241541699997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21989999999999998s for a response
W20260709 13:03:28.225059  8647 connection.cc:256] RPC call timeout handler was delayed by -2.23241541699997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21989999999999998s for a response
W20260709 13:03:28.225370  8647 connection.cc:256] RPC call timeout handler was delayed by -2.1920431539999754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1781s for a response
W20260709 13:03:28.225669  8647 connection.cc:256] RPC call timeout handler was delayed by -2.1658154169999762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22730000000000003s for a response
W20260709 13:03:28.225970  8647 connection.cc:256] RPC call timeout handler was delayed by -2.1658154169999762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22730000000000003s for a response
W20260709 13:03:28.226279  8647 connection.cc:256] RPC call timeout handler was delayed by -2.1442154169999412s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22970000000000002s for a response
I20260709 13:03:28.026118  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:28.228286  8647 connection.cc:256] RPC call timeout handler was delayed by -1.9199125679999725s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
W20260709 13:03:28.229153  8647 connection.cc:256] RPC call timeout handler was delayed by -1.7545154169999932s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.273s for a response
W20260709 13:03:28.230075  8647 connection.cc:256] RPC call timeout handler was delayed by -1.6301399029999857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
W20260709 13:03:28.230531  8647 connection.cc:256] RPC call timeout handler was delayed by -1.6249154170000111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28740000000000004s for a response
W20260709 13:03:28.231001  8647 connection.cc:256] RPC call timeout handler was delayed by -1.6108936360000143s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
I20260709 13:03:28.043354 12378 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.044015 12422 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:28.046036  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 591) took 3045 ms. Trace:
I20260709 13:03:28.046494  8747 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:28.049213 12334 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:27.999760 12325 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:28.055074  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:28.056835  8829 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:28.003638 12382 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:28.063264  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:28.065181 11711 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.066696  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:28.257433  8822 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:28.257637  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 191117us
I20260709 13:03:28.258824  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:28.069386 12504 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 13:03:28.026454  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:28.189813  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.852816  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:28.199800  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 59948.5us
I20260709 13:03:28.259986  8794 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:28.239943 12280 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:28.009610  8675 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:28.271617  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:28.274320  8675 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:28.041729 12361 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:28.231623  8647 connection.cc:256] RPC call timeout handler was delayed by -1.5781154169999354s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2926s for a response
I20260709 13:03:28.233886  8674 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:28.227631  8677 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
W20260709 13:03:28.275871  9355 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.010s	sys 0.018s
W20260709 13:03:28.028791 12516 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 13:03:28.242046  9380 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.016s	sys 0.011s
I20260709 13:03:28.244695  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:28.246690  8681 rpcz_store.cc:276] 0709 13:03:25.000245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.183254 (+2183009us) service_pool.cc:224] Handling call
0709 13:03:27.214293 (+ 31039us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.232324 (+ 18031us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:27.232379 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.250106 (+ 17727us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.652227 (+402121us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:27.652296 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.652381 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.045743 (+393362us) raft_consensus.cc:1685] finished
0709 13:03:28.045768 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.045936 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17581,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":10132,"thread_start_us":371436,"threads_started":1,"wal-append.queue_time_us":632100}
I20260709 13:03:27.860841 12427 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.800052 12418 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:28.267720 12262 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.019s	sys 0.009s
I20260709 13:03:28.250524  8749 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:28.241444 12353 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:28.266572  8748 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:28.025348  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 657) took 3551 ms. Trace:
W20260709 13:03:28.267084  8672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.002s	sys 0.026s
I20260709 13:03:28.301227  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:28.301546  8672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.002s	sys 0.026s
W20260709 13:03:28.267431  8796 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.008s	sys 0.020s
I20260709 13:03:28.278354 12334 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:03:28.277390  9355 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.011s	sys 0.019s
W20260709 13:03:28.305416  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 556130us
I20260709 13:03:28.277436  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:28.307483  8747 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
W20260709 13:03:28.278393  8647 connection.cc:256] RPC call timeout handler was delayed by -1.4425936359999696s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
W20260709 13:03:28.280719  9380 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.016s	sys 0.011s
I20260709 13:03:28.283530  8823 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:03:28.284780  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:28.309103  8677 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:28.250478 12273 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:28.287204  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:28.290051 12491 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.008s	sys 0.021s
W20260709 13:03:28.250345  8722 leader_election.cc:336] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.072s (SENT)
I20260709 13:03:28.318529  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:28.264412  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1653) took 3406 ms. Trace:
I20260709 13:03:28.270385 12411 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:28.247641 12378 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:28.302457  8796 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.008s	sys 0.020s
I20260709 13:03:28.273298  8793 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:28.325295  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 52101.5us
I20260709 13:03:28.300396  8830 rpcz_store.cc:276] 0709 13:03:24.474239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.746019 (+271780us) service_pool.cc:224] Handling call
0709 13:03:24.746339 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.752829 (+  6490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.752863 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:24.752908 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:24.752940 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.024943 (+3272003us) spinlock_profiling.cc:243] Waited 6.23 ms on lock 0x7b5400267ad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:28.025023 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.025250 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14340352}
I20260709 13:03:28.326002  8675 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:28.326102  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:28.326916  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 76604.8us
W20260709 13:03:28.310006  8647 connection.cc:256] RPC call timeout handler was delayed by -1.1273431539999592s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2964s for a response
W20260709 13:03:28.297530 12262 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.019s	sys 0.009s
I20260709 13:03:28.298548 12393 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:28.265061 12382 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:28.316474  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1659) took 3456 ms. Trace:
I20260709 13:03:28.332738  8823 rpcz_store.cc:276] 0709 13:03:24.860261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.257041 (+3396780us) service_pool.cc:224] Handling call
0709 13:03:28.276724 (+ 19683us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.316073 (+ 39349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.316377 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:28.333535  8749 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:28.317637  8751 raft_consensus.cc:2410] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:03:28.319455  8822 rpcz_store.cc:276] 0709 13:03:24.858005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.243255 (+3385250us) service_pool.cc:224] Handling call
0709 13:03:28.257229 (+ 13974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.264123 (+  6894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.264352 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:28.318210 12491 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.008s	sys 0.021s
I20260709 13:03:28.331128 12371 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:28.313314 12389 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:28.297670 12427 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:28.321612  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:28.347275  8754 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:28.348116  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.313012  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:28.267565 11711 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:28.281821 12280 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:28.324703  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 633555us
I20260709 13:03:28.279387 12422 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:03:28.305284 12334 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.321419  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:28.321547 12411 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.332826 12496 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:28.352792  8796 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:28.353420  8823 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:28.352579  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:28.369915  8675 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:28.331074 12393 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.246886 12492 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:28.361040  8747 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:28.365542  8822 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:28.343753 12371 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.343930 12349 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:28.329887 12511 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:03:28.371716  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:28.380609  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1853) took 3230 ms. Trace:
I20260709 13:03:28.322196 12273 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:28.388480  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:28.371971  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1851) took 3239 ms. Trace:
I20260709 13:03:28.357892 12409 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.418846  8822 rpcz_store.cc:276] 0709 13:03:25.150480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.334942 (+3184462us) service_pool.cc:224] Handling call
0709 13:03:28.357904 (+ 22962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.380183 (+ 22279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.380528 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:28.419793  8823 rpcz_store.cc:276] 0709 13:03:25.133023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.333778 (+3200755us) service_pool.cc:224] Handling call
0709 13:03:28.352583 (+ 18805us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.371407 (+ 18824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.371898 (+   491us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:28.421566  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.362183 12334 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:28.371343 12411 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:28.436450  8675 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:28.380303 12297 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:28.444514  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1647) took 4704 ms. Trace:
I20260709 13:03:28.351248 12389 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:28.443877  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.367274 12496 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:28.381131 12393 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:28.444972  8824 rpcz_store.cc:276] 0709 13:03:23.739848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.378027 (+638179us) service_pool.cc:224] Handling call
0709 13:03:24.399177 (+ 21150us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:26.013572 (+1614395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.444211 (+2430639us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54003ac0d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:28.444446 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3710843776}
I20260709 13:03:28.381826 12371 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:28.452665  8823 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:28.472518  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1856) took 3321 ms. Trace:
I20260709 13:03:28.473178  8823 rpcz_store.cc:276] 0709 13:03:25.151519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.432908 (+3281389us) service_pool.cc:224] Handling call
0709 13:03:28.452386 (+ 19478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.472069 (+ 19683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.472406 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:28.475034  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.478911  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:28.474437  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.450171 12297 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.450163 12403 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:28.457110 12496 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:28.445179 12433 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.482854  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.437896 12409 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:28.492730  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:28.494524  8681 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:28.498011  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.499070  8675 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:28.499158  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.499843  8681 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:28.500712  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:28.501253  8681 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:28.491389  8823 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:03:28.502951  8823 pending_rounds.cc:85] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 5
I20260709 13:03:28.503252  8823 pending_rounds.cc:107] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:03:28.503520  8823 raft_consensus.cc:2889] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:28.504294  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1667) took 3101 ms. Trace:
I20260709 13:03:28.504608  8823 rpcz_store.cc:276] 0709 13:03:25.403142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.480834 (+3077692us) service_pool.cc:224] Handling call
0709 13:03:28.490861 (+ 10027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.504002 (+ 13141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.504230 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:28.488714 12297 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:28.511389  8796 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:28.511592 12461 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:28.519487 12461 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:28.519675 12525 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:03:28.519938 12461 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:28.521217  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 658) took 3667 ms. Trace:
I20260709 13:03:28.521780  8827 rpcz_store.cc:276] 0709 13:03:24.853876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.050377 (+3196501us) service_pool.cc:224] Handling call
0709 13:03:28.051023 (+   646us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.209948 (+158925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:28.210040 (+    92us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:28.220136 (+ 10096us) log.cc:844] Serialized 26 byte log entry
0709 13:03:28.296756 (+ 76620us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:28.296815 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.296922 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.520956 (+224034us) raft_consensus.cc:1685] finished
0709 13:03:28.520979 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.521143 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":159861,"raft.queue_time_us":158471,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":22043,"thread_start_us":34594,"threads_started":1,"wal-append.queue_time_us":113384}
I20260709 13:03:28.526957  8823 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:28.527863  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 661) took 2052 ms. Trace:
I20260709 13:03:28.528391  8824 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:28.528345  8823 rpcz_store.cc:276] 0709 13:03:26.475674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.512009 (+2036335us) service_pool.cc:224] Handling call
0709 13:03:28.526580 (+ 14571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.527470 (+   890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.527709 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:28.529667  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.530845  8675 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
W20260709 13:03:28.532069  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 225035us
I20260709 13:03:28.535665  8681 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:28.536286  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.511524 12510 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.543983  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.537019  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.495298 12403 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:28.552655  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.514356 12507 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:28.535282 12508 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:03:28.550565  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1814) took 3551 ms. Trace:
I20260709 13:03:28.545228  8675 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:28.561940  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1832) took 1013 ms. Trace:
I20260709 13:03:28.562388  8675 rpcz_store.cc:276] 0709 13:03:27.548956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.537764 (+988808us) service_pool.cc:224] Handling call
0709 13:03:28.561857 (+ 24093us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:28.552150  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 195065us
W20260709 13:03:28.559193 12521 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 13:03:28.547263  8681 raft_consensus.cc:2410] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:03:28.508232 12433 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:28.585976  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.587707  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.588503  8681 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:28.589083  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1830) took 1058 ms. Trace:
I20260709 13:03:28.589468  8681 rpcz_store.cc:276] 0709 13:03:27.530474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.587414 (+1056940us) service_pool.cc:224] Handling call
0709 13:03:28.589016 (+  1602us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:28.549387 12509 log.cc:927] Time spent T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
W20260709 13:03:28.536809  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 223914us
I20260709 13:03:28.575490  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:28.541654  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 662) took 2065 ms. Trace:
I20260709 13:03:28.577104  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:28.564592 12378 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:28.553097 12418 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:28.551339 12262 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:28.564358 12507 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:28.551494 12510 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:28.821388 12382 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:29.178510  8722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.023s	sys 0.030s
I20260709 13:03:28.785627  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2337) took 1357 ms. Trace:
W20260709 13:03:29.181223  9354 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.003s	sys 0.008s
W20260709 13:03:29.235878  9354 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.004s	sys 0.008s
W20260709 13:03:29.204851  9381 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.725s	user 0.014s	sys 0.000s
I20260709 13:03:29.236770  9381 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
W20260709 13:03:29.237488  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.716773  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2340) took 1269 ms. Trace:
I20260709 13:03:29.235610  8753 rpcz_store.cc:276] 0709 13:03:27.428671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.428847 (+   176us) service_pool.cc:224] Handling call
0709 13:03:27.429296 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.491959 (+ 62663us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:27.507768 (+ 15809us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.835979 (+328211us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.836232 (+   253us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.836298 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:27.836370 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.785320 (+948950us) spinlock_profiling.cc:243] Waited 62.4 ms on lock 0x7b5400138154. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:28.785373 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.785549 (+   176us) inbound_call.cc:177] Queueing success response
0709 13:03:28.803376 (+ 17827us) spinlock_profiling.cc:243] Waited 7.55 ms on lock 0x7b180035c260. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
Metrics: {"prepare.queue_time_us":32528,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":106,"raft.queue_time_us":17162,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":18486,"spinlock_wait_cycles":177536000,"thread_start_us":327389,"threads_started":1,"wal-append.queue_time_us":791788}
I20260709 13:03:29.239099  8752 rpcz_store.cc:276] 0709 13:03:27.450532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.475264 (+ 24732us) service_pool.cc:224] Handling call
0709 13:03:27.475769 (+   505us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:27.925754 (+449985us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.925800 (+    46us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:27.926857 (+  1057us) log.cc:844] Serialized 78 byte log entry
0709 13:03:28.381803 (+454946us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:03:28.381854 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.381939 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.716476 (+334537us) raft_consensus.cc:1685] finished
0709 13:03:28.716509 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.716700 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10080,"raft.queue_time_us":61852,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":28028,"thread_start_us":855374,"threads_started":2,"wal-append.queue_time_us":635509}
I20260709 13:03:28.889446  8568 maintenance_manager.cc:419] P 06d3808b553142ef8f7a4e57845757cd: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033399
I20260709 13:03:29.552271  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:29.553433  8752 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 5.
W20260709 13:03:29.481876  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 906260us
I20260709 13:03:28.561690  8672 rpcz_store.cc:276] 0709 13:03:24.999487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.302849 (+1303362us) service_pool.cc:224] Handling call
0709 13:03:26.319892 (+ 17043us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.320435 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:27.206725 (+886290us) log.cc:844] Serialized 14 byte log entry
0709 13:03:28.549700 (+1342975us) log.cc:844] Serialized 14 byte log entry
0709 13:03:28.550029 (+   329us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:28.550102 (+    73us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:28.550191 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.550291 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.550477 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70996,"prepare.queue_time_us":1462719,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":121,"raft.queue_time_us":109943,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":9572,"thread_start_us":2141428,"threads_started":4,"wal-append.queue_time_us":538641}
I20260709 13:03:28.625732  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2274) took 1107 ms. Trace:
I20260709 13:03:29.557667  8675 rpcz_store.cc:276] 0709 13:03:27.518374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.575320 (+1056946us) service_pool.cc:224] Handling call
0709 13:03:28.625648 (+ 50328us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.559736  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:29.559930  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:28.661895  8677 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:03:28.590552  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:29.561802  8677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 601) took 2063 ms (client timeout 2113 ms). Trace:
I20260709 13:03:29.561976  8681 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
W20260709 13:03:29.020740  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:29.234637  8722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.023s	sys 0.031s
I20260709 13:03:29.644815  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:29.528477  8747 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.007s	sys 0.004s
W20260709 13:03:29.650699  8747 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.007s	sys 0.005s
W20260709 13:03:29.303194  8796 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.034s	sys 0.138s
W20260709 13:03:29.698171  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:29.308705  8829 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.009s	sys 0.030s
W20260709 13:03:29.282923  8830 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.002s	sys 0.015s
I20260709 13:03:28.704412  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 664) took 1240 ms. Trace:
I20260709 13:03:29.699034 12405 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:29.727902  8825 rpcz_store.cc:276] 0709 13:03:27.463838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.674487 (+1210649us) service_pool.cc:224] Handling call
0709 13:03:28.703614 (+ 29127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.703909 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:28.703945 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:28.704008 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:28.704072 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.704147 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.704315 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:29.583473  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 813804us
W20260709 13:03:28.670958  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 75641.6us
I20260709 13:03:29.726190  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1973) took 1509 ms. Trace:
I20260709 13:03:29.735746  8747 rpcz_store.cc:276] 0709 13:03:28.213189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.805303 (+592114us) service_pool.cc:224] Handling call
0709 13:03:28.805709 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.806197 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:28.806241 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:28.809032 (+  2791us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.703046 (+894014us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:29.703151 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.703294 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.703349 (+    55us) raft_consensus.cc:1685] finished
0709 13:03:29.703364 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.722826 (+ 19462us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20080,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":4394,"thread_start_us":676197,"threads_started":1,"wal-append.queue_time_us":564478}
W20260709 13:03:29.285972  9381 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 602) took 4684 ms (client timeout 1795 ms). Trace:
W20260709 13:03:29.273567 11276 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.012s	sys 0.004s
W20260709 13:03:29.738794 11276 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.290s	user 0.014s	sys 0.004s
W20260709 13:03:29.738531  9381 rpcz_store.cc:269] 0709 13:03:24.552932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.553700 (+   768us) service_pool.cc:224] Handling call
0709 13:03:28.478742 (+3925042us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:03:29.237376 (+758634us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8450432}
I20260709 13:03:28.715392 11711 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:29.623466  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.626324  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1822) took 2098 ms. Trace:
I20260709 13:03:29.742931  8681 rpcz_store.cc:276] 0709 13:03:27.527875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.590261 (+1062386us) service_pool.cc:224] Handling call
0709 13:03:29.626184 (+1035923us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.745518  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:29.562433  8677 rpcz_store.cc:269] 0709 13:03:27.498167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.284458 (+786291us) service_pool.cc:224] Handling call
0709 13:03:29.561460 (+1277002us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.748005  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:29.749728  8677 raft_consensus.cc:2410] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:03:29.760574  8830 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.381s	user 0.003s	sys 0.015s
I20260709 13:03:29.762475 12383 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:29.651644  8749 raft_consensus.cc:2410] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:29.562721  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1833) took 2004 ms. Trace:
I20260709 13:03:28.715966 12273 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:28.583974 12280 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:29.235563 12297 consensus_meta.cc:287] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.715s	user 0.010s	sys 0.001s
I20260709 13:03:29.780689 12297 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:29.153112 12349 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.783560 12349 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:29.323609 12361 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.034s	user 0.011s	sys 0.003s
I20260709 13:03:28.807070 12389 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:29.387715 12393 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.909s	user 0.005s	sys 0.009s
I20260709 13:03:29.788822 12458 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.789601 12393 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:29.789789 12458 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:29.791592  9382 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:28.788400 12397 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:29.232784 12398 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:28.800823 12403 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:28.715220 12409 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:29.390025 12411 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.936s	user 0.013s	sys 0.000s
I20260709 13:03:29.796007 12411 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:28.671494 12427 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:29.332866 12433 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:29.225229 12461 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.705s	user 0.010s	sys 0.003s
I20260709 13:03:28.719399 12492 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:29.800509 12461 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:29.390952 12496 consensus_meta.cc:287] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.883s	user 0.010s	sys 0.004s
I20260709 13:03:29.802698 12496 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:29.803248 12393 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:29.805549 12496 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:28.632640 12501 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:03:29.392409 12507 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:29.393584 12510 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:28.775424 12521 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 13:03:28.717487 12523 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:03:29.807688  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:29.807056 12461 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:29.808826  9388 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:29.348800 12542 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:29.696844 12425 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 4.559s	user 0.007s	sys 0.033s
W20260709 13:03:29.698567  8796 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.181s	user 0.035s	sys 0.139s
I20260709 13:03:29.811951  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2259) took 4889 ms. Trace:
I20260709 13:03:29.812659  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:28.613020  8827 rpcz_store.cc:276] 0709 13:03:26.475920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.524098 (+2048178us) service_pool.cc:224] Handling call
0709 13:03:28.540848 (+ 16750us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.541147 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:28.541184 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:28.541247 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:28.541311 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.541383 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.541561 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.813732  9384 raft_consensus.cc:2376] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11. Re-sending same reply.
I20260709 13:03:29.812481  8678 rpcz_store.cc:276] 0709 13:03:24.922275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.922396 (+   121us) service_pool.cc:224] Handling call
0709 13:03:24.937915 (+ 15519us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:24.938420 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.938455 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:24.939183 (+   728us) log.cc:844] Serialized 52 byte log entry
0709 13:03:25.058628 (+119445us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:25.058681 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.058781 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.692026 (+4633245us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:29.811114 (+119088us) raft_consensus.cc:1685] finished
0709 13:03:29.811136 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.811651 (+   515us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3599,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":1435,"spinlock_wait_cycles":36276608,"thread_start_us":118984,"threads_started":1,"wal-append.queue_time_us":197478}
I20260709 13:03:29.814558  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:29.815351  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:29.815696  8827 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:29.816291  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2281) took 2079 ms. Trace:
I20260709 13:03:29.816742  8678 rpcz_store.cc:276] 0709 13:03:27.737109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.815006 (+2077897us) service_pool.cc:224] Handling call
0709 13:03:29.816201 (+  1195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.817081  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:29.818030  8827 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:29.783632  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:29.712020  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 155292us
W20260709 13:03:29.813138  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 1.30181e+06us
I20260709 13:03:29.723639 12378 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:29.822647  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:29.823205 12378 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:29.824262 12433 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:29.825701 12411 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:29.726949  8829 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.162s	user 0.009s	sys 0.031s
I20260709 13:03:29.826596  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:29.827498 12411 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:29.827715  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1658) took 4967 ms. Trace:
I20260709 13:03:29.627951  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:29.763795  8672 rpcz_store.cc:276] 0709 13:03:27.557171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.558845 (+2001674us) service_pool.cc:224] Handling call
0709 13:03:29.561023 (+  2178us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:29.762809  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 49320.8us
I20260709 13:03:29.750463  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2293) took 1710 ms. Trace:
I20260709 13:03:29.828049  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:29.699041 12418 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:29.827836 12433 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:29.770514  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2297) took 1728 ms. Trace:
I20260709 13:03:29.832077  8681 rpcz_store.cc:276] 0709 13:03:28.040847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.744823 (+1703976us) service_pool.cc:224] Handling call
0709 13:03:29.769190 (+ 24367us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.805585 12297 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:29.830499  8677 rpcz_store.cc:276] 0709 13:03:28.039795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.747471 (+1707676us) service_pool.cc:224] Handling call
0709 13:03:29.750373 (+  2902us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.834133  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:29.834363  8827 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:03:29.839670 12543 log.cc:927] Time spent T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.064s	user 0.000s	sys 0.002s
I20260709 13:03:29.828164  8829 rpcz_store.cc:276] 0709 13:03:24.860009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.251155 (+3391146us) service_pool.cc:224] Handling call
0709 13:03:28.251576 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.254577 (+  3001us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:28.254628 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:28.255221 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 13:03:28.512280 (+257059us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:28.548655 (+ 36375us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.827193 (+1278538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.827340 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.827446 (+   106us) raft_consensus.cc:1685] finished
0709 13:03:29.827460 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.827626 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7534,"prepare.queue_time_us":36008,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":4105,"raft.queue_time_us":812986,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":12984,"thread_start_us":1490905,"threads_started":2,"wal-append.queue_time_us":292615,"wal-append.run_cpu_time_us":3020,"wal-append.run_wall_time_us":1146526}
I20260709 13:03:29.839308 12411 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:29.838702  8672 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:29.849336  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:29.850019  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:29.849419  8752 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:29.854979  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1660) took 4994 ms. Trace:
I20260709 13:03:29.855471  8830 rpcz_store.cc:276] 0709 13:03:24.863898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.330777 (+3466879us) service_pool.cc:224] Handling call
0709 13:03:28.346054 (+ 15277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:28.346472 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:28.346505 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:28.362394 (+ 15889us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.761468 (+1399074us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:29.840816 (+ 79348us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.842160 (+  1344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.842300 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.842376 (+    76us) raft_consensus.cc:1685] finished
0709 13:03:29.842392 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.854907 (+ 12515us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79705,"prepare.queue_time_us":83967,"raft.queue_time_us":2801,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":5831,"thread_start_us":1382115,"threads_started":1,"wal-append.queue_time_us":968631}
I20260709 13:03:29.858093  8830 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:29.856755  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1860) took 2309 ms. Trace:
I20260709 13:03:29.858646  8829 rpcz_store.cc:276] 0709 13:03:27.546928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.855585 (+2308657us) service_pool.cc:224] Handling call
0709 13:03:29.856053 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.856401 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.856427 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:29.856489 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:29.856521 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.856573 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.856677 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.826413 12378 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:29.861987  8750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2347) took 2302 ms (client timeout 1835 ms). Trace:
I20260709 13:03:29.863180  8753 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:29.864742 12550 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:29.858682  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 593) took 4851 ms. Trace:
I20260709 13:03:29.869992  8672 rpcz_store.cc:276] 0709 13:03:25.000700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.828703 (+4828003us) service_pool.cc:224] Handling call
0709 13:03:29.838419 (+  9716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.851894 (+ 13475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.852091 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.830505 12349 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:29.866480  8647 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:29.870272  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1863) took 2320 ms. Trace:
I20260709 13:03:29.873631  8830 rpcz_store.cc:276] 0709 13:03:27.547762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.857414 (+2309652us) service_pool.cc:224] Handling call
0709 13:03:29.857802 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.867770 (+  9968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.868365 (+   595us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.875238  8830 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:29.875787  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1870) took 2290 ms. Trace:
W20260709 13:03:29.867326  8750 rpcz_store.cc:269] 0709 13:03:27.559580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.561888 (+  2308us) service_pool.cc:224] Handling call
0709 13:03:29.861445 (+2299557us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540034ba54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:29.861719 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":394834688}
W20260709 13:03:29.877825 12549 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:03:29.879851  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2265) took 4712 ms. Trace:
I20260709 13:03:29.876056  8830 rpcz_store.cc:276] 0709 13:03:27.586021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.874356 (+2288335us) service_pool.cc:224] Handling call
0709 13:03:29.874831 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.875573 (+   742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.875735 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.880443  8672 rpcz_store.cc:276] 0709 13:03:25.167292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.878256 (+4710964us) service_pool.cc:224] Handling call
0709 13:03:29.878862 (+   606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.879330 (+   468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.879375 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:29.879441 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:29.879497 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.879583 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.879747 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.893445  8827 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:29.896186  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 48584.5us
I20260709 13:03:29.897640  8793 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:29.897787  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 665) took 2420 ms. Trace:
W20260709 13:03:29.897936 12371 consensus_meta.cc:287] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.416s	user 0.013s	sys 0.005s
I20260709 13:03:29.898371  8825 rpcz_store.cc:276] 0709 13:03:27.474393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.730512 (+2256119us) service_pool.cc:224] Handling call
0709 13:03:29.860440 (+129928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.894292 (+ 33852us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.894332 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:29.894444 (+   112us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:29.894493 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.894571 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.894781 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.891841 12550 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:29.902997  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:29.904276  8676 raft_consensus.cc:2410] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:03:29.905035  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2298) took 1863 ms. Trace:
I20260709 13:03:29.905380  8676 rpcz_store.cc:276] 0709 13:03:28.041091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.818432 (+1777341us) service_pool.cc:224] Handling call
0709 13:03:29.904930 (+ 86498us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.907222  8676 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:29.907491  8679 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:29.907958  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2269) took 4568 ms. Trace:
I20260709 13:03:29.908206  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2271) took 2448 ms. Trace:
I20260709 13:03:29.908443  8676 rpcz_store.cc:276] 0709 13:03:25.339324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.906274 (+4566950us) service_pool.cc:224] Handling call
0709 13:03:29.906905 (+   631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.907636 (+   731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.907861 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23552}
I20260709 13:03:29.908723  8679 rpcz_store.cc:276] 0709 13:03:27.459666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.906378 (+2446712us) service_pool.cc:224] Handling call
0709 13:03:29.906867 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.907957 (+  1090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.908152 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.904248  8825 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:29.912814  8823 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:29.911459  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 668) took 2162 ms. Trace:
I20260709 13:03:29.914331  8825 rpcz_store.cc:276] 0709 13:03:27.748971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.903539 (+2154568us) service_pool.cc:224] Handling call
0709 13:03:29.904045 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.911093 (+  7048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.911338 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:29.911417  8828 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1662) took 5046 ms (client timeout 1954 ms). Trace:
I20260709 13:03:29.901311  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1868) took 2319 ms. Trace:
I20260709 13:03:29.913519 12411 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:29.919523  8827 rpcz_store.cc:276] 0709 13:03:27.576027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.867038 (+2291011us) service_pool.cc:224] Handling call
0709 13:03:29.891241 (+ 24203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.895030 (+  3789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.895280 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.921268  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 596) took 2424 ms. Trace:
I20260709 13:03:29.921698  8676 rpcz_store.cc:276] 0709 13:03:27.496824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.909744 (+2412920us) service_pool.cc:224] Handling call
0709 13:03:29.910185 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.910598 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.915302 (+  4704us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:29.920921 (+  5619us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:29.920979 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.921053 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.921200 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.929639  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 599) took 2432 ms. Trace:
I20260709 13:03:29.930160  8679 rpcz_store.cc:276] 0709 13:03:27.497660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.909849 (+2412189us) service_pool.cc:224] Handling call
0709 13:03:29.910336 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.929235 (+ 18899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.929266 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:29.929320 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:29.929365 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.929434 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.929572 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9600}
W20260709 13:03:29.933245 12267 consensus_meta.cc:287] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 6.271s	user 0.024s	sys 0.037s
I20260709 13:03:29.934482 11953 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.935192  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 600) took 2437 ms. Trace:
W20260709 13:03:29.918980  8828 rpcz_store.cc:269] 0709 13:03:24.864532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.117680 (+253148us) service_pool.cc:224] Handling call
0709 13:03:29.910960 (+4793280us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b540009c9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:29.911159 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34984320}
I20260709 13:03:29.935262 11953 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:29.941541 12334 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.491s	user 0.010s	sys 0.012s
I20260709 13:03:29.941888 11953 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:29.914340 12550 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:29.943153 12550 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:29.943315  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:29.945626 12458 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:29.947672  9382 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 738) took 5086 ms (client timeout 2211 ms). Trace:
W20260709 13:03:29.948207  9382 rpcz_store.cc:269] 0709 13:03:24.861242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.869549 (+  8307us) service_pool.cc:224] Handling call
0709 13:03:29.947314 (+5077765us) spinlock_profiling.cc:243] Waited 4.85 s on lock 0x7b5400510254. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:29.947507 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11155302144}
W20260709 13:03:29.949623  8722 connection.cc:256] RPC call timeout handler was delayed by -2.0200598610000498s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20790000000000003s for a response
W20260709 13:03:29.950228  8722 connection.cc:256] RPC call timeout handler was delayed by -1.9880839459999606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2923s for a response
I20260709 13:03:29.901659 12371 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:29.945662  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:29.935820  8676 rpcz_store.cc:276] 0709 13:03:27.497912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.931711 (+2433799us) service_pool.cc:224] Handling call
0709 13:03:29.932396 (+   685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.932745 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.934677 (+  1932us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:29.934776 (+    99us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:29.934845 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.934927 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.935090 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.954131 12370 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:29.954303  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 88137.4us
I20260709 13:03:29.954803 12370 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.955044  8822 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 9. Ops: []
I20260709 13:03:29.955307 12370 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:29.955700  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:29.953553  9388 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:29.958562 12371 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:29.960618  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 595) took 4959 ms. Trace:
I20260709 13:03:29.961233  8677 rpcz_store.cc:276] 0709 13:03:25.001173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.834134 (+4832961us) service_pool.cc:224] Handling call
0709 13:03:29.959326 (+125192us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:03:29.959663 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.960090 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.960129 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:29.960197 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:29.960255 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.960335 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.960504 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10144128}
I20260709 13:03:29.963013 12366 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:29.963570 12366 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.964133 12366 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:29.965454  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:29.966480  8678 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:29.968353  9389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 544) took 2407 ms. Trace:
I20260709 13:03:29.968752  9389 rpcz_store.cc:276] 0709 13:03:27.561112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.627604 (+ 66492us) service_pool.cc:224] Handling call
0709 13:03:27.715432 (+ 87828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.723626 (+  8194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.967861 (+2244235us) spinlock_profiling.cc:243] Waited 87.4 ms on lock 0x7b540061c1d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:29.968099 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201131264}
I20260709 13:03:29.967967  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 594) took 4967 ms. Trace:
I20260709 13:03:29.969494 12542 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:29.933949 12267 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:29.970733  8681 rpcz_store.cc:276] 0709 13:03:25.000931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.833138 (+4832207us) service_pool.cc:224] Handling call
0709 13:03:29.958504 (+125366us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:03:29.967228 (+  8724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.967510 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.967534 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:29.967607 (+    73us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:29.967655 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.967716 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.967864 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12073216}
I20260709 13:03:29.977685  8677 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:29.978659  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1829) took 2448 ms. Trace:
I20260709 13:03:29.979116  8677 rpcz_store.cc:276] 0709 13:03:27.530140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.970116 (+2439976us) service_pool.cc:224] Handling call
0709 13:03:29.970675 (+   559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.978287 (+  7612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.978541 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.982795  8672 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:03:29.989210  8673 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:29.991446  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1977) took 1192 ms. Trace:
I20260709 13:03:29.996760  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2266) took 4818 ms. Trace:
I20260709 13:03:29.997364  8672 rpcz_store.cc:276] 0709 13:03:25.178264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.884637 (+4706373us) service_pool.cc:224] Handling call
0709 13:03:29.925770 (+ 41133us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.996338 (+ 70568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.996676 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.942237 12334 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:29.996472  8753 rpcz_store.cc:276] 0709 13:03:28.799252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.288187 (+488935us) service_pool.cc:224] Handling call
0709 13:03:29.862585 (+574398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.871068 (+  8483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.991127 (+120059us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b540034ba50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:29.991345 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":663633280}
W20260709 13:03:30.002068  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 48867.2us
I20260709 13:03:30.006062  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 666) took 2255 ms. Trace:
I20260709 13:03:30.006815  8830 rpcz_store.cc:276] 0709 13:03:27.748312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.884595 (+2136283us) service_pool.cc:224] Handling call
0709 13:03:29.895511 (+ 10916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.000150 (+104639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.000197 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.002942 (+  2745us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:30.003031 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.003141 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.003295 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:30.020324  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 669) took 2271 ms. Trace:
I20260709 13:03:30.020802  8825 rpcz_store.cc:276] 0709 13:03:27.749248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.919090 (+2169842us) service_pool.cc:224] Handling call
0709 13:03:29.999610 (+ 80520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.004585 (+  4975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.004619 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.020064 (+ 15445us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:30.020101 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.020152 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.020268 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:30.004367 11953 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:30.022668  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 65876.2us
I20260709 13:03:30.001479 12458 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.026994  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1978) took 1227 ms. Trace:
I20260709 13:03:30.026866 12458 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:30.027483  8747 rpcz_store.cc:276] 0709 13:03:28.799596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.737699 (+938103us) service_pool.cc:224] Handling call
0709 13:03:29.739872 (+  2173us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.859863 (+119991us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.859907 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.884611 (+ 24704us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.981734 (+ 97123us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:29.981775 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.981846 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.026701 (+ 44855us) raft_consensus.cc:1685] finished
0709 13:03:30.026731 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.026913 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37554,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":66781,"thread_start_us":191185,"threads_started":2,"wal-append.queue_time_us":114238}
I20260709 13:03:30.030706  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2291) took 1991 ms. Trace:
I20260709 13:03:30.031169  8677 rpcz_store.cc:276] 0709 13:03:28.039218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.021933 (+1982715us) service_pool.cc:224] Handling call
0709 13:03:30.029722 (+  7789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.030249 (+   527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.030290 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.030359 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:30.030412 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.030482 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.030640 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":137,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":191}
I20260709 13:03:30.032625 12550 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:30.043964  9352 connection.cc:256] RPC call timeout handler was delayed by -2.2515081590000818s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
W20260709 13:03:30.044545  9352 connection.cc:256] RPC call timeout handler was delayed by -2.1750081590000718s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
W20260709 13:03:30.044909  9352 connection.cc:256] RPC call timeout handler was delayed by -2.1066081590000749s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
W20260709 13:03:30.045218  9352 connection.cc:256] RPC call timeout handler was delayed by -1.9734081590000869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
W20260709 13:03:30.045514  9352 connection.cc:256] RPC call timeout handler was delayed by -1.9104081590000987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2015s for a response
I20260709 13:03:30.047605  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 1964) took 2046 ms. Trace:
I20260709 13:03:30.052278 12267 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:30.021836 12371 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:30.052264  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2334) took 4133 ms. Trace:
I20260709 13:03:30.029614 12458 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:30.053682  8676 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:30.046264 12291 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:30.076614  8755 rpcz_store.cc:276] 0709 13:03:25.918962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.983749 (+ 64787us) service_pool.cc:224] Handling call
0709 13:03:25.991745 (+  7996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.154298 (+162553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.051687 (+3897389us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54002ace54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:30.052022 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340382592}
I20260709 13:03:30.056028 12370 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:30.083374 12483 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 2.621s	user 0.023s	sys 0.072s
I20260709 13:03:30.084298 12370 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:30.063066  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.064704 12550 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:30.090090  8754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 609) took 2037 ms (client timeout 1636 ms). Trace:
I20260709 13:03:30.091570  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.093525  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.094902  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.095396  8647 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:30.096438 12433 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:30.068861  8751 rpcz_store.cc:276] 0709 13:03:28.001260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.001868 (+   608us) service_pool.cc:224] Handling call
0709 13:03:30.047103 (+2045235us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b5400102a54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:30.047386 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":706698624}
W20260709 13:03:30.068192  9352 connection.cc:256] RPC call timeout handler was delayed by -1.8222081590000698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21130000000000002s for a response
I20260709 13:03:30.079530  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2284) took 2300 ms. Trace:
I20260709 13:03:30.081231  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1821) took 2553 ms. Trace:
W20260709 13:03:30.098629  9352 connection.cc:256] RPC call timeout handler was delayed by -1.7187081590001299s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22280000000000003s for a response
I20260709 13:03:30.098901  8676 rpcz_store.cc:276] 0709 13:03:27.527481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.976428 (+2448947us) service_pool.cc:224] Handling call
0709 13:03:30.022672 (+ 46244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.080875 (+ 58203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.081159 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:30.095686  8677 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:30.067420  8678 raft_consensus.cc:2410] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 13:03:30.099515  9352 connection.cc:256] RPC call timeout handler was delayed by -1.7043081590001066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22440000000000004s for a response
W20260709 13:03:30.097309  8754 rpcz_store.cc:269] 0709 13:03:28.052391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.239745 (+187354us) service_pool.cc:224] Handling call
0709 13:03:30.089622 (+1849877us) spinlock_profiling.cc:243] Waited 66.9 ms on lock 0x7b5400157cd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:30.089867 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153852672}
I20260709 13:03:30.103060  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.100625  8681 rpcz_store.cc:276] 0709 13:03:27.779483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.977021 (+2197538us) service_pool.cc:224] Handling call
0709 13:03:29.977641 (+   620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.978039 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:03:30.019642 (+ 41603us) log.cc:844] Serialized 14 byte log entry
0709 13:03:30.079001 (+ 59359us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.079065 (+    64us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:30.079127 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.079252 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.079441 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":46200,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":161,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":156,"spinlock_wait_cycles":21888,"thread_start_us":52709,"threads_started":2,"wal-append.queue_time_us":27840}
I20260709 13:03:30.104568  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2277) took 2527 ms. Trace:
I20260709 13:03:30.108531  8678 rpcz_store.cc:276] 0709 13:03:27.573445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.817959 (+2244514us) service_pool.cc:224] Handling call
0709 13:03:29.965154 (+147195us) spinlock_profiling.cc:243] Waited 994 us on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:03:30.100279 (+135125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2286848}
W20260709 13:03:30.109925  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 87592.1us
W20260709 13:03:30.110690  8722 connection.cc:256] RPC call timeout handler was delayed by -1.7955031739999185s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
I20260709 13:03:30.133725  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.136135  8676 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:30.143761  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:30.147404  8647 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:30.149226  8672 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:03:30.165513 12514 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.299s	user 0.004s	sys 0.036s
W20260709 13:03:30.166106 12514 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.384s	user 0.004s	sys 0.037s
I20260709 13:03:30.166846 12550 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:30.175189  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:30.176517  8722 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.079s (SENT) [suppressed 42 similar messages]
W20260709 13:03:30.177693  8722 leader_election.cc:336] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.121s (SENT)
W20260709 13:03:30.191102  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.152096  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2285) took 2346 ms. Trace:
I20260709 13:03:30.193521  8672 rpcz_store.cc:276] 0709 13:03:27.805304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.001357 (+2196053us) service_pool.cc:224] Handling call
0709 13:03:30.131520 (+130163us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.149783 (+ 18263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.151998 (+  2215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:30.195232  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.196429  8672 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:30.197881  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:30.198416  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.199378  9352 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:30.200714 12417 raft_consensus.cc:2764] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:03:30.201691  9352 leader_election.cc:400] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11.
I20260709 13:03:30.208025  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.222632  8677 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:30.227658  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 94994.9us
I20260709 13:03:30.228922  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:30.230882 12550 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:30.260592  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 59187.9us
I20260709 13:03:30.263063  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.210384  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.169701 12366 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:30.333690  8647 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:30.336519  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 604) took 2835 ms. Trace:
I20260709 13:03:30.338732  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.339478  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:30.341962  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:30.337057  8679 rpcz_store.cc:276] 0709 13:03:27.498981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.931894 (+2432913us) service_pool.cc:224] Handling call
0709 13:03:29.932396 (+   502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.932830 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:30.003102 (+ 70272us) log.cc:844] Serialized 14 byte log entry
0709 13:03:30.333777 (+330675us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:30.333829 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:30.333876 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.333931 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.334057 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":116264,"prepare.queue_time_us":90986,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":122481,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":165,"thread_start_us":185553,"threads_started":2,"wal-append.queue_time_us":139797}
I20260709 13:03:30.345212  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.347237 12354 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:30.349329  8672 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:30.355602  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 177988us
I20260709 13:03:30.363543  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.366417  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.368289  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.369788  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:30.356971  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 85464.7us
I20260709 13:03:30.371325  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.371671  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:30.372722 12514 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:30.373169  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 2.08272e+06us
I20260709 13:03:30.373355  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:30.374047  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.374296 12514 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:30.379491  8647 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:30.380303 12325 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:30.381572 12325 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:30.382030 12325 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:30.374279  8794 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:30.378104  8677 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:30.385563  8672 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:30.385993  8681 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:30.379443  8679 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:30.392441 12366 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:30.395051  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 125495us
I20260709 13:03:30.396065  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:30.390816 12550 raft_consensus.cc:2764] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:03:30.402434  8673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2250) took 6709 ms (client timeout 3223 ms). Trace:
W20260709 13:03:30.405879  8673 rpcz_store.cc:269] 0709 13:03:23.690458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.911995 (+221537us) service_pool.cc:224] Handling call
0709 13:03:30.394714 (+6482719us) spinlock_profiling.cc:243] Waited 6.08 s on lock 0x7b5400353c54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:30.399341 (+  4627us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13973388160}
I20260709 13:03:30.408586  8680 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:30.409350  8672 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:30.409320  8719 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:30.404515 12349 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:30.405543 12334 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:30.419839  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 336902us
I20260709 13:03:30.419387  8794 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:30.420902  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:30.414312 12349 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:30.425935  9389 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:30.417575 12301 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:30.423260 12380 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:30.411935 12323 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:03:30.429509 12366 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:30.427474 12349 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:30.450906  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 72882.5us
I20260709 13:03:30.458863  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.455622  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.437876 12301 raft_consensus.cc:697] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:30.418205 12325 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:30.446382 12380 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:30.465230  8755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "293a9dab5be3479a90d5803585682a00"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:35702
I20260709 13:03:30.471722  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.465564  8747 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:35702
I20260709 13:03:30.467984  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:30.471235  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.470517  8755 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:30.479683  8755 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:30.472529  8747 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:30.483367  8753 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:30.483880  8747 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:30.478963  8830 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:30.491364 12301 consensus_queue.cc:237] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:30.494777  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.495947  8821 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:30.496966 12323 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:30.497088  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.498467 12380 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:30.500943  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.500190  8794 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:30.497512 12323 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.489280  8825 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:30.509589  8828 raft_consensus.cc:2410] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:30.508810  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.512953  8827 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:03:30.516683  8647 leader_election.cc:336] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.781s (SENT)
I20260709 13:03:30.522998  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:30.524997  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.527406  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:30.507671 12323 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:30.528185  8681 raft_consensus.cc:2376] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8. Re-sending same reply.
I20260709 13:03:30.527783  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:30.538491  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 116700us
I20260709 13:03:30.541121  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:30.543761  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:30.545082  8647 connection.cc:256] RPC call timeout handler was delayed by -1.4160002820000273s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
I20260709 13:03:30.546537 12433 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.547276 12433 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:30.550809  8647 connection.cc:256] RPC call timeout handler was delayed by -1.3629002820000551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
W20260709 13:03:30.551306  8647 connection.cc:256] RPC call timeout handler was delayed by -1.3629002820000551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
W20260709 13:03:30.551739  8647 connection.cc:256] RPC call timeout handler was delayed by -1.2387002820000816s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
W20260709 13:03:30.552032  8647 connection.cc:256] RPC call timeout handler was delayed by -1.2000002820000191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18200000000000002s for a response
W20260709 13:03:30.552302  8647 connection.cc:256] RPC call timeout handler was delayed by -1.1559002820000615s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1869s for a response
I20260709 13:03:30.553079  8825 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:30.554812  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.556068  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.558003  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.551082  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.560619  8673 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 5.
I20260709 13:03:30.537109 12325 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:30.508924 12014 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:03:30.563078  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.538430 12366 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:30.550292 12433 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:30.570133  8830 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:30.577400 11953 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:30.585813  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:30.580672 12402 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:30.583204  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:30.585546  8827 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:30.595543  8821 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:30.567545 12366 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.592370  8673 raft_consensus.cc:2376] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6. Re-sending same reply.
I20260709 13:03:30.603962  8748 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:30.587291 11953 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:30.615052  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.614060  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:30.626890  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:30.632233  8673 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:30.607112 12349 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:30.636193  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 136200us
I20260709 13:03:30.636564 12349 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:30.637236  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:30.603663 12366 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:30.643352  8793 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:30.649119  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.650987  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.645661  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:30.653991  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 251115us
I20260709 13:03:30.642807  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:30.648710  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:30.661983  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 45042.5us
I20260709 13:03:30.660089  8755 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:30.667317  8749 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:30.668084  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:30.669780  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 62179.9us
I20260709 13:03:30.680702  8753 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:30.682693  8755 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:30.685779  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:30.674525  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 54220us
I20260709 13:03:30.666884 12402 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:30.700775  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.703712  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.704573  8821 raft_consensus.cc:2410] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:03:30.705658  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 45804.2us
I20260709 13:03:30.680720 12433 raft_consensus.cc:2764] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:03:30.711112  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.712200  8828 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:30.712217  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:30.679095 12417 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:30.714553  8679 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:30.685474 12334 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:30.686983 12349 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:30.721990  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 84880.3us
I20260709 13:03:30.721484  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:30.707278 12564 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:30.703305 12402 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:30.728500  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:30.732985  8821 raft_consensus.cc:2376] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7. Re-sending same reply.
I20260709 13:03:30.716804 12417 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.735221  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.743032  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:30.744495  8825 raft_consensus.cc:2410] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:03:30.750015  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 81703.1us
I20260709 13:03:30.732345 12402 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:30.728837 12564 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:30.752369  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:30.751220  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.724953 12349 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:30.737990 12417 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:30.757194  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:30.756961 12566 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:30.757601  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:30.734334 12354 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:03:30.730798 12301 consensus_queue.cc:1059] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
W20260709 13:03:30.758814  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 96233.9us
I20260709 13:03:30.763931  8722 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:30.773018  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:30.775604  9388 raft_consensus.cc:2368] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6.
I20260709 13:03:30.755071 12564 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:30.782575  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 62867.4us
W20260709 13:03:30.789007  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 47728.5us
I20260709 13:03:30.790899 12401 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:30.791465 12401 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.791818 12401 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:30.797482 12565 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:30.797183 12514 consensus_queue.cc:1059] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:30.798105 12565 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:30.798657 12565 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:30.801321  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:30.782127  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:30.802457  8674 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 7: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:30.805269  8673 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:03:30.788496  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:30.775532 12497 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.394s	user 0.014s	sys 0.042s
I20260709 13:03:30.770419  8747 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:30.810830 12354 raft_consensus.cc:3055] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:30.811403 12354 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:30.812475 12354 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:30.782585  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:30.833009  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:30.834596  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 56131.5us
W20260709 13:03:30.809934 12497 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.486s	user 0.014s	sys 0.043s
I20260709 13:03:30.837373  8722 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:30.838156 12325 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:30.842960  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.846637 12354 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:30.847144 12334 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:30.850589 12301 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:30.851248 12301 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:30.851686 12301 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:30.852977 12323 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:30.855870 12349 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:30.856083  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:30.844794 12325 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.859256 12325 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:30.860231  8647 leader_election.cc:400] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6.
I20260709 13:03:30.860669  8647 leader_election.cc:403] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:03:30.834088  8673 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:30.855826 12323 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:30.863091 12418 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:30.863696 12418 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:30.864750 12418 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:30.865332  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1855) took 5714 ms. Trace:
I20260709 13:03:30.865758  8822 rpcz_store.cc:276] 0709 13:03:25.151205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.431417 (+3280212us) service_pool.cc:224] Handling call
0709 13:03:28.445434 (+ 14017us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.955667 (+1510233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.864951 (+909284us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54003ddbd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:30.865239 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3464336640}
I20260709 13:03:30.866909 12433 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:30.872463  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:30.876415 12433 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:30.802772 12401 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:30.888119  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:30.894971 12401 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:30.895313  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:30.896133  9383 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:30.894387  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:30.900390  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.898064  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:30.904731  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:30.905819  9388 raft_consensus.cc:2368] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9.
W20260709 13:03:30.903321  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 131790us
I20260709 13:03:30.914253  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:30.906642 12349 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:30.921983  8678 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:30.926376  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:30.931643  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:30.945326  9383 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:30.928318 12497 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:03:30.950242  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:30.917433 11953 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:30.956902  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:30.936975 12402 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:30.963671 12301 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:31.151245  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.155758 12569 log.cc:927] Time spent T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:03:31.156728 12433 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 5. Current term is 6.
I20260709 13:03:31.157385  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:31.157845  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:31.158355  9389 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:31.159447  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:31.160024 11953 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:31.160283  9388 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:31.157465  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:31.164098  8678 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:31.164037  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:31.170498  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:31.172336  9388 raft_consensus.cc:2410] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:31.170775  9385 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:31.179816  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:31.184937 12402 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:31.187770 11666 raft_consensus.cc:3055] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:31.188283 11666 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:31.185557 12301 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:31.166669 12366 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:31.217020 12564 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:31.222270 12564 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:31.226188 12565 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:31.230962 12565 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:31.259963  8753 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.578s	user 0.013s	sys 0.001s
I20260709 13:03:31.260601  8753 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:31.189517 11666 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:31.263535 11666 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:31.267004  8673 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
W20260709 13:03:31.323339  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.327941 12572 raft_consensus.cc:3055] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:31.328593 12572 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:31.329733 12572 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:31.331820 12572 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:31.335436  8567 maintenance_manager.cc:643] P 06d3808b553142ef8f7a4e57845757cd: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 1.783s	user 0.134s	sys 0.067s Metrics: {"bytes_written":29203,"cfile_init":1,"dirs.queue_time_us":1194,"dirs.run_cpu_time_us":449,"dirs.run_wall_time_us":1197,"drs_written":1,"lbm_read_time_us":603,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4469,"lbm_writes_lt_1ms":29,"peak_mem_usage":0,"rows_written":63,"thread_start_us":4189,"threads_started":2,"wal-append.queue_time_us":3963}
I20260709 13:03:31.337831 12550 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:31.338590 12550 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:31.339663  8568 maintenance_manager.cc:419] P 06d3808b553142ef8f7a4e57845757cd: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8656 bytes on disk
I20260709 13:03:31.341261 12550 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:31.343163  8753 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 8. Ops: []
W20260709 13:03:31.343709 12417 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.567s	user 0.016s	sys 0.000s
I20260709 13:03:31.344146 12417 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:31.348305 12417 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:31.346258  8753 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:31.351326  8567 maintenance_manager.cc:643] P 06d3808b553142ef8f7a4e57845757cd: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":252,"lbm_reads_lt_1ms":4}
I20260709 13:03:31.353017  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1857) took 6128 ms. Trace:
I20260709 13:03:31.353547  8824 rpcz_store.cc:276] 0709 13:03:25.225422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.458161 (+3232739us) service_pool.cc:224] Handling call
0709 13:03:28.458564 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.529156 (+ 70592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.952221 (+2423065us) spinlock_profiling.cc:243] Waited 66.6 ms on lock 0x7b54003dd6d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:31.352796 (+400575us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153226496}
I20260709 13:03:31.383020 12565 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.383781 12565 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:31.384438  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.386310 12565 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:31.386435  8722 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:31.387516 12564 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.401636 12564 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:31.403514  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:31.404484 12564 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:31.405207 12564 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.405910 12323 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:31.405738 12564 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:31.406592 12323 raft_consensus.cc:697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:31.387791 12574 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:31.407754 12323 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:31.408257 12574 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:31.411330 12574 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:31.414021 12564 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:31.419943  8647 leader_election.cc:336] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.820s (SENT)
I20260709 13:03:31.422854  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:31.420049  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:31.396114  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.423744  8828 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:31.425786 12366 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:31.433276  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.010s	sys 0.019s
W20260709 13:03:31.433909  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.010s	sys 0.019s
W20260709 13:03:31.434289  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 638191us
I20260709 13:03:31.438467  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:31.439656 12354 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:31.440622 12354 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:31.441151 12354 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:31.442513  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:31.443297  8674 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:31.443742  8828 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:31.459307  8645 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:31.461701  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:31.462983  8822 raft_consensus.cc:2376] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10. Re-sending same reply.
I20260709 13:03:31.466238 12418 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:31.470760 12418 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:31.471302 12418 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:31.477195  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:31.478585 12514 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:31.478928  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:31.480010  8825 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
W20260709 13:03:31.484321  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 55867.1us
I20260709 13:03:31.504106 12418 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:31.512655  8674 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:31.491382 12514 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:03:31.528960 12418 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:31.532428 12354 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:31.543255  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 590) took 7755 ms. Trace:
W20260709 13:03:31.545872  9355 connection.cc:256] RPC call timeout handler was delayed by -2.0813317660000621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
W20260709 13:03:31.547948  9355 connection.cc:256] RPC call timeout handler was delayed by -1.4171277140000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.164s for a response
W20260709 13:03:31.548379  9355 connection.cc:256] RPC call timeout handler was delayed by -1.3460277140000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
I20260709 13:03:31.548811  8747 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:31.548882  9355 connection.cc:256] RPC call timeout handler was delayed by -1.0544277139999849s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
I20260709 13:03:31.547726  8680 rpcz_store.cc:276] 0709 13:03:23.787684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.978009 (+190325us) service_pool.cc:224] Handling call
0709 13:03:30.408188 (+6430179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.412164 (+  3976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.542933 (+1130769us) spinlock_profiling.cc:243] Waited 6.42 s on lock 0x7b5400353c50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:31.543154 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14767442688}
I20260709 13:03:31.550736  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:35702 (request call id 615) took 1105 ms. Trace:
I20260709 13:03:31.551194  8747 rpcz_store.cc:276] 0709 13:03:30.444658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.451981 (+  7323us) service_pool.cc:224] Handling call
0709 13:03:31.549588 (+1097607us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.564257  8796 leader_election.cc:400] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9.
I20260709 13:03:31.564805  8796 leader_election.cc:403] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:31.565234  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:31.549944 12514 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:31.566654 12572 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:03:31.567137 12514 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:31.568290 12354 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:31.569737 12514 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:31.576777  8796 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:31.581743  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 46619.6us
I20260709 13:03:31.548799 12433 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:31.584161 12366 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:31.585938  8675 raft_consensus.cc:2410] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:03:31.586483 12366 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:31.588557 12366 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:31.591912 12353 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:31.594331  8722 leader_election.cc:336] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.549s (SENT)
I20260709 13:03:31.595168  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:31.586046 12433 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:31.601476 12433 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:31.603591 12433 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:31.603056 12572 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9.
I20260709 13:03:31.586086 12592 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:03:31.618430  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:31.610251  8826 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1852) took 6476 ms (client timeout 1781 ms). Trace:
W20260709 13:03:31.635102  8826 rpcz_store.cc:269] 0709 13:03:25.150007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.239266 (+1089259us) service_pool.cc:224] Handling call
0709 13:03:31.609703 (+5370437us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b54003dead4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:31.610005 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":779632384}
I20260709 13:03:31.637426 12353 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:31.608204 12325 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:31.641332 12325 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:31.642859  8754 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 9: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:31.643261  8579 catalog_manager.cc:5697] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 2 to 5, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 5 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:03:31.621567  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:31.653977  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:31.665715 12450 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:31.672072 12450 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:31.646018  8579 catalog_manager.cc:5697] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 4 to 7, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 7 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:31.674021 12450 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:31.678534  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 618) took 1116 ms. Trace:
W20260709 13:03:31.679566  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 103812us
I20260709 13:03:31.682600  8681 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:31.683597  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2307) took 1150 ms. Trace:
I20260709 13:03:31.683862  8821 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:31.683966  8681 rpcz_store.cc:276] 0709 13:03:30.533524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.545689 (+ 12165us) service_pool.cc:224] Handling call
0709 13:03:31.621324 (+1075635us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:03:31.683529 (+ 62205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13406976}
I20260709 13:03:31.679754  8749 rpcz_store.cc:276] 0709 13:03:30.561936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.562150 (+   214us) service_pool.cc:224] Handling call
0709 13:03:30.569253 (+  7103us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.678835 (+109582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.678146 (+999311us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b54002fe4d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:31.678417 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222953344}
W20260709 13:03:31.686085  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 91843.7us
I20260709 13:03:31.687101  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:31.689512  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:31.691077  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:31.691761  9383 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:31.692087  9383 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:31.690367  9384 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:31.693434  9384 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:31.694489 12450 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:31.695928  9384 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:31.697129  9384 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:31.699942  8719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.957s	user 0.004s	sys 0.011s
I20260709 13:03:31.700816  8829 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:31.702170  8719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.005s	sys 0.012s
I20260709 13:03:31.703038  9383 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:31.704238  9383 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:03:31.706831  9384 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
W20260709 13:03:31.707382  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 983660us
I20260709 13:03:31.708729  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:31.724180  9383 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:31.728583 12566 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:31.729097  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.730871 12597 consensus_queue.cc:1059] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:03:31.733423  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.731182  8719 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:31.730289  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:31.745005 12566 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:31.749729  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:31.740927 12462 raft_consensus.cc:3055] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:31.755898 12462 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:31.757076 12462 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:31.758693 12462 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:31.743533  8679 consensus_meta.cc:287] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.026s	user 0.015s	sys 0.000s
I20260709 13:03:31.761070  8679 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
W20260709 13:03:31.761583  8751 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.018s	user 0.007s	sys 0.030s
I20260709 13:03:31.761740  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2323) took 1183 ms. Trace:
W20260709 13:03:31.762020  8751 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.022s	user 0.007s	sys 0.030s
I20260709 13:03:31.762233  8679 rpcz_store.cc:276] 0709 13:03:30.583602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.704798 (+121196us) service_pool.cc:224] Handling call
0709 13:03:31.761652 (+1056854us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.750679 12566 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:31.762926  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 613) took 1319 ms. Trace:
I20260709 13:03:31.763393  8751 rpcz_store.cc:276] 0709 13:03:30.443967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.445341 (+  1374us) service_pool.cc:224] Handling call
0709 13:03:30.455253 (+  9912us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:30.658330 (+203077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.658364 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:30.719767 (+ 61403us) log.cc:844] Serialized 26 byte log entry
0709 13:03:31.762480 (+1042713us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:31.762533 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.762635 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.762692 (+    57us) raft_consensus.cc:1685] finished
0709 13:03:31.762708 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.762845 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":159520,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":1189859,"threads_started":2,"wal-append.queue_time_us":218106}
W20260709 13:03:31.739260  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.753014 12325 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:31.772593  9383 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:03:31.776554  8752 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:31.778524  8749 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:03:31.779026  8749 pending_rounds.cc:85] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 6
I20260709 13:03:31.779309  8749 pending_rounds.cc:107] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:03:31.779601  8749 raft_consensus.cc:2889] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:31.792622  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 1.01799e+06us
I20260709 13:03:31.793715  8794 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:31.795568  8793 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:31.754945  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1864) took 4206 ms. Trace:
I20260709 13:03:31.806802  8829 rpcz_store.cc:276] 0709 13:03:27.548013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.867694 (+2319681us) service_pool.cc:224] Handling call
0709 13:03:31.612591 (+1744897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.701831 (+ 89240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.753920 (+ 52089us) spinlock_profiling.cc:243] Waited 87.9 ms on lock 0x7b54003dead4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:31.754713 (+   793us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54003dead0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:31.754866 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4161402880}
W20260709 13:03:31.803902  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.803830 12325 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:31.811216 12325 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:31.811764 12325 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:31.814124  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:31.814280  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:31.815214  9388 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:31.815824  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:31.816483  8681 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:31.816517  8679 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:31.818394  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:31.820034  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 663) took 4361 ms. Trace:
I20260709 13:03:31.820427  8823 rpcz_store.cc:276] 0709 13:03:27.458819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.529539 (+1070720us) service_pool.cc:224] Handling call
0709 13:03:29.911937 (+1382398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.913539 (+  1602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.819815 (+1906276us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b540009c9d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:31.819971 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3142421888}
I20260709 13:03:31.821415  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:31.822320 12566 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:31.822902 12566 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:03:31.823235  8675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2290) took 3784 ms (client timeout 2930 ms). Trace:
I20260709 13:03:31.823474  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:31.824397  9383 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:31.823933 12566 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:31.825573  8681 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:31.826851  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:31.827446  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:31.828222  9388 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:03:31.823724  8675 rpcz_store.cc:269] 0709 13:03:28.038870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.559374 (+1520504us) service_pool.cc:224] Handling call
0709 13:03:31.822970 (+2263596us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b5400175554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:31.823132 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4656440576}
I20260709 13:03:31.829217  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:31.830168  9384 raft_consensus.cc:2410] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:31.831220 12418 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:31.831957 12418 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:31.835470  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:31.836148  9388 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:31.839763  8681 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:03:31.842823 12418 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:31.845036  9380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 546) took 4231 ms. Trace:
W20260709 13:03:31.849567  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.845561  9380 rpcz_store.cc:276] 0709 13:03:27.613096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.629149 (+ 16053us) service_pool.cc:224] Handling call
0709 13:03:27.717449 (+ 88300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.717862 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:28.311827 (+593965us) log.cc:844] Serialized 14 byte log entry
0709 13:03:28.551347 (+239520us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:28.551417 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:28.551469 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.551537 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.844621 (+3293084us) spinlock_profiling.cc:243] Waited 87.9 ms on lock 0x7b54002369d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:31.844802 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31126,"prepare.queue_time_us":637294,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":119748,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":39994,"spinlock_wait_cycles":202215296,"thread_start_us":813200,"threads_started":2,"wal-append.queue_time_us":349860,"wal-append.run_cpu_time_us":1576,"wal-append.run_wall_time_us":1018478}
I20260709 13:03:31.851023 12353 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:31.851900 12353 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:31.852181 12301 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:31.853324 12353 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:31.854274 12402 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:31.770502  9353 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:31.856518  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 86170.2us
I20260709 13:03:31.857470 12408 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:31.858676  9352 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:31.859978  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:31.860116  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.861043 12095 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:31.862052 12095 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:31.863158 12095 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:31.869736  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 79256.8us
W20260709 13:03:31.870802  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 77233.3us
I20260709 13:03:31.863696  8722 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:31.804431  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 54791.7us
I20260709 13:03:31.883356  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:31.884299  8677 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:31.879048 12334 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:03:31.885380 12565 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:31.886164  8579 catalog_manager.cc:5697] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:03:31.887009 12323 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:31.888043 12402 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:31.890415 12592 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:31.891628 12417 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:31.892937 12417 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:31.893359 12417 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:31.897070  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:31.740928  9355 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:31.894162 12408 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:03:31.900871 12408 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:31.901369 12408 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:31.901408  8794 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:31.902971  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:31.905951  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:31.906711  8675 raft_consensus.cc:2376] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7. Re-sending same reply.
I20260709 13:03:31.907272 12402 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:31.921700  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:31.925436 12401 consensus_queue.cc:1059] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:31.894362 12417 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:31.930181 12462 raft_consensus.cc:3055] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:03:31.939361  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 198541us
I20260709 13:03:31.940241  9355 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:31.941596  9355 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:31.927424 12095 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:31.943575  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:31.947387  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:31.929054 12514 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.949558  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:31.950371  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.952600  8579 catalog_manager.cc:5697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 7 to 8, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 8 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:31.956897  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:31.957741  8677 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:31.959100  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:31.960016  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:31.961709  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 53318us
I20260709 13:03:31.963706  8749 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:31.928787 12301 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:31.937484 12462 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:31.966521  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:31.968400  8674 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:31.971239  8679 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:31.965718  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:31.975551  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:31.978385  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:31.979841  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:31.902526 12408 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:31.993081  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:31.993291  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:31.994221  8675 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:03:31.992988  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50415.1us
I20260709 13:03:31.949836 12450 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:31.990535  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:31.998066  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:31.998418  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:31.976714  8680 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:32.000993  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:32.002164  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:32.002890  8679 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:32.012061  8681 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:31.964715 12594 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:32.013361  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:32.020973  8747 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:32.022439  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:32.022704  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:32.023644  8680 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:32.023764  8673 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:32.000349 12417 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:32.030416  8747 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:32.031916 12408 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:32.032904  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:32.033784  8756 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:03:32.035300  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 63188us
I20260709 13:03:31.970606 12514 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:32.043449  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:32.043954  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 102425us
I20260709 13:03:32.044275  8747 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:32.042208  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:32.059481 12612 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.060225 12612 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:32.062311  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:32.062232  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:32.063416  8749 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:32.054886  9355 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:32.065050  8677 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:32.087352  8673 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:32.050550 12450 raft_consensus.cc:697] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:32.039850 12613 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:32.100483  8749 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:32.109058 12613 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.109771 12613 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:32.062137 12572 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:32.112283  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:32.025892 12594 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:32.104874  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:32.115644  8756 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:32.111972 12572 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:32.117700 12572 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:32.085008 12095 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:03:32.121587 12095 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:32.122164 12095 raft_consensus.cc:697] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:32.042603 12462 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:32.167626  8722 leader_election.cc:336] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.954s (SENT)
W20260709 13:03:32.168787 12604 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 13:03:32.169653 12601 log.cc:927] Time spent T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:03:32.169474  9355 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:32.185701  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 166879us
W20260709 13:03:32.187829  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 151800us
I20260709 13:03:32.194978  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:32.199311  8748 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:03:32.208367  8829 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:03:32.211515  8747 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:03:32.211365  8827 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:32.213272  8826 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:32.206712 12353 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:32.220475  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:32.221532  8755 raft_consensus.cc:2410] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:03:32.221997 12602 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 13:03:32.224664  8610 debug-util.cc:398] Leaking SignalData structure 0x7b080008b0e0 after lost signal to thread 7529
W20260709 13:03:32.226060  8610 debug-util.cc:398] Leaking SignalData structure 0x7b080003c300 after lost signal to thread 8613
W20260709 13:03:32.227787  8610 debug-util.cc:398] Leaking SignalData structure 0x7b08003d21a0 after lost signal to thread 8705
W20260709 13:03:32.229429  8610 debug-util.cc:398] Leaking SignalData structure 0x7b08001d58e0 after lost signal to thread 8780
W20260709 13:03:32.229802  8610 debug-util.cc:398] Leaking SignalData structure 0x7b08002be720 after lost signal to thread 8781
I20260709 13:03:32.221997  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:32.209532  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:32.234035  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:32.234761  8755 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:32.235286  8610 debug-util.cc:398] Leaking SignalData structure 0x7b08002079a0 after lost signal to thread 8854
I20260709 13:03:32.233034  8756 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
W20260709 13:03:32.237764  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68365.3us
W20260709 13:03:32.244763  8610 debug-util.cc:398] Leaking SignalData structure 0x7b080034ab60 after lost signal to thread 9413
W20260709 13:03:32.245434  8610 debug-util.cc:398] Leaking SignalData structure 0x7b08003713c0 after lost signal to thread 9414
I20260709 13:03:32.258529  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:32.275445  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:32.290496  8748 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:32.291708  8747 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
W20260709 13:03:32.318150  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.332468  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 136471us
I20260709 13:03:32.332803  8677 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:32.335464  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:32.337211  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:32.336273  8749 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:32.224737 12618 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:03:32.340386  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 428867us
I20260709 13:03:32.341352  8748 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:03:32.342978  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 49581.8us
I20260709 13:03:32.346508  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:32.349121  8823 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:32.352939  8748 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
W20260709 13:03:32.377204  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 246748us
W20260709 13:03:32.378141  8722 leader_election.cc:336] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.022s (SENT)
I20260709 13:03:32.378659  8755 raft_consensus.cc:2410] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:03:32.363683  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:32.384287  8754 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:32.378571  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:32.386412  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:32.397452  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 64027.5us
I20260709 13:03:32.399567  8756 raft_consensus.cc:2410] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:32.401176  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:32.402446  8755 raft_consensus.cc:2368] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8.
I20260709 13:03:32.408033  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:32.412161  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:32.380121 12592 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:32.440886  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:32.441509 12612 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:32.444669  8755 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:32.449870  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 116230us
I20260709 13:03:32.348335 12450 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:32.402736 12514 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:32.646404  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:32.651722  8756 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:32.707607  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 360260us
W20260709 13:03:32.707872  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 258504us
I20260709 13:03:32.347577 12095 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:32.446003 12334 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:32.444406 12592 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:32.736886  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 358766us
I20260709 13:03:32.769265  8755 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:32.445883 12565 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:32.806908  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.809410  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 411044us
W20260709 13:03:32.816821  8722 leader_election.cc:336] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.524s (SENT)
I20260709 13:03:32.817494  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:32.725524 12402 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:32.806641  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.820071  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.825865 12402 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:32.828471 12402 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:32.831499 12623 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.379s	user 0.000s	sys 0.000s
W20260709 13:03:32.833088 12627 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:03:32.834506 12619 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.428s	user 0.000s	sys 0.000s
I20260709 13:03:32.837813  8821 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:03:32.838307  8821 pending_rounds.cc:85] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 3
I20260709 13:03:32.838544  8821 pending_rounds.cc:107] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:03:32.838809  8821 raft_consensus.cc:2889] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:32.838943  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:32.839787  8829 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:32.840278  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:32.840801  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:32.841672  8826 raft_consensus.cc:2393] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:32.842931  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:32.843894  8825 raft_consensus.cc:2368] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11.
I20260709 13:03:32.855820 12632 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:32.857704 12632 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:32.841054  8830 raft_consensus.cc:2368] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:03:32.859616  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:32.868088  8754 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:03:32.870167  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:32.877072 12622 consensus_queue.cc:1059] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:32.818928 12612 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:32.879632 12612 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:32.883669 12629 raft_consensus.cc:3055] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:32.884380 12629 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:32.882259 12612 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:32.885519 12629 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:32.888634  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:32.890244  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:32.891467  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:32.891824  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:32.892943  8755 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:32.894393  8748 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
W20260709 13:03:32.896301  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 79981.5us
I20260709 13:03:32.898116  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:32.905918  8754 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:32.775072 12325 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.963s	user 0.016s	sys 0.001s
I20260709 13:03:32.912406  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:32.912317  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:32.929106  8751 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.070s	sys 0.118s
W20260709 13:03:32.929577  8751 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.070s	sys 0.119s
I20260709 13:03:32.930593  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:32.934540  8754 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:32.944780  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:32.899115 12323 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:32.996315  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 287665us
I20260709 13:03:32.735319 12408 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.184s
I20260709 13:03:32.993271  8748 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
W20260709 13:03:32.993688  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 63203.4us
I20260709 13:03:32.994179  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:33.006413  8748 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:33.008504  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:33.009895  8754 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 8: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:32.725901 12594 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:33.010288  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:33.013448  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:33.039279  8756 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:33.041493  8830 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:33.006062 12572 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:33.041429  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:33.047215  8823 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:33.049067  8829 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:03:33.057523  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 53941.9us
I20260709 13:03:33.058526  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:33.058979  8755 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:33.060117  8825 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:03:32.946508 12325 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:33.060504  8825 pending_rounds.cc:85] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 5
I20260709 13:03:33.065897  8825 pending_rounds.cc:107] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:03:33.066183  8825 raft_consensus.cc:2889] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:33.067324  8830 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:33.067371  8829 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:33.068612  8793 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:33.074879  8755 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:03:33.077714  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 260953us
I20260709 13:03:33.079980  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:33.081333  8829 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:33.087881  8719 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:33.068573  9354 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.007s	sys 0.032s
W20260709 13:03:33.089345  9354 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.008s	sys 0.033s
I20260709 13:03:33.089998  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:33.068930 12402 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:33.069564 12572 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:33.101610  8830 raft_consensus.cc:2393] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:33.105664  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:33.111114  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:33.074644 12334 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:33.099958 12450 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:33.121277  8719 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:33.109077 12639 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:33.134438  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:33.132114  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:33.139133  8830 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:33.139761  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 71271.9us
I20260709 13:03:33.139168  8719 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:33.140596  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:33.141378  8829 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:33.138543 12565 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:33.131181 12450 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:33.145802 12450 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:33.125250 12417 consensus_meta.cc:287] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.087s	user 0.009s	sys 0.012s
I20260709 13:03:33.151340  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:33.135637 12613 consensus_meta.cc:287] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.025s	user 0.015s	sys 0.006s
I20260709 13:03:33.140789  8793 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:33.239276  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 160655us
I20260709 13:03:33.256253  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:33.260108  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 119397us
I20260709 13:03:33.261340  8793 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:33.263051  8722 leader_election.cc:336] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.079s (SENT)
I20260709 13:03:33.263612  8722 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:33.272562  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.273383  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.281339  8829 raft_consensus.cc:2393] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:33.295521  9384 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:03:33.338495  9380 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:03:33.230405 12594 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.386081  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:33.383288  8823 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:33.240051 12417 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.387673  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:33.441949  9388 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:03:33.442109  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 302995us
I20260709 13:03:33.447328  8829 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 13:03:33.472069  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 209039us
W20260709 13:03:33.472980  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.383121 12550 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:33.238703 12613 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:33.433385 12323 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:33.535025  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:33.451210 12612 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:33.537741  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:33.527405  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 84041.8us
I20260709 13:03:33.547343 12450 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:33.551673 12633 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.433s	user 0.001s	sys 0.000s
I20260709 13:03:33.529968 12640 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:33.578013 12592 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.771s	user 0.011s	sys 0.002s
I20260709 13:03:33.695277 12592 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:33.654649 12641 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:03:33.656127 12325 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:33.262538 12572 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:33.699060  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.658205 12565 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:33.702636 12565 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:33.705969 12565 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:33.658841 12323 raft_consensus.cc:697] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:33.707765 12323 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:33.432512 12594 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:33.713074 12594 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:33.515057 12402 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:33.714166 12402 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:33.715484 12402 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:33.514102 12645 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:03:33.718900  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 487327us
W20260709 13:03:33.582295 12636 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.431s	user 0.000s	sys 0.001s
I20260709 13:03:33.600920  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:33.450809 12417 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:33.758675  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.770972  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 170176us
I20260709 13:03:33.383255 12334 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:33.795701  8754 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:33.851660 12647 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.853857 12616 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.872696 12648 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.881345  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:33.882984  8823 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:03:33.899529  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:33.952224  8719 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:33.536916 12550 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:33.794221 12612 raft_consensus.cc:2764] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:03:33.675239 12450 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:33.993887  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:33.994900  8829 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
W20260709 13:03:34.009085  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 288243us
I20260709 13:03:34.011879 12658 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:34.020524  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1991) took 1784 ms. Trace:
W20260709 13:03:34.022424  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.002s	sys 0.009s
I20260709 13:03:34.021188  8753 rpcz_store.cc:276] 0709 13:03:32.238594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.288696 (+ 50102us) service_pool.cc:224] Handling call
0709 13:03:32.348287 (+ 59591us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:32.356206 (+  7919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:32.356256 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:32.387234 (+ 30978us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.899612 (+512378us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:32.899679 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.899794 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.020177 (+1120383us) raft_consensus.cc:1685] finished
0709 13:03:34.020207 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.020421 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6786,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":6541,"thread_start_us":195660,"threads_started":1,"wal-append.queue_time_us":660239}
W20260709 13:03:34.023015  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.002s	sys 0.009s
I20260709 13:03:34.025359  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:34.027160  8748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.008s	sys 0.017s
W20260709 13:03:34.027688  8748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.008s	sys 0.018s
I20260709 13:03:34.028138  8829 raft_consensus.cc:2393] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:33.991103 12592 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:34.029253  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:34.027446 12645 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:34.034832 12612 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:34.023504  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 762277us
I20260709 13:03:34.039687 12450 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:34.040024 12650 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:03:34.056386  9354 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.007s	sys 0.012s
W20260709 13:03:34.057008  9354 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.007s	sys 0.012s
I20260709 13:03:34.058655  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.043006  8793 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:34.043430  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:34.071811 12572 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:34.072357  8722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.002s	sys 0.011s
I20260709 13:03:34.072330  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:03:34.075971  8722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.002s	sys 0.011s
W20260709 13:03:34.076411  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 600694us
I20260709 13:03:34.081599  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 635) took 1160 ms. Trace:
W20260709 13:03:34.083351  8722 leader_election.cc:336] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.559s (SENT)
I20260709 13:03:34.083614  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.088861  8825 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:34.087627  8755 rpcz_store.cc:276] 0709 13:03:32.921305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.110922 (+189617us) service_pool.cc:224] Handling call
0709 13:03:33.115632 (+  4710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.125017 (+  9385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.125073 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:33.233761 (+108688us) log.cc:844] Serialized 26 byte log entry
0709 13:03:33.665688 (+431927us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:33.665752 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.665860 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.081335 (+415475us) raft_consensus.cc:1685] finished
0709 13:03:34.081360 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.081501 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5897,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":13436,"thread_start_us":405623,"threads_started":1,"wal-append.queue_time_us":758302}
I20260709 13:03:34.044870 12612 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:03:34.075065  8753 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:03:34.092821  8823 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:34.095249  8722 leader_election.cc:336] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.580s (SENT)
I20260709 13:03:34.096174  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:34.097102  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 54226.4us
I20260709 13:03:34.080305 12417 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:34.057349 12658 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:34.102941  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 59612.6us
I20260709 13:03:34.109383  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:34.106115  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.106348 12664 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:34.108457 12661 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:34.080523 12334 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:34.104768 12417 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:34.111369  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:34.131704  8719 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:34.123266  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:33.852622 12647 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:33.873767 12648 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.126415 12612 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:34.128151 12661 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.854764 12616 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.157727 12616 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:34.157891 12612 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:34.072682 12572 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:34.149341 12657 raft_consensus.cc:476] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Not starting pre-election -- already a leader
I20260709 13:03:34.036048 12645 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:34.151405 12665 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:34.153075 12647 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:34.147859 12334 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:34.137980 12417 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.145638 12672 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:34.149883 12402 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:34.155200 12648 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:34.169816  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 74580.2us
W20260709 13:03:34.171661  8721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.002s	sys 0.007s
I20260709 13:03:34.176944  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:34.177429  8721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.003s	sys 0.008s
I20260709 13:03:34.128757 12658 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.163575 12661 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:34.188284  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 50612.5us
I20260709 13:03:34.201018  8826 raft_consensus.cc:2410] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:34.178897  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.201642  9353 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:34.175959  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:34.202817  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:34.176616  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 44981.8us
I20260709 13:03:34.211506  8821 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:34.211689  8719 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:34.204329  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:34.187668  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2383) took 1265 ms. Trace:
I20260709 13:03:34.218825  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:34.220947  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:34.223774  8794 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:34.224131  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:34.230433  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.234490  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:34.213160  8748 rpcz_store.cc:276] 0709 13:03:32.921945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.137609 (+215664us) service_pool.cc:224] Handling call
0709 13:03:33.138072 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.335006 (+196934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.335076 (+    70us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:33.335768 (+   692us) log.cc:844] Serialized 26 byte log entry
0709 13:03:34.028139 (+692371us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:34.028191 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.028274 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.187370 (+159096us) raft_consensus.cc:1685] finished
0709 13:03:34.187395 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.187583 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96711,"raft.queue_time_us":153645,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":93105,"thread_start_us":540024,"threads_started":1,"wal-append.queue_time_us":670208}
I20260709 13:03:34.240374  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.241315  8829 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:34.243659  8722 leader_election.cc:336] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.706s (SENT)
I20260709 13:03:34.244191  8722 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:34.245005  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 174458us
I20260709 13:03:34.178932 12662 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.185550 12572 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:34.221997 12616 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:34.172646 12665 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:34.184310 12417 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:34.249541  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.183785 12550 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:34.251113  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:34.259555  8821 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:34.243291 12566 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:34.253261  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:34.253576  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:34.254303  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:34.255733  8722 leader_election.cc:336] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.547s (SENT)
I20260709 13:03:34.249626  8719 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:34.257902  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 49480.3us
I20260709 13:03:34.247825  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:34.243462 12675 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.263880  8825 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:34.253010 12402 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:34.274086  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:34.263978  8722 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:34.179697 12334 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:34.265501  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:34.266587  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.260469 12665 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:34.283301  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.278869  8826 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:34.285961  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.287192  8826 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:34.297436  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.276898 12325 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:34.262764 12673 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:34.274502 12657 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:34.281114 12572 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:34.300619  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:34.300413  8794 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:34.282105 12612 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:34.256688 12662 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:34.313251  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 637) took 1229 ms. Trace:
I20260709 13:03:34.281610 12675 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.269865 12616 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:03:34.328634  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:34.330767  8826 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:34.405555  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.345746  8824 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:03:34.350963  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 96967.6us
I20260709 13:03:34.387214  8821 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:34.299376 12670 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:34.390810  8754 rpcz_store.cc:276] 0709 13:03:33.083844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.910417 (+826573us) service_pool.cc:224] Handling call
0709 13:03:33.977127 (+ 66710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:34.066385 (+ 89258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:34.066441 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:34.071463 (+  5022us) log.cc:844] Serialized 26 byte log entry
0709 13:03:34.135733 (+ 64270us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:34.135781 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.135856 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.312953 (+177097us) raft_consensus.cc:1685] finished
0709 13:03:34.312976 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.313161 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1613,"raft.queue_time_us":48076,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":125236,"threads_started":2,"wal-append.queue_time_us":181392}
I20260709 13:03:34.346827 12677 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:34.299602 12658 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:34.406320  8826 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
W20260709 13:03:34.441931 12679 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:03:34.388952 12657 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:34.390826 12612 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:34.481228 12676 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:03:34.488305 12678 log.cc:927] Time spent T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:03:34.422946 12677 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.520747  8825 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
W20260709 13:03:34.628013  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 375458us
I20260709 13:03:34.637996  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:34.631438  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 353213us
W20260709 13:03:34.644421  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 361799us
I20260709 13:03:34.464037 12658 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:34.503180 12612 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:34.655215  8722 leader_election.cc:336] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.651s (SENT)
W20260709 13:03:34.656347  8722 leader_election.cc:336] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.813s (SENT)
I20260709 13:03:34.656781  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:34.629555 12677 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:34.664414 12629 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:34.440428 12670 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.669073 12670 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:34.439668 12325 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.224s
I20260709 13:03:34.510522 12657 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:34.678738 12682 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:34.679522 12682 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.680786  9385 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:34.591003 12645 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.682756 12645 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:34.684257  9385 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 552) took 1787 ms (client timeout 2202 ms). Trace:
W20260709 13:03:34.684723  9385 rpcz_store.cc:269] 0709 13:03:32.896415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.896561 (+   146us) service_pool.cc:224] Handling call
0709 13:03:34.684015 (+1787454us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b5400469954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:34.684150 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4032873344}
I20260709 13:03:34.685211 12645 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:34.687332 12662 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:34.691744 12675 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:34.693171  8821 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:34.705684  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:34.694587  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.723829  8825 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:34.725822  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:34.731648  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:34.729316 12675 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:34.682042 12682 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:34.760082  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 262213us
W20260709 13:03:34.765647  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.725543 12325 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:03:34.767172  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.773705  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.784016 12615 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:03:34.785444 12665 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.790868 12685 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:34.791494 12685 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:34.792583 12685 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:34.771837 12325 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:34.799961  8750 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:34.800987 12325 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:34.803593  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 503261us
I20260709 13:03:34.804833  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:34.805882  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.806592  8825 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:34.807565  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.808132  8825 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:34.810200  8754 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:34.805657 12661 consensus_meta.cc:287] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.000s
I20260709 13:03:34.816107  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:34.804869 12402 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.525s	user 0.012s	sys 0.000s
I20260709 13:03:34.816787  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:34.823472 12695 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:34.824112 12695 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:34.823869 12612 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.824621 12695 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:34.829515 12665 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:34.829249 12677 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.830902 12402 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:34.832422 12657 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:34.834110 12661 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:34.815727 12687 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:34.836927 12661 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:34.838263  8828 raft_consensus.cc:2368] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10.
I20260709 13:03:34.839061 12630 raft_consensus.cc:3055] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:34.842577  8825 raft_consensus.cc:2376] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8. Re-sending same reply.
I20260709 13:03:34.846397  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:34.847420 12687 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:34.847885 12687 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:34.848315 12687 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:34.843175 12630 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:34.851558  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1873) took 2829 ms. Trace:
I20260709 13:03:34.851408 12630 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.851970  8674 rpcz_store.cc:276] 0709 13:03:32.021997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.022389 (+   392us) service_pool.cc:224] Handling call
0709 13:03:32.035263 (+ 12874us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:32.041222 (+  5959us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:32.063509 (+ 22287us) log.cc:844] Serialized 14 byte log entry
0709 13:03:32.333778 (+270269us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:32.352657 (+ 18879us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.352845 (+   188us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:32.352880 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.352958 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.851263 (+2498305us) raft_consensus.cc:1685] finished
0709 13:03:34.851294 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.851468 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":466,"prepare.queue_time_us":14734,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":60,"raft.queue_time_us":2296,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":16463,"thread_start_us":230995,"threads_started":1,"wal-append.queue_time_us":799359,"wal-append.run_cpu_time_us":3078,"wal-append.run_wall_time_us":1005116}
I20260709 13:03:34.856050  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:34.826457 12612 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:34.856225  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 627) took 2651 ms. Trace:
I20260709 13:03:34.857612  8681 rpcz_store.cc:276] 0709 13:03:32.204795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.204965 (+   170us) service_pool.cc:224] Handling call
0709 13:03:32.216694 (+ 11729us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:32.217181 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:32.217219 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:32.217794 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.363556 (+145762us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:32.363623 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.363732 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.855951 (+2492219us) raft_consensus.cc:1685] finished
0709 13:03:34.855979 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.856144 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5665,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":145248,"threads_started":1,"wal-append.queue_time_us":231856,"wal-append.run_cpu_time_us":2023,"wal-append.run_wall_time_us":1373335}
I20260709 13:03:34.859086 12657 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:34.861946 12677 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:34.865551  8647 connection.cc:256] RPC call timeout handler was delayed by -2.8785356810000167s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
I20260709 13:03:34.866391 12595 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.853178  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:34.867019 12595 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:34.867594 12661 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:34.868101 12661 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.868481 12661 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:34.869597 12595 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:34.870286  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.870766  8647 connection.cc:256] RPC call timeout handler was delayed by -2.7894356809999863s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1879s for a response
W20260709 13:03:34.871184  8647 connection.cc:256] RPC call timeout handler was delayed by -2.7066356809999661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19710000000000003s for a response
I20260709 13:03:34.871886 12595 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:34.871958 12572 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:03:34.878518  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 235548us
W20260709 13:03:34.883618  8647 connection.cc:256] RPC call timeout handler was delayed by -2.6139356810000436s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2074s for a response
W20260709 13:03:34.890090  8647 connection.cc:256] RPC call timeout handler was delayed by -2.595035681000013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20950000000000002s for a response
W20260709 13:03:34.890784  8647 connection.cc:256] RPC call timeout handler was delayed by -2.59323568100001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2097s for a response
I20260709 13:03:34.872565 12595 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.891978  8829 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
W20260709 13:03:34.891288  8647 connection.cc:256] RPC call timeout handler was delayed by -2.5709714969999595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
I20260709 13:03:34.894171  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:34.894506  8647 connection.cc:256] RPC call timeout handler was delayed by -2.5647768860001179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1932s for a response
W20260709 13:03:34.894881  8647 connection.cc:256] RPC call timeout handler was delayed by -2.5178714969999874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
I20260709 13:03:34.894992  9385 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
W20260709 13:03:34.895236  8647 connection.cc:256] RPC call timeout handler was delayed by -2.4879356809999535s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2214s for a response
W20260709 13:03:34.895533  8647 connection.cc:256] RPC call timeout handler was delayed by -2.3936714970000139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
I20260709 13:03:34.897306  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:34.899921  8575 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43478 (request call id 255) took 28902 ms (client timeout 15000 ms). Trace:
I20260709 13:03:34.900134  9384 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:34.903360 12595 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:34.900401  8575 rpcz_store.cc:269] 0709 13:03:05.983671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.984017 (+   346us) service_pool.cc:224] Handling call
0709 13:03:06.750897 (+766880us) spinlock_profiling.cc:243] Waited 47.8 ms on lock 0x7b54002d7dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2463090 00007f32e2501f3a 00007f32e82b81cc 00007f32e83b637e 00007f32e68772a4 00007f32e6877232 00007f32e6877161 00007f32e68770dc 00007f32e6876392 00007f32e083448c 00007f32e08338c6
0709 13:03:34.880014 (+28129117us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:17.351549 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:17.384624 (+ 33075us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:17.384663 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:03:17.384681 (+    18us) tablet.cc:662] Decoding operations
0709 13:03:17.468194 (+ 83513us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:17.468271 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:17.468299 (+    28us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:03:17.470077 (+  1778us) tablet.cc:701] Row locks acquired
0709 13:03:17.470101 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:03:17.470283 (+   182us) write_op.cc:270] Start()
0709 13:03:17.470404 (+   121us) write_op.cc:276] Timestamp: P: 1783602197470227 usec, L: 0
0709 13:03:17.470427 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:03:17.589226 (+118799us) log.cc:844] Serialized 9052 byte log entry
0709 13:03:17.808863 (+219637us) op_driver.cc:464] REPLICATION: finished
0709 13:03:17.957436 (+148573us) write_op.cc:301] APPLY: starting
0709 13:03:17.958587 (+  1151us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:18.126989 (+168402us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:18.147475 (+ 20486us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:18.285337 (+137862us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:18.296011 (+ 10674us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=23
0709 13:03:18.296108 (+    97us) write_op.cc:312] APPLY: finished
0709 13:03:18.296907 (+   799us) log.cc:844] Serialized 158 byte log entry
0709 13:03:18.297237 (+   330us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:18.297561 (+   324us) write_op.cc:454] Released schema lock
0709 13:03:18.297666 (+   105us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:18.298311 (+   645us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:18.298355 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5985623,"reported_tablets":47,"spinlock_wait_cycles":109993856,"child_traces":[["op",{"apply.queue_time_us":128536,"apply.run_cpu_time_us":9633,"apply.run_wall_time_us":348586,"num_ops":23,"prepare.queue_time_us":260003,"prepare.run_cpu_time_us":28787,"prepare.run_wall_time_us":306638,"raft.queue_time_us":115207,"raft.run_cpu_time_us":7335,"raft.run_wall_time_us":74420,"replication_time_us":336370,"thread_start_us":280654,"threads_started":4,"wal-append.queue_time_us":57415,"wal-append.run_cpu_time_us":34724,"wal-append.run_wall_time_us":1655379}]]}
I20260709 13:03:34.905812  8794 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:34.907821 12334 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:03:34.908238  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1876) took 2069 ms (client timeout 2097 ms). Trace:
W20260709 13:03:34.908704  8822 rpcz_store.cc:269] 0709 13:03:32.838464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.869790 (+ 31326us) service_pool.cc:224] Handling call
0709 13:03:34.907954 (+2038164us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400267ad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:34.908136 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2786550144}
I20260709 13:03:34.909595  8794 leader_election.cc:400] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8.
I20260709 13:03:34.909948  8794 leader_election.cc:403] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:03:34.910460 12595 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.910123  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.910019 12695 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.911113 12595 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.912498  8822 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:34.913736  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:34.914557  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 258238us
I20260709 13:03:34.915416  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:34.917443  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.918740  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.921365  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.929205  8822 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:34.923472  8821 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:34.927949 12595 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:34.944276 12595 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:34.944720  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.944882 12595 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.946250  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.952956  8822 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:34.954496  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.928079 12661 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:34.920142 12695 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:34.919360  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:34.933883 12462 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:34.938922  8722 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.820s (SENT)
I20260709 13:03:34.943100 12613 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:34.968924 12702 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:34.982422  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 72990us
W20260709 13:03:34.970180  8647 connection.cc:256] RPC call timeout handler was delayed by -2.2253714969999692s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
I20260709 13:03:34.971120 12705 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.983794  8794 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:34.979470  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:34.984669 12652 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8.
I20260709 13:03:34.984769 12705 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:34.992652 12702 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:34.972669  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.027493 12572 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:34.980717  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:35.030249  8827 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:34.980829  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:34.981930  9384 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:03:34.984123  8647 connection.cc:256] RPC call timeout handler was delayed by -2.2055714969999372s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
I20260709 13:03:34.996886 12687 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:34.972649  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.016213  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:35.018719  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:35.034521  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:35.035805 12572 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:35.022404  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 83525.8us
I20260709 13:03:35.024374 12657 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:35.024953 12705 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:35.039592  8722 leader_election.cc:336] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.888s (SENT)
I20260709 13:03:35.040086  8722 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:35.040135 12705 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:35.036011 12687 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:35.041795 12703 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.042397 12703 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:35.045128 12703 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:35.032291  8647 connection.cc:256] RPC call timeout handler was delayed by -1.9700145980000343s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
I20260709 13:03:35.035991  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:35.045773 12657 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:35.028610  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:03:35.046900  8722 leader_election.cc:336] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.980s (SENT)
I20260709 13:03:34.994217 12696 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.076943  8827 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:03:35.078301  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:35.079427  8822 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:35.038213  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:35.055931  8756 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:35.054225 12705 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:35.059077  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:35.080375  8826 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:03:35.083865  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55357us
I20260709 13:03:35.085232  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:35.050455 12661 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:35.033607  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 1994) took 2683 ms. Trace:
I20260709 13:03:35.089730  8822 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:35.089922  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:35.064469  8647 connection.cc:256] RPC call timeout handler was delayed by -1.7522111179999911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
I20260709 13:03:35.071084  8722 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:35.079355  8827 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:35.048950 12702 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:35.033288 12595 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:35.091627  8647 connection.cc:256] RPC call timeout handler was delayed by -1.6913333410000178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1869s for a response
I20260709 13:03:35.083223 12696 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:35.087291 12703 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:35.087672  8749 rpcz_store.cc:276] 0709 13:03:32.350717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.413063 (+ 62346us) service_pool.cc:224] Handling call
0709 13:03:32.703146 (+290083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.703500 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:32.708555 (+  5055us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.946412 (+2237857us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b1800157000. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:03:35.033225 (+ 86813us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:35.033285 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:35.033331 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.033410 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:35.033541 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4560,"prepare.queue_time_us":4838,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":62,"raft.queue_time_us":1108,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"spinlock_wait_cycles":5665536,"thread_start_us":86494,"threads_started":1,"wal-append.queue_time_us":2351134}
I20260709 13:03:35.096712  8827 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:35.097733  8826 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:03:35.099483  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 52630.4us
W20260709 13:03:35.100795  8647 connection.cc:256] RPC call timeout handler was delayed by -1.6136122420000447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
W20260709 13:03:35.107272  8647 connection.cc:256] RPC call timeout handler was delayed by -1.4169188060000124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20850000000000002s for a response
W20260709 13:03:35.108944  8647 connection.cc:256] RPC call timeout handler was delayed by -1.3369954320000943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22410000000000002s for a response
W20260709 13:03:35.109351  8647 connection.cc:256] RPC call timeout handler was delayed by -1.0464185859999589s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
I20260709 13:03:35.088127 12334 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:35.113878  8722 leader_election.cc:336] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.825s (SENT)
I20260709 13:03:35.095162 12705 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:35.087270 12706 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:35.109789  8647 connection.cc:256] RPC call timeout handler was delayed by -1.0378443910000215s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
I20260709 13:03:35.107792  8827 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:35.107712 12645 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:35.130887 12696 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:35.102138 12611 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:35.102458 12702 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.144060  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:35.108453 12687 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:35.149677  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:35.142843  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 89804.7us
I20260709 13:03:35.112965 12658 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:03:35.097420 12616 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:03:35.153131  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:35.110551 12703 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:35.108884 12661 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:03:35.141858 12334 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:35.148931 12702 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:35.151815 12611 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:35.165707  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51862.7us
W20260709 13:03:35.166347  8722 leader_election.cc:336] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.923s (SENT)
I20260709 13:03:35.166831  8722 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:35.148195 12595 consensus_queue.cc:1059] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:03:35.169701  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:35.159320 12689 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.178557  8722 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:35.179883  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:35.168411 12664 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:35.189555  9383 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:03:35.165266 12572 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:35.192673  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:35.191457 12566 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:35.199963 12702 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:35.192551 12689 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:35.168267 12353 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:35.168538 12696 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:35.167346 12645 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:35.214955  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 110237us
I20260709 13:03:35.145597 12706 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:35.215886  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:35.214794 12696 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:35.214567 12702 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:35.191097 12657 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:35.190655 12611 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:35.228732  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1700) took 1023 ms. Trace:
I20260709 13:03:35.242136  8824 rpcz_store.cc:276] 0709 13:03:34.205196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.205673 (+   477us) service_pool.cc:224] Handling call
0709 13:03:34.232799 (+ 27126us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:34.410802 (+178003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.225173 (+814371us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b540016ab54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:35.228638 (+  3465us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258808192}
I20260709 13:03:35.229835  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:35.237550  9383 raft_consensus.cc:2410] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:35.203014 12572 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:35.253253 12703 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:35.240535  8794 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.881s (SENT) [suppressed 37 similar messages]
I20260709 13:03:35.239254  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:35.254194 12703 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:35.290680 12618 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.290434 12703 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:35.291335 12618 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:35.253055 12572 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:35.294920 12572 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:35.294991 12634 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:35.295674 12634 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:35.244494 12611 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.296638 12611 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:35.298097 12634 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:35.300282  8579 catalog_manager.cc:5697] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 4 to 9, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 9 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:35.301079 12703 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:35.254992  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:35.193790 12664 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:35.237941 12702 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:35.244385 12713 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:35.256660 12689 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:35.257140 12657 raft_consensus.cc:697] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:35.229369 12353 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.272794 12696 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:35.255031  8748 raft_consensus.cc:2410] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:35.246191  8749 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:35.275348 12706 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:35.283573 12645 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:34.983484  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:35.270303  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1699) took 1066 ms. Trace:
I20260709 13:03:35.789507  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:35.790036  8828 rpcz_store.cc:276] 0709 13:03:34.203858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.204019 (+   161us) service_pool.cc:224] Handling call
0709 13:03:35.269996 (+1065977us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b5400126954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:35.270217 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1437728512}
W20260709 13:03:35.260005  8750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2366) took 3039 ms (client timeout 1881 ms). Trace:
W20260709 13:03:35.885671  8750 rpcz_store.cc:269] 0709 13:03:32.220289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.222716 (+  2427us) service_pool.cc:224] Handling call
0709 13:03:34.725596 (+2502880us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b5800083198. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:03:35.259707 (+534111us) spinlock_profiling.cc:243] Waited 73.1 ms on lock 0x7b54001586d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:35.259893 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205436672}
I20260709 13:03:35.302520  8579 catalog_manager.cc:5697] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 6 to 7, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 7 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:35.266417  9380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 636) took 1091 ms. Trace:
I20260709 13:03:35.249271  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:35.494107  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.021606  8678 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:03:35.498839  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:36.023052  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:36.023488  8679 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:36.023713  8678 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:35.501363  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:35.238327 12334 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:35.514907  8674 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:03:36.031077 12716 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.031780  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.031728 12716 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.032606  8674 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:35.515404  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:35.518949  8676 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:36.035434  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:36.036082  8676 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:36.037796 12716 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:35.305913 12664 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:35.495751 12702 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:35.495311 12713 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.042958 12713 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:35.522514  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:35.522549  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 369572us
I20260709 13:03:36.044205  8681 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:03:35.512573 12353 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:36.045266  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:35.523972  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.046988  8680 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:03:36.046821 12706 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:36.048095  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.048249 12723 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:36.049476  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.049896 12723 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:35.523922  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.050971  8673 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:36.051100  8679 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:03:36.052052  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.052740  8679 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:36.053666  8673 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:35.524959 12657 consensus_queue.cc:237] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:35.536013  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.054447  9380 rpcz_store.cc:276] 0709 13:03:34.175825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.175967 (+   142us) service_pool.cc:224] Handling call
0709 13:03:34.176341 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:34.190290 (+ 13949us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:34.645751 (+455461us) log.cc:844] Serialized 14 byte log entry
0709 13:03:35.264443 (+618692us) spinlock_profiling.cc:243] Waited 8.52 ms on lock 0x7b18003f89e0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e20a14bc 00007f32df3ef48b 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a
0709 13:03:35.265038 (+   595us) log.cc:844] Serialized 14 byte log entry
0709 13:03:35.265302 (+   264us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:35.265352 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:35.265407 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.266105 (+   698us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b540046a5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:35.266143 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:35.266315 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":580137,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":2798,"raft.queue_time_us":27888,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":198,"spinlock_wait_cycles":50866304,"thread_start_us":580440,"threads_started":2,"wal-append.queue_time_us":194110}
W20260709 13:03:35.570703 12638 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 2.315s	user 0.008s	sys 0.014s
I20260709 13:03:36.059799  9352 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:36.061538 12683 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:36.061882  9352 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:36.062073 12683 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.594732 12641 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:36.062558 12683 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:36.062965 12641 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:36.065302 12641 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:35.594849 12670 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:36.066149 12616 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:36.066819 12670 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:36.067785 12670 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.068291 12616 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:36.068912 12670 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:36.069682 12663 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:36.070145 12663 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:03:35.595476  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.070894 12353 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:36.070526 12663 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:36.073768 12683 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:35.606056  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:36.074936  8748 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:36.075083  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:36.076145  9352 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:36.077328 12353 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:36.077590 12592 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:36.078073 12592 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:35.642275  8645 connection.cc:256] RPC call timeout handler was delayed by -2.7688160970000126s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2719s for a response
I20260709 13:03:35.642484 12718 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.698352 12630 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:36.078747  8645 connection.cc:256] RPC call timeout handler was delayed by -2.58064686299997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
I20260709 13:03:35.488821 12618 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:35.718240  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:35.724141 12595 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.011s	sys 0.004s
W20260709 13:03:35.733220 12631 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 2.653s	user 0.000s	sys 0.026s
I20260709 13:03:35.741820 12639 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.744699  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:35.783339  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:36.097951  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 353363us
I20260709 13:03:36.099516  8722 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:35.783588  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.101312 12639 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:36.103361 12652 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.103816 12639 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:36.104003 12652 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:36.107420  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2355) took 4176 ms. Trace:
I20260709 13:03:36.109282  8823 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:03:36.109678  8823 pending_rounds.cc:85] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 3
I20260709 13:03:36.107884  8751 rpcz_store.cc:276] 0709 13:03:31.930591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.930769 (+   178us) service_pool.cc:224] Handling call
0709 13:03:31.944292 (+ 13523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.947391 (+  3099us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:32.349506 (+402115us) log.cc:844] Serialized 14 byte log entry
0709 13:03:32.930083 (+580577us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:32.996415 (+ 66332us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.996597 (+   182us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:32.996634 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.996714 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:35.732456 (+2735742us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:36.107056 (+374600us) raft_consensus.cc:1685] finished
0709 13:03:36.107186 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.107348 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15413,"prepare.queue_time_us":57564,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":883,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":67114,"spinlock_wait_cycles":5677440,"thread_start_us":887794,"threads_started":2,"wal-append.queue_time_us":666317}
W20260709 13:03:35.812222  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.813676 12402 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.110522  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:36.111217  8824 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:36.110970 12402 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.112257  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:35.820915  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.113485  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:36.113833  8747 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:35.790526  8749 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:35.828555  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.114816 12641 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:36.115255 12641 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:36.115656  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.116343  8752 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:36.116644  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:36.117394  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.118043  8752 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:36.118191 12700 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:35.848749  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 689) took 1794 ms. Trace:
I20260709 13:03:35.846202 12685 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:36.119073 12700 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:36.119544 12700 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:36.120191 12700 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:35.862303  8754 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:03:35.868399  9384 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.000s	sys 0.014s
W20260709 13:03:36.160962  9384 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.001s	sys 0.014s
W20260709 13:03:35.882509  8677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1864) took 3923 ms (client timeout 4176 ms). Trace:
W20260709 13:03:36.162398  8677 rpcz_store.cc:269] 0709 13:03:31.958647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.958843 (+   196us) service_pool.cc:224] Handling call
0709 13:03:35.882272 (+3923429us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54002567d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:35.882426 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242083200}
I20260709 13:03:35.889690  8579 catalog_manager.cc:5697] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 4 to 7, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 7 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:35.893064  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:36.165287  8750 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:35.898293 12417 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.004s	sys 0.008s
W20260709 13:03:35.917596  8794 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.012s	sys 0.028s
W20260709 13:03:35.925580  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:35.927389  8825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1730) took 1272 ms (client timeout 1504 ms). Trace:
W20260709 13:03:36.169181  8825 rpcz_store.cc:269] 0709 13:03:34.655090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.811679 (+156589us) service_pool.cc:224] Handling call
0709 13:03:35.927167 (+1115488us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b540009c9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:35.927318 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58995584}
I20260709 13:03:36.170750  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:36.171605  8825 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:35.930547 12611 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.633s	user 0.011s	sys 0.003s
I20260709 13:03:36.172583 12611 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:36.173278 12639 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.173909 12639 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:36.174958  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:36.175313 12716 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:36.175724  9382 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:35.290469  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1705) took 1046 ms. Trace:
W20260709 13:03:36.007826  8756 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.912s	user 0.070s	sys 0.107s
I20260709 13:03:36.027033  8675 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:03:36.077950 12353 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:36.078478 12592 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:36.178306 12611 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:36.080071  8755 raft_consensus.cc:2410] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:03:36.082999  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:36.077391  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:36.058424  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.182426  8674 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:36.181886  8681 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:36.086833  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:36.082470 12595 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.012s	sys 0.004s
I20260709 13:03:36.079003 12718 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:36.109901  8823 pending_rounds.cc:107] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:03:36.188648  8823 raft_consensus.cc:2889] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:35.830410  9355 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:36.191004  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 360669us
I20260709 13:03:36.186558  8796 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:36.177127 12716 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:36.154872  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:36.199193 12716 consensus_queue.cc:1243] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:36.199350  8827 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:36.199894  8579 catalog_manager.cc:5697] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 5 to 6, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 6 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:03:36.201526 12703 raft_consensus.cc:3055] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:36.201918 12703 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:36.202811 12703 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.203945 12703 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:36.124732 12652 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:36.157593 12664 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:36.208283 12664 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:36.208725 12664 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:36.202492  8579 catalog_manager.cc:5697] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:36.208761 12663 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.209756 12664 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:36.092314 12616 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:36.210834 12616 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:36.210338  8579 catalog_manager.cc:5697] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 4 to 5, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 5 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:36.211316 12616 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:36.212180  8579 catalog_manager.cc:5697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:36.215495 12595 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:36.216063 12664 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:36.216132 12595 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:36.219661  8756 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.124s	user 0.071s	sys 0.108s
I20260709 13:03:36.128278  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:36.155483  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:36.220912  8747 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:36.221185  8826 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:36.157297  9383 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:36.224427  8750 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:36.113487 12402 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:36.125422 12683 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:36.139178  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:36.123188  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:36.108116  8828 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:36.245971  8579 catalog_manager.cc:5697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:36.155750  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:36.115690 12641 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:36.176870  8829 rpcz_store.cc:276] 0709 13:03:34.244282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.253301 (+  9019us) service_pool.cc:224] Handling call
0709 13:03:35.290232 (+1036931us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b54002189d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:35.290368 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1306248448}
W20260709 13:03:36.080049  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:36.277848  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.158838  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:36.282833  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 124363us
I20260709 13:03:36.280956  8678 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:36.284662  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:36.285081  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.184376  8680 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:36.184809  9353 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:36.184774  8749 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
W20260709 13:03:36.187804  8794 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.012s	sys 0.028s
W20260709 13:03:36.357395  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 1.11684e+06us
I20260709 13:03:36.358927  8755 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:36.193012  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 115715us
I20260709 13:03:36.157266  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:36.197167 12639 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:36.198215 12353 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:36.178896 12417 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.086s	user 0.004s	sys 0.008s
I20260709 13:03:36.118777  8830 rpcz_store.cc:276] 0709 13:03:34.054347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.054677 (+   330us) service_pool.cc:224] Handling call
0709 13:03:34.076506 (+ 21829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:34.240523 (+164017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:34.240560 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:34.240608 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:34.240641 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.848390 (+1607749us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54002689d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:35.848456 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:35.848664 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376664960}
W20260709 13:03:36.371335  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 186662us
I20260709 13:03:36.225884  8754 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:03:36.244031  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 191153us
I20260709 13:03:36.244251  8673 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:36.252084  8748 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:36.259101  8747 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
W20260709 13:03:36.260666  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 103103us
I20260709 13:03:36.270639  8750 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:36.286970  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.375317  8677 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 6.
I20260709 13:03:36.179136  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:36.286255  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.192476 12645 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:36.317514  9383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 642) took 1040 ms. Trace:
I20260709 13:03:36.226821 12656 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:36.218554 12595 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:36.214087 12713 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.243496 12630 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:36.193207  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:36.328069  8681 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:36.334087  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.335843 12700 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:36.180697 12688 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.140s
I20260709 13:03:36.347607  8676 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
W20260709 13:03:36.434898  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 113557us
I20260709 13:03:36.276566  8579 catalog_manager.cc:5697] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 1 to 4, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 4 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:03:36.364967  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:36.435801  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.365767  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:36.436688  8747 raft_consensus.cc:2410] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:36.369258  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:36.370018  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:36.430244  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.438133  8755 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:36.372344  9353 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:36.439232  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 67014.5us
I20260709 13:03:36.439352  8748 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:36.381043  9383 rpcz_store.cc:276] 0709 13:03:35.248614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.248825 (+   211us) service_pool.cc:224] Handling call
0709 13:03:36.288427 (+1039602us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400520b54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:36.288639 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312874752}
I20260709 13:03:36.381973  8752 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:36.340737 12702 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.386835  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:36.393783  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:36.395077  8822 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:36.445394  8678 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:36.387656 12656 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:36.383562  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 100009us
I20260709 13:03:36.381436 12657 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:36.387247 12645 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.457897 12645 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:36.380043 12687 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.405697  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.389585 12618 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:36.407439  8672 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:36.388178 12663 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:36.400923 12595 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:36.412776  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:36.397455 12718 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:36.415825 12592 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.401163 12334 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.523620 12334 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.524835 12592 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:36.390069 12353 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:03:36.523694  9380 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:36.420176 12713 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:36.420778  9384 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:36.422487 12685 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:36.528597  8680 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:36.423110  8753 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:03:36.397161 12732 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:03:36.425885  9381 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:36.427460  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.404628 12630 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:36.424958  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.539911  8679 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:03:36.321794  8645 connection.cc:256] RPC call timeout handler was delayed by -2.4925229340000215s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
W20260709 13:03:36.540717  8645 connection.cc:256] RPC call timeout handler was delayed by -2.38812293400008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1932s for a response
W20260709 13:03:36.541091  8645 connection.cc:256] RPC call timeout handler was delayed by -2.0694468630000529s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20850000000000002s for a response
W20260709 13:03:36.541388  8645 connection.cc:256] RPC call timeout handler was delayed by -1.7146178049999889s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 13:03:36.541769  8645 connection.cc:256] RPC call timeout handler was delayed by -1.350346863000027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2884s for a response
W20260709 13:03:36.542119  8645 connection.cc:256] RPC call timeout handler was delayed by -1.2937229340000158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3148s for a response
I20260709 13:03:36.442759  8821 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 8: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:36.448340  8674 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:36.446712  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:36.545471  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 654) took 1457 ms. Trace:
W20260709 13:03:36.545784  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 99177.5us
I20260709 13:03:36.452333  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:36.461701  8754 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:36.447640 12462 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:36.470893  8673 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:03:36.472919  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 87206.3us
I20260709 13:03:36.449999 12656 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:36.423483 12730 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
W20260709 13:03:36.553820  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 101576us
I20260709 13:03:36.478456  8678 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:36.487461 12417 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:36.453801 12657 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:36.458860 12683 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:03:36.492341 12702 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:36.496834 12663 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:36.463088 12618 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.467420 12687 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:36.580153 12733 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:36.581830 12687 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:36.554934 12685 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.585310 12685 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:36.550508 12462 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:36.585999 12738 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:36.586463 12462 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:36.586753 12738 raft_consensus.cc:697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:36.587759 12738 consensus_queue.cc:237] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:36.554590 12639 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:36.592087 12639 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:36.593382 12723 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:36.593135 12639 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.593870 12723 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:36.594915 12723 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.595996 12683 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:36.596187 12723 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:36.596453 12683 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:36.597376 12683 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.565964 12657 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:36.602820 12334 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:36.566118 12652 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:03:36.604403 12652 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:36.604801 12652 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:36.606001 12618 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:36.605687 12652 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.461432 12716 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
W20260709 13:03:36.500530  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 50978.9us
I20260709 13:03:36.433863 12616 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.467432 12572 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.509081  9383 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
W20260709 13:03:36.519361  8702 debug-util.cc:398] Leaking SignalData structure 0x7b0800197ec0 after lost signal to thread 7529
I20260709 13:03:36.470422 12595 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:36.530298  8681 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:36.532014  8753 pending_rounds.cc:85] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 5
W20260709 13:03:36.539299  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.704082  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:36.547562  8719 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:36.546224  8756 rpcz_store.cc:276] 0709 13:03:35.088387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.088977 (+   590us) service_pool.cc:224] Handling call
0709 13:03:35.089362 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.089562 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:36.220236 (+1130674us) log.cc:844] Serialized 14 byte log entry
0709 13:03:36.504338 (+284102us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b18003accc0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:03:36.545068 (+ 40730us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:36.545144 (+    76us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:36.545209 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.545273 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.545409 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":447173,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":208,"raft.queue_time_us":64795,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":3035,"spinlock_wait_cycles":5767936,"thread_start_us":1448539,"threads_started":3,"wal-append.queue_time_us":150753}
I20260709 13:03:36.555326  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:36.564635  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:36.705917  8680 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:36.582533 12687 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:36.499548 12700 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:36.623289  8673 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:36.643579  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:36.646160  9352 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:36.661691  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 220242us
W20260709 13:03:36.708088  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 99145.6us
I20260709 13:03:36.700986 12683 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:36.702921  8702 debug-util.cc:398] Leaking SignalData structure 0x7b08001e2640 after lost signal to thread 8613
W20260709 13:03:36.714823  8702 debug-util.cc:398] Leaking SignalData structure 0x7b08001392c0 after lost signal to thread 8705
W20260709 13:03:36.715183  8702 debug-util.cc:398] Leaking SignalData structure 0x7b0800170760 after lost signal to thread 8706
W20260709 13:03:36.716483  8702 debug-util.cc:398] Leaking SignalData structure 0x7b080034e640 after lost signal to thread 8780
W20260709 13:03:36.716770  8702 debug-util.cc:398] Leaking SignalData structure 0x7b08003489e0 after lost signal to thread 8781
W20260709 13:03:36.718024  8702 debug-util.cc:398] Leaking SignalData structure 0x7b0800059c20 after lost signal to thread 8854
W20260709 13:03:36.704365  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.549347  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 634) took 1056 ms. Trace:
W20260709 13:03:36.718326  8702 debug-util.cc:398] Leaking SignalData structure 0x7b08003c6e80 after lost signal to thread 8855
I20260709 13:03:36.718643  8672 rpcz_store.cc:276] 0709 13:03:35.493064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.493214 (+   150us) service_pool.cc:224] Handling call
0709 13:03:36.549139 (+1055925us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b54002a8fd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:36.549281 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":858192256}
W20260709 13:03:36.721928  8702 debug-util.cc:398] Leaking SignalData structure 0x7b08003c9dc0 after lost signal to thread 9413
W20260709 13:03:36.722254  8702 debug-util.cc:398] Leaking SignalData structure 0x7b08003c1ba0 after lost signal to thread 9414
W20260709 13:03:36.723162  8702 debug-util.cc:398] Leaking SignalData structure 0x7b08003c2320 after lost signal to thread 12622
I20260709 13:03:36.710501  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:36.711548  8753 pending_rounds.cc:107] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:03:36.711973  9352 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:36.727041  8753 raft_consensus.cc:2889] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:36.729673  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 83173.8us
W20260709 13:03:36.731007  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 183526us
W20260709 13:03:36.731323  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 176089us
I20260709 13:03:36.731743  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:36.733134  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:36.733433  8722 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:36.735196  8647 leader_election.cc:400] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11.
I20260709 13:03:36.735695  8647 leader_election.cc:403] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:03:36.661275 12640 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:36.738042  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 544880us
I20260709 13:03:36.745194  9355 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:36.749912  8680 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:03:36.753545  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2392) took 1161 ms. Trace:
I20260709 13:03:36.755017  8753 rpcz_store.cc:276] 0709 13:03:35.592144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.709677 (+117533us) service_pool.cc:224] Handling call
0709 13:03:35.710068 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:36.737317 (+1027249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.753279 (+ 15962us) spinlock_profiling.cc:243] Waited 713 ms on lock 0x7b540038b3d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:36.753473 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1638773632}
I20260709 13:03:36.759285 12749 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:36.766407 12742 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:36.766800 12741 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:36.727898 12652 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:36.776388  8719 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:36.727949 12663 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:03:36.785688  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 212398us
I20260709 13:03:36.570464 12739 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.177s
W20260709 13:03:36.808903  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 55697.8us
W20260709 13:03:36.821550  8645 leader_election.cc:336] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 2.719s (SENT)
I20260709 13:03:36.737603 12611 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:36.828593  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 95214.8us
I20260709 13:03:36.753114 12702 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11.
I20260709 13:03:36.711328 12334 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 13:03:36.708877 12572 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:36.847865 12637 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:36.848320 12572 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:36.849514 12754 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:36.848438 12637 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:36.737251 12740 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:36.851202  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 93220.7us
I20260709 13:03:36.818049 12732 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:36.840833  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 130416us
I20260709 13:03:36.770812 12700 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:36.865857  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:36.754304 12733 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:36.756855 12640 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:36.744226 12670 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
W20260709 13:03:36.860805  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.867159  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:36.853840  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.884143  8680 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:36.867782  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:36.869383  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 138766us
I20260709 13:03:36.879441 12742 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:36.756544 12641 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.880681  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:36.900674  8672 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:36.823944 12402 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:36.887447  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 74366.9us
W20260709 13:03:36.907626  8645 leader_election.cc:336] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 3.148s (SENT)
I20260709 13:03:36.908252  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:36.891530  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:36.880156  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 61093.2us
W20260709 13:03:36.891723  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 157097us
I20260709 13:03:36.909198  8681 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:36.850168 12749 raft_consensus.cc:697] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:36.742969 12630 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:36.898912  8674 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:36.904757 12663 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:36.865694 12700 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:36.914177  8647 leader_election.cc:400] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:03:36.881753 12750 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:36.871294 12572 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:36.882972 12733 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.886988 12592 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.935328  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.937117  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.938741  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.932607  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.963312  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:36.945021  8749 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:36.900252 12685 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:36.967262  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:36.948637  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:36.905118 12687 consensus_queue.cc:1243] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:03:36.857885 12611 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:36.960500  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51158.3us
I20260709 13:03:36.980785  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.968225  8679 raft_consensus.cc:2393] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:36.948637 12700 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:36.974992  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 76727us
W20260709 13:03:36.983033  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 137672us
I20260709 13:03:36.983256  8676 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:03:36.984464  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.986004  8647 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:36.986725  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.987468  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.946633 12713 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:36.988179  8722 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:36.988384  8748 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:36.999446  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.999678  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:36.918061 12334 consensus_queue.cc:1059] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
W20260709 13:03:36.992668  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 82080.5us
I20260709 13:03:36.992165  8672 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:36.995929  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:37.004627  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:37.010345  8754 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:37.007180  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:36.950191 12417 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:37.001776  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:37.020684  8748 raft_consensus.cc:2410] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 13:03:36.992126  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 43604.5us
I20260709 13:03:36.961894 12733 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:37.016114  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:37.016330  8752 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:37.017061  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:37.031767  8755 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
W20260709 13:03:37.030155  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 163065us
I20260709 13:03:37.035705  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:36.974735 12685 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:36.986725 12687 raft_consensus.cc:3055] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:03:36.928550 12402 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.063910  8821 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:37.015487 12645 consensus_meta.cc:287] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.546s	user 0.013s	sys 0.000s
I20260709 13:03:37.065935  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:37.069480  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:37.001993 12759 raft_consensus.cc:3055] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:37.153986  8672 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:03:37.122807  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.153457  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:36.976956 12592 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.199757  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:37.154721  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 113723us
W20260709 13:03:37.207152  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.037129 12702 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:37.191885  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:37.210060  8822 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:36.989630 12769 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.170s
W20260709 13:03:37.217515  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 229414us
I20260709 13:03:37.213232  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:37.214555  8676 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:37.044690 12751 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:37.218437  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:37.057965 12685 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:37.044972 12417 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:37.156050 12737 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.444s	user 0.000s	sys 0.002s
I20260709 13:03:37.225528  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:37.068120 12334 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:37.242501  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:37.045644 12572 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.174s
I20260709 13:03:37.246670  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:36.962018 12637 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.236953  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:37.249471  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2397) took 1655 ms. Trace:
I20260709 13:03:37.249929  8751 rpcz_store.cc:276] 0709 13:03:35.593696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.125107 (+531411us) service_pool.cc:224] Handling call
0709 13:03:36.154164 (+ 29057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.154691 (+   527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.154723 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.168426 (+ 13703us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.426388 (+257962us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:36.426457 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.426673 (+   216us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.249186 (+822513us) raft_consensus.cc:1685] finished
0709 13:03:37.249216 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.249379 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":112,"thread_start_us":107992,"threads_started":1,"wal-append.queue_time_us":310788}
I20260709 13:03:37.251339 12780 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:37.252321  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:37.253868  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:37.255347 12771 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:37.255928 12771 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:37.256338 12771 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:37.259021 12774 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.259757 12774 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.208747 12740 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:37.263001 12740 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:37.068861 12723 consensus_queue.cc:1059] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.166s
I20260709 13:03:37.060397 12687 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:37.228654  8722 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:36.987972 12758 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.215s
W20260709 13:03:37.304631  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 269086us
I20260709 13:03:37.305812 12732 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:37.305980  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:37.155236 12759 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:37.320343 12759 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:37.266721  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.322495  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 69065.1us
I20260709 13:03:37.268760  8672 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:37.058297 12716 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.166s
I20260709 13:03:37.097133 12462 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.067101 12718 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.175s
W20260709 13:03:37.311825  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 289647us
I20260709 13:03:37.152716 12616 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:37.236029 12751 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:37.318377  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 89822.6us
I20260709 13:03:37.329612  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:37.351876  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:37.325376  8647 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:37.216547 12713 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:03:37.238827 12703 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:37.099864 12645 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.333281  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:37.336917  9352 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.007s	sys 0.007s
I20260709 13:03:37.066001 12726 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.201s
I20260709 13:03:37.246719 12753 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:37.328789 12702 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:37.375941  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 50668.7us
I20260709 13:03:37.407949  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:37.388360  8672 raft_consensus.cc:2393] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:03:37.386566  9352 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.007s	sys 0.007s
W20260709 13:03:37.409091  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 590962us
I20260709 13:03:37.409785  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:37.385504  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:37.399983 12726 consensus_queue.cc:1243] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:37.402426 12713 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:37.099215 12663 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.328961 12462 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:37.338429 12780 raft_consensus.cc:2764] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:03:37.286851 12741 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.119s
I20260709 13:03:37.436502 12664 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:37.437765 12618 consensus_meta.cc:287] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.831s	user 0.014s	sys 0.000s
I20260709 13:03:37.438218  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:37.439263  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:37.439779 12687 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:37.440320 12702 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:37.442381 12417 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:37.444893 12402 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:37.389108 12640 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:37.446983 12595 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:37.448347 12713 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:37.394377 12730 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:37.455862 12592 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:37.458097 12645 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:37.561118  8675 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: [3.2-9.5]
I20260709 13:03:37.461208 12769 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:37.463172 12726 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:37.463898 12703 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:37.466434 12781 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:03:37.468622  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 58917.2us
I20260709 13:03:37.376792 12764 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.480647  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:37.503875  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:37.378882 12763 raft_consensus.cc:3055] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:37.406932 12753 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:37.587178  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 234389us
I20260709 13:03:37.523479  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:37.526002  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 664) took 1451 ms. Trace:
I20260709 13:03:37.589396  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:37.590330  8826 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:37.530891  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:37.530891  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:37.190488 12742 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.591423  8751 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:37.434746 12641 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:37.594063  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:37.504242 12789 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.156510 12749 consensus_queue.cc:237] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.414961 12783 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:37.559966  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:37.579252 12687 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:37.460361 12739 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:37.562590  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:37.604226  8677 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
W20260709 13:03:37.586843 12656 consensus_meta.cc:287] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.021s	user 0.012s	sys 0.001s
I20260709 13:03:37.563763  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:37.530304 12618 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.565743 12790 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:37.556820 12738 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:03:37.566908 12770 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:37.432125 12732 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:37.569836  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 76780.7us
I20260709 13:03:37.572540  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:37.537093 12723 consensus_queue.cc:1243] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:03:37.569198 12766 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:37.576099  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:37.711867  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 135859us
I20260709 13:03:37.579227  8679 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:37.588048  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:37.713950  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 126020us
W20260709 13:03:37.592619  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 286767us
I20260709 13:03:37.589159  8749 rpcz_store.cc:276] 0709 13:03:36.074459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.360596 (+286137us) service_pool.cc:224] Handling call
0709 13:03:36.360950 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:36.964950 (+604000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.525621 (+560671us) spinlock_profiling.cc:243] Waited 584 ms on lock 0x7b54000901d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:37.525912 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1342727936}
W20260709 13:03:37.586091  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 151733us
W20260709 13:03:37.596022  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 188142us
I20260709 13:03:37.718559  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:37.574427 12402 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:37.574064 12726 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:37.737478 12792 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:03:37.737727 12726 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:37.738936 12791 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:37.739187 12726 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:37.740573 12784 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:03:37.722184  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:37.745096 12788 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:03:37.746090 12630 consensus_queue.cc:1059] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:37.728068 12641 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:37.747460 12630 raft_consensus.cc:3055] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:37.747896 12630 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:37.570900 12730 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:37.600955  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.600770  8674 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:03:37.584591 12763 raft_consensus.cc:740] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:37.755213  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:37.636694  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 284634us
I20260709 13:03:37.643187  8672 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:37.757100  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:37.605677  8677 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:37.586730 12753 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:37.726462 12683 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.998s	user 0.014s	sys 0.008s
I20260709 13:03:37.645506  8673 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 13:03:37.585435 12764 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:37.547294 12785 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.127s
I20260709 13:03:37.776877  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:37.664546  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:03:37.639210 12754 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.027s	sys 0.033s
I20260709 13:03:37.585913 12334 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:37.694175  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.792246  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.605134 12739 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:37.723004  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:37.710834  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:37.793476  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 70559.3us
I20260709 13:03:37.727491  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:37.647153 12783 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:37.728174  8679 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:37.735661  8676 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:03:37.735986 12786 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:37.569820 12774 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:37.806007  8679 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:37.750262  9355 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:37.808180  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 58022.1us
I20260709 13:03:37.644649  8680 raft_consensus.cc:2410] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:03:37.729532 12657 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.131s	user 0.016s	sys 0.005s
I20260709 13:03:37.766103  8674 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:37.773728 12743 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:37.705022 12790 raft_consensus.cc:697] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:37.812014  8679 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:37.773882  8722 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:37.810964  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:37.777606  8677 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:37.719431 12770 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:37.635031 12656 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.734488 12402 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.781703 12730 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:37.782274  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:37.783008  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:37.784310 12787 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:03:37.784704 12776 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.605553 12789 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.788002 12618 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:37.788589  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.751859 12641 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:37.776249  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:37.796327 12763 consensus_queue.cc:260] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.585271 12718 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
W20260709 13:03:37.780099 12754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.027s	sys 0.033s
I20260709 13:03:37.713598 12639 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:37.676699 12713 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:37.827049  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:37.761386 12703 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:03:37.832885  8679 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:37.797891 12702 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:37.837810  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:37.748804 12630 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:37.841018  8680 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:03:37.844389  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 70614.3us
I20260709 13:03:37.805337 12723 consensus_queue.cc:1059] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:03:37.826781 12786 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:37.806143 12740 consensus_queue.cc:1059] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
W20260709 13:03:37.848136  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 48584us
I20260709 13:03:37.798204 12739 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:37.917798  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:38.168229  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1903) took 1242 ms. Trace:
W20260709 13:03:37.862995  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.857142  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:38.226583  8647 connection.cc:256] RPC call timeout handler was delayed by -1.516231638000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
W20260709 13:03:38.215801  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.908435  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:37.877362  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:38.213876  9389 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 8: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:03:38.241930  8647 connection.cc:256] RPC call timeout handler was delayed by -1.3614316379999991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 13:03:38.242658  8647 connection.cc:256] RPC call timeout handler was delayed by -1.1112316380000493s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19930000000000003s for a response
I20260709 13:03:38.240002  8681 rpcz_store.cc:276] 0709 13:03:36.914989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.937177 (+ 22188us) service_pool.cc:224] Handling call
0709 13:03:36.971539 (+ 34362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.971869 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.971910 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.985465 (+ 13555us) log.cc:844] Serialized 26 byte log entry
0709 13:03:37.311467 (+326002us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:37.311530 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.311626 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:38.157046 (+845420us) raft_consensus.cc:1685] finished
0709 13:03:38.157084 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.157327 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":254819,"threads_started":1,"wal-append.queue_time_us":588080}
W20260709 13:03:38.243335  8647 leader_election.cc:336] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.543s (SENT)
I20260709 13:03:38.243407  8749 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:38.224763  8794 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:37.858019  9384 raft_consensus.cc:2393] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:37.826584 12462 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:38.225631  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:38.213528  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2350) took 1022 ms. Trace:
W20260709 13:03:38.031898  8793 leader_election.cc:336] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.532s (SENT)
I20260709 13:03:37.872581  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:37.952589  8830 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
W20260709 13:03:38.227367  8751 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.635s	user 0.012s	sys 0.000s
W20260709 13:03:37.897037  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.227020  8676 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:37.827600 12743 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:38.309861  8751 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:03:38.310369  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:38.327993  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:38.331724  8672 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:03:37.927030 12796 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:38.145578 12733 consensus_meta.cc:287] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.079s	user 0.014s	sys 0.000s
I20260709 13:03:38.328912  8673 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:38.345706  9353 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:37.927009 12778 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.921941 12611 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:38.201500 12770 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:37.943615 12716 raft_consensus.cc:3055] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:38.335618  9383 raft_consensus.cc:2368] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8.
I20260709 13:03:37.939718 12774 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:38.309423  8680 rpcz_store.cc:276] 0709 13:03:37.191131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.809866 (+618735us) service_pool.cc:224] Handling call
0709 13:03:37.833914 (+ 24048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.213151 (+379237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.213445 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:38.250380  8722 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:37.867100 12750 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.899s	user 0.010s	sys 0.004s
I20260709 13:03:37.830791 12657 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:37.918011 12797 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:38.305248 12592 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.014s	sys 0.013s
W20260709 13:03:38.343147  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.010s	sys 0.005s
W20260709 13:03:38.433643  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.010s	sys 0.005s
W20260709 13:03:38.434067  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 636645us
W20260709 13:03:37.946141 12700 consensus_meta.cc:287] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.948s	user 0.013s	sys 0.001s
I20260709 13:03:38.226208 12637 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:38.335696  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:38.190287 12766 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:37.918740 12794 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:38.347443 12780 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.918084 12806 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:37.850718 12732 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:38.368359 12703 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:37.936625 12736 raft_consensus.cc:3055] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:37.945209 12402 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:38.261955  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:03:38.361534  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 117174us
W20260709 13:03:38.439914 12592 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.019s	sys 0.015s
I20260709 13:03:37.866079 12751 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:38.199222 12652 consensus_meta.cc:287] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.388s	user 0.006s	sys 0.009s
I20260709 13:03:37.938812 12764 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:38.417120  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 166886us
I20260709 13:03:38.383601  8749 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:38.215441 12730 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:38.363835 12716 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:38.418329  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 193579us
I20260709 13:03:38.332038 12799 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:37.924358 12752 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.976s	user 0.004s	sys 0.000s
I20260709 13:03:38.437621  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:38.438261  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:38.565590  8680 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:38.441869  8645 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:38.566025 12810 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.452s	user 0.000s	sys 0.010s
W20260709 13:03:38.566450  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 124701us
I20260709 13:03:38.443135  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:38.446954  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 415135us
I20260709 13:03:38.433995  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:38.356624 12733 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:38.571007  8645 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:38.572065 12789 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:38.455654  8756 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:38.436641 12572 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.000s	sys 0.040s
I20260709 13:03:38.365433 12417 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:38.573297 12572 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.000s	sys 0.040s
I20260709 13:03:38.080246 12795 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:38.226327 12754 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:37.860436 12776 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:38.574734  8645 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:38.373136 12759 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:38.308734 12462 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.431680 12774 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.356875 12778 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:38.433843 12750 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:38.446153 12738 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.450865 12770 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.116937 12334 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:38.497663 12771 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.241s	user 0.013s	sys 0.004s
I20260709 13:03:37.927358 12783 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:38.363415 12641 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:38.506705 12772 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:03:38.509454 12726 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.770s	user 0.011s	sys 0.003s
I20260709 13:03:37.936340 12790 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:38.512771  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 707) took 1205 ms. Trace:
I20260709 13:03:38.485303 12818 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:03:38.476707 12685 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.212s	user 0.015s	sys 0.000s
I20260709 13:03:38.456565 12700 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:38.471371 12736 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:38.463881 12703 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:38.625103  8822 rpcz_store.cc:276] 0709 13:03:37.307051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.307227 (+   176us) service_pool.cc:224] Handling call
0709 13:03:37.307619 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.529606 (+221987us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.529657 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:37.547120 (+ 17463us) log.cc:844] Serialized 26 byte log entry
0709 13:03:37.678314 (+131194us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:37.678386 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.678503 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:38.512534 (+834031us) raft_consensus.cc:1685] finished
0709 13:03:38.512558 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.512700 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99655,"raft.queue_time_us":335646,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":57331,"thread_start_us":208317,"threads_started":2,"wal-append.queue_time_us":938901}
I20260709 13:03:38.524281 12716 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:38.485814 12753 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.709s	user 0.013s	sys 0.000s
I20260709 13:03:38.456444 12806 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:38.535357 12402 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:38.538601 12779 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:03:38.543133 12595 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:38.545610  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:38.553072  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:38.658509  9384 raft_consensus.cc:2410] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:03:38.563066 12645 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.663033 12645 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:38.565815  9383 raft_consensus.cc:2410] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:03:38.571292  8826 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 13:03:38.571218  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.667901  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.578099  9353 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:38.668501  8793 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.678s (SENT)
W20260709 13:03:38.668627  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 90636.2us
I20260709 13:03:38.486217 12732 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:38.668956  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:38.669939  9384 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:38.580135  8821 raft_consensus.cc:2410] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:03:38.466508  8672 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
W20260709 13:03:38.583850  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 94864.1us
I20260709 13:03:38.671855  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:38.673511  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:38.627056 12772 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:03:38.586723  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 96812.1us
I20260709 13:03:38.572454 12733 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:38.573530 12417 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:38.622799 12820 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:38.588913 12774 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:38.586268 12462 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:38.617846 12723 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:38.624747 12687 consensus_meta.cc:287] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.991s	user 0.012s	sys 0.003s
I20260709 13:03:38.644984 12778 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:38.648466  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 667) took 1857 ms. Trace:
I20260709 13:03:38.653635 12806 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:38.624504 12771 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:38.615638 12741 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.615648 12738 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:38.618860 12770 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:38.661881 12656 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:38.572965 12789 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.684517  8749 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:38.686079  8747 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:38.690222  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:38.638913 12726 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:38.701604  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:38.702342  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:38.702734  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:38.747792  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:38.748369  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.675328 12818 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:38.756523  8678 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:38.756716  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:38.757086  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:38.757442  8827 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:38.758216  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:38.759796  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.732136  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:38.760738  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:38.761229  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:38.761812  8749 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:38.733326  8753 rpcz_store.cc:276] 0709 13:03:36.791483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.791635 (+   152us) service_pool.cc:224] Handling call
0709 13:03:36.792137 (+   502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.816203 (+ 24066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.816237 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.838084 (+ 21847us) log.cc:844] Serialized 26 byte log entry
0709 13:03:37.515908 (+677824us) spinlock_profiling.cc:243] Waited 6.39 ms on lock 0x7b18003acea0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:03:37.713180 (+197272us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:37.713222 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.713301 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:38.647998 (+934697us) raft_consensus.cc:1685] finished
0709 13:03:38.648033 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.648373 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20298,"raft.queue_time_us":95364,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":46936,"spinlock_wait_cycles":14688640,"thread_start_us":196032,"threads_started":1,"wal-append.queue_time_us":1573197}
I20260709 13:03:38.700316 12716 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:38.767836  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:38.768746 12770 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:38.768998  8824 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:38.768771  8825 raft_consensus.cc:2393] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:38.769893  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:38.676837 12772 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:38.770557  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:38.641611 12685 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:38.771067  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:38.771238 12775 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.738731 12703 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:38.771862  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:38.752875 12750 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:38.739348  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 317467us
I20260709 13:03:38.743696  9383 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:38.705325 12732 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:38.704923 12820 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.744625  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:38.703706  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:38.748826  9381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 798) took 1837 ms. Trace:
W20260709 13:03:38.749670  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 325398us
I20260709 13:03:38.752377  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2340) took 1770 ms. Trace:
I20260709 13:03:38.753434  8822 raft_consensus.cc:2393] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:38.666304 12645 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:38.758948  8755 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:38.806231  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:38.758538  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:38.762181  8747 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:38.763482  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:38.727681 12776 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:38.772224  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:38.657021 12753 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:38.802551  9381 rpcz_store.cc:276] 0709 13:03:36.911204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.911350 (+   146us) service_pool.cc:224] Handling call
0709 13:03:36.940308 (+ 28958us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.940828 (+   520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.940874 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.970026 (+ 29152us) log.cc:844] Serialized 26 byte log entry
0709 13:03:37.211795 (+241769us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:37.211846 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.211929 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:38.748362 (+1536433us) raft_consensus.cc:1685] finished
0709 13:03:38.748413 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.748676 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34429,"raft.queue_time_us":49561,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":18026,"thread_start_us":174124,"threads_started":1,"wal-append.queue_time_us":1338092}
I20260709 13:03:38.751233 12726 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.802706  8673 rpcz_store.cc:276] 0709 13:03:36.981985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.767360 (+785375us) service_pool.cc:224] Handling call
0709 13:03:37.795213 (+ 27853us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.346083 (+550870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.752016 (+405933us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b54002562d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:38.752291 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1226814848}
I20260709 13:03:38.798859  8752 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:38.748010 12670 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:38.822938  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:38.823779  8680 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:03:38.824383  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:38.825855 12685 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:38.754676 12796 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:38.775732 12733 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:38.807962  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 137738us
W20260709 13:03:38.848409  8793 leader_election.cc:336] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.895s (SENT)
W20260709 13:03:38.808537  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 232320us
I20260709 13:03:38.808743  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:38.812076  8753 raft_consensus.cc:2410] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:03:38.815639  8754 raft_consensus.cc:2410] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:03:38.734673  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.822819 12726 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:38.850724  8647 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:38.493306 12803 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.326s
I20260709 13:03:38.740996 12741 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:38.861778  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:38.807966 12657 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:38.810408 12778 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.876245  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:38.807447 12820 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:38.564355 12799 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:38.833807 12785 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:03:38.877692  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 74508.5us
I20260709 13:03:38.809442 12770 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:03:38.760308 12789 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:38.469213 12794 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:38.811210 12818 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.120s
I20260709 13:03:38.440868 12797 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:38.470216 12780 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:38.573952 12795 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:38.771800 12775 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:38.476338 12786 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:38.575078 12754 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:38.454979 12743 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:39.142323  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 291707us
W20260709 13:03:39.099726  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 295696us
I20260709 13:03:39.148870  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:38.833214 12645 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:39.149638  9352 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:39.037853 12685 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.120s
I20260709 13:03:38.878796  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:39.158092  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:39.154848  8647 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:38.832980 12417 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:39.021641  8680 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
W20260709 13:03:39.171344  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.176048  8679 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:39.192874  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:38.853519 12611 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.195238  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:39.192987  8647 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:39.199699  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:39.203526  8752 raft_consensus.cc:2368] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
I20260709 13:03:39.203900  8754 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:39.204926  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:39.206017  9381 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:39.218647  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:39.219477  8752 raft_consensus.cc:2376] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6. Re-sending same reply.
I20260709 13:03:39.220598  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:39.221798  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:38.973743 12778 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:39.223764 12685 consensus_queue.cc:1243] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:39.118845 12825 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:39.226698 12825 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:39.228106  8752 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: [9.9-11.10]
I20260709 13:03:39.229574  8752 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:39.230837  8752 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:39.197293 12785 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:39.231526 12750 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.235000 12750 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:39.236449  8677 raft_consensus.cc:3055] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:39.236904  8677 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:39.237785  8677 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:39.118582 12782 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:39.239662 12826 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.138s	user 0.001s	sys 0.001s
I20260709 13:03:39.239259 12782 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:39.227043  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:39.247229  9386 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:39.242738  8673 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:03:39.245608  8754 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:39.186507 12334 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.161003 12834 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:38.867169 12733 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:39.293931  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:39.272712 12806 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.507s	user 0.001s	sys 0.011s
W20260709 13:03:39.283509 12713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.016s	sys 0.005s
I20260709 13:03:39.272344 12685 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:39.288122 12739 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.956s	user 0.012s	sys 0.002s
I20260709 13:03:39.275856 12645 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.228173 12829 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.381916  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:39.383302  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:39.283177 12751 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:39.301424 12723 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:03:39.399670  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.243903 12611 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:39.380690 12713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.016s	sys 0.005s
W20260709 13:03:39.428897  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 550268us
I20260709 13:03:39.371554 12670 raft_consensus.cc:697] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:03:39.466223  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 273344us
W20260709 13:03:39.468926  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 165210us
I20260709 13:03:39.470425  8794 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:39.471871  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:39.473155  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:39.475871  8673 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:39.396171 12764 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:39.296334 12334 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:39.495484  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 334225us
I20260709 13:03:39.161775 12796 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:39.377023 12806 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:39.451587 12846 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:39.516569 12687 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.000s	sys 0.021s
W20260709 13:03:39.525051  8647 connection.cc:256] RPC call timeout handler was delayed by -1.1945595639999738s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3852s for a response
I20260709 13:03:39.428182 12739 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:39.522351 12723 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:39.476284 12639 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:39.563724  8681 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:39.430238 12645 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:39.553733  8822 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.008s	sys 0.016s
W20260709 13:03:39.552217 12687 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.000s	sys 0.021s
I20260709 13:03:39.463382 12751 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:39.568388  8822 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.008s	sys 0.016s
I20260709 13:03:39.569396 12753 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:39.570199 12799 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:39.569844 12639 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:39.570986 12851 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:39.573983 12839 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 13:03:39.553550 12848 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:39.575498 12775 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:39.576056 12782 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:39.576539 12670 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:39.579278 12780 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:39.539686 12741 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:39.584429  8747 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:39.604123 12753 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.622722 12753 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:39.623862 12733 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:39.590803 12785 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:39.587518  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 739103us
I20260709 13:03:39.447335 12829 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:39.564733 12783 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:39.574143 12637 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.120s	user 0.012s	sys 0.000s
W20260709 13:03:39.614506  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 144189us
I20260709 13:03:39.618556  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:39.640743  9381 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:39.641053  8794 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:39.642153  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:39.583194 12806 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:39.603574 12771 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:39.621464 12730 consensus_meta.cc:287] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.101s	user 0.014s	sys 0.002s
I20260709 13:03:39.663910  9380 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:39.672447 12794 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:39.627514 12797 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:39.675581  8754 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:39.677022 12645 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:39.677448 12833 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:39.678020 12833 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.677037  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:39.685073 12685 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:39.685750 12685 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.686193 12685 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:39.639838 12785 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:03:39.678427 12833 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:39.695377  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:39.696501  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.642300  9381 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:39.647559 12767 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:39.698450 12767 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:39.699887 12767 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:39.701476 12767 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:39.633204 12772 consensus_meta.cc:287] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.862s	user 0.013s	sys 0.000s
I20260709 13:03:39.696484  8673 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:39.622063  8796 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:39.691421 12840 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:03:39.681571 12783 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:39.733539 12783 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.734444 12783 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:39.736733 12854 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:03:39.704262 12716 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.925s	user 0.009s	sys 0.004s
I20260709 13:03:39.708226  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:39.704255 12799 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:03:39.742720  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:39.746071  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 131263us
I20260709 13:03:39.746338  8673 raft_consensus.cc:2393] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:03:39.702419  8793 leader_election.cc:336] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.189s (SENT)
W20260709 13:03:39.757998  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 55597.4us
I20260709 13:03:39.758133  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:39.759303  9381 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:03:39.762754  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 242374us
W20260709 13:03:39.762307  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 235251us
I20260709 13:03:39.772091  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:39.767977  8645 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:39.777984  8796 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:39.781489 12645 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:39.791781  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1907) took 1437 ms. Trace:
I20260709 13:03:39.793073 12763 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:39.792342  8676 rpcz_store.cc:276] 0709 13:03:38.354367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.556684 (+202317us) service_pool.cc:224] Handling call
0709 13:03:38.557074 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:38.557551 (+   477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.557592 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:38.588298 (+ 30706us) log.cc:844] Serialized 26 byte log entry
0709 13:03:38.849403 (+261105us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:38.849444 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.849517 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:39.791496 (+941979us) raft_consensus.cc:1685] finished
0709 13:03:39.791520 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.791704 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":60112,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":99,"thread_start_us":218438,"threads_started":1,"wal-append.queue_time_us":1039309}
I20260709 13:03:39.735625 12772 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:39.786427  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1909) took 1365 ms. Trace:
I20260709 13:03:39.794592  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:39.795833  8681 rpcz_store.cc:276] 0709 13:03:38.421133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.570075 (+148942us) service_pool.cc:224] Handling call
0709 13:03:38.570459 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.564259 (+993800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.786109 (+221850us) spinlock_profiling.cc:243] Waited 993 ms on lock 0x7b5400175554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:39.786348 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2283827072}
W20260709 13:03:39.794127 12759 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.197s	user 0.014s	sys 0.000s
I20260709 13:03:39.798658  8676 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:39.813563  9387 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:03:39.811681  9352 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:39.818374  9381 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:03:39.821875 12595 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.149s	user 0.008s	sys 0.006s
I20260709 13:03:39.822963  8674 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:39.823994 12819 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.824783 12819 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:39.827291  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:39.838961  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.795331 12859 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:39.845600 12334 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:39.847080 12831 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.847412 12759 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:39.855834 12881 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:39.856531 12829 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:39.830370 12687 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.864737  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 665) took 1629 ms. Trace:
I20260709 13:03:39.866982  8678 rpcz_store.cc:276] 0709 13:03:38.235719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.235893 (+   174us) service_pool.cc:224] Handling call
0709 13:03:38.756119 (+520226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.757017 (+   898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.864447 (+1107430us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b54002562d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:39.864655 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1195674880}
I20260709 13:03:39.868423  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:39.869697 12726 consensus_meta.cc:287] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.938s	user 0.012s	sys 0.001s
I20260709 13:03:39.870198 12726 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:39.872644 12726 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:39.869693  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:39.880159  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:39.881213  8678 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:39.887146  8673 raft_consensus.cc:2376] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10. Re-sending same reply.
W20260709 13:03:39.892800 12853 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:03:39.893819  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:39.895351 12850 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:03:39.897189  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2412) took 1205 ms. Trace:
I20260709 13:03:39.897728  8749 rpcz_store.cc:276] 0709 13:03:38.691597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.778599 (+ 87002us) service_pool.cc:224] Handling call
0709 13:03:38.779151 (+   552us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:38.816789 (+ 37638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.816823 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:38.870547 (+ 53724us) log.cc:844] Serialized 26 byte log entry
0709 13:03:39.400276 (+529729us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:39.400339 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.400445 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:39.896895 (+496450us) raft_consensus.cc:1685] finished
0709 13:03:39.896925 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.897106 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27867,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":20660,"thread_start_us":334573,"threads_started":1,"wal-append.queue_time_us":785192}
I20260709 13:03:39.887917  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:39.859830 12759 raft_consensus.cc:697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:39.906713  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:39.908279  9383 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:03:39.907349 12462 consensus_meta.cc:287] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.169s	user 0.009s	sys 0.006s
I20260709 13:03:39.820513 12783 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:39.931445 12820 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.745s	user 0.008s	sys 0.006s
I20260709 13:03:39.932011 12820 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:39.934266 12645 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:39.935424  8749 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:39.908597 12846 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:39.847898 12831 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:39.884639 12687 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:39.945922 12778 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:39.953131 12611 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:39.953738 12703 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:39.954850 12750 consensus_meta.cc:287] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.719s	user 0.012s	sys 0.002s
I20260709 13:03:39.928597 12462 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:39.959924  9382 raft_consensus.cc:2368] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
W20260709 13:03:39.961301  9382 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 569) took 2088 ms (client timeout 2039 ms). Trace:
W20260709 13:03:39.961735  9382 rpcz_store.cc:269] 0709 13:03:37.873463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.261477 (+388014us) service_pool.cc:224] Handling call
0709 13:03:39.961037 (+1699560us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b5400469954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:39.961201 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3427533568}
I20260709 13:03:39.927186 12753 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:39.950304 12630 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:39.955384 12750 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:39.986111 12750 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:39.953137 12639 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:39.942935 12864 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
W20260709 13:03:40.016502 12874 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:03:40.068593  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.071882  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2329) took 3126 ms. Trace:
I20260709 13:03:40.072594  8675 rpcz_store.cc:276] 0709 13:03:36.944730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.571786 (+627056us) service_pool.cc:224] Handling call
0709 13:03:37.582537 (+ 10751us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.582975 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.583015 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:37.587474 (+  4459us) log.cc:844] Serialized 26 byte log entry
0709 13:03:37.778024 (+190550us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:37.778083 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.778164 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.014975 (+2236811us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b2400195ea8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df2929c4 00007f32df290acb 00007f32df44bdc9 00007f32df44ba55 00007f32df44afb0 00007f32e20c1488 00007f32e20c1be8 00007f32e2099398 00007f32e209476a 00007f32e2093bb6
0709 13:03:40.070935 (+ 55960us) raft_consensus.cc:1685] finished
0709 13:03:40.071031 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.071563 (+   532us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56232832,"thread_start_us":131410,"threads_started":1,"wal-append.queue_time_us":335408}
W20260709 13:03:40.078895  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.083609  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1916) took 1352 ms. Trace:
W20260709 13:03:40.086141  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.084347  8825 rpcz_store.cc:276] 0709 13:03:38.731491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.798020 (+ 66529us) service_pool.cc:224] Handling call
0709 13:03:38.800609 (+  2589us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:38.852423 (+ 51814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.852461 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:38.888523 (+ 36062us) log.cc:844] Serialized 26 byte log entry
0709 13:03:39.426120 (+537597us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:39.726596 (+300476us) log.cc:844] Serialized 14 byte log entry
0709 13:03:39.729692 (+  3096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.731084 (+  1392us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.082911 (+351827us) raft_consensus.cc:1685] finished
0709 13:03:40.082955 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.083334 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":283700,"prepare.queue_time_us":220407,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":30003,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":19586,"thread_start_us":653106,"threads_started":2,"wal-append.queue_time_us":889394}
I20260709 13:03:39.979461 12685 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:40.089710 12687 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:39.994354 12759 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:40.091908 12462 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:40.069932  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:40.111925 12772 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:40.114171  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:40.116178 12890 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
I20260709 13:03:40.125695  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:40.135251 12774 consensus_meta.cc:287] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.382s	user 0.010s	sys 0.006s
I20260709 13:03:40.135577 12854 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:40.136607 12774 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:40.140242 12774 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:40.153005  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 668) took 1376 ms. Trace:
I20260709 13:03:40.153654 12833 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:40.157374 12833 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:40.150377  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.168344 12703 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.169534 12703 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:40.170267 12718 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.171404 12718 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:40.179382  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:40.181041 12846 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.182253 12846 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:40.187000  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:40.194947  8675 raft_consensus.cc:2410] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:03:40.186403 12846 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:40.189206  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:40.200314  8673 raft_consensus.cc:2410] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:03:40.206234 12685 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:40.210355  8754 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:40.153609  8672 rpcz_store.cc:276] 0709 13:03:38.776297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.776535 (+   238us) service_pool.cc:224] Handling call
0709 13:03:38.814285 (+ 37750us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.535062 (+720777us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:39.535104 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:39.535161 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:39.535201 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.152310 (+617109us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b54000da2d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:40.152384 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.152736 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":222200,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":29752,"spinlock_wait_cycles":899201408,"thread_start_us":311186,"threads_started":1}
I20260709 13:03:40.218288  8677 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:40.220947  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 712) took 2469 ms. Trace:
I20260709 13:03:40.221446  8826 rpcz_store.cc:276] 0709 13:03:37.751502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.751706 (+   204us) service_pool.cc:224] Handling call
0709 13:03:37.752108 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:38.667383 (+915275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.220282 (+1552899us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b5400125f54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:40.220731 (+   449us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1883412352}
W20260709 13:03:40.224530  9380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 564) took 2427 ms (client timeout 1619 ms). Trace:
W20260709 13:03:40.225270  9380 rpcz_store.cc:269] 0709 13:03:37.796815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.811264 (+ 14449us) service_pool.cc:224] Handling call
0709 13:03:40.223930 (+2412666us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54005ea1d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:40.224218 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3308894080}
W20260709 13:03:40.227957  8830 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 708) took 2647 ms (client timeout 1674 ms). Trace:
W20260709 13:03:40.228626  8830 rpcz_store.cc:269] 0709 13:03:37.580616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.580799 (+   183us) service_pool.cc:224] Handling call
0709 13:03:40.227360 (+2646561us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b540032ee54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:40.227667 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":822464256}
I20260709 13:03:40.240873  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 803) took 2232 ms. Trace:
I20260709 13:03:40.205483 12718 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:40.241207  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:40.243680  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2407) took 1560 ms. Trace:
I20260709 13:03:40.246353  8755 rpcz_store.cc:276] 0709 13:03:38.682972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.685200 (+  2228us) service_pool.cc:224] Handling call
0709 13:03:38.732849 (+ 47649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.809413 (+ 76564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.241475 (+1432062us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b540038c2d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:40.243387 (+  1912us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59118336}
I20260709 13:03:40.241600  9384 rpcz_store.cc:276] 0709 13:03:38.003654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.662060 (+658406us) service_pool.cc:224] Handling call
0709 13:03:38.669740 (+  7680us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.670410 (+   670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.234536 (+1564126us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x7b5400477a50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:40.235272 (+   736us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16434432}
W20260709 13:03:40.255059 12892 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:03:40.255295  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:40.256650 12718 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:40.253423  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:40.265029  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:40.263270  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:40.262533  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:40.266319  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:40.267621 12656 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.269420  8830 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:40.234166 12670 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:03:40.268733 12656 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:40.276814  8676 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:40.279217 12656 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:40.279790  8676 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:40.281307  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:40.281574 12789 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:40.292049  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:40.299654  8672 raft_consensus.cc:2410] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:03:40.301525  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:40.306797  8830 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:40.300740  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:40.307787  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:40.311178  8755 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:40.311919  9384 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:40.309302  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:40.316195  8826 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:40.317703  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:40.319420  8826 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
W20260709 13:03:40.320031  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.320895  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:40.321542 12713 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.322023  8826 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:40.331089  9383 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:40.327178  8752 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:40.351665  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2006) took 3421 ms. Trace:
I20260709 13:03:40.379947  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:40.349895  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.359598  9387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 565) took 2562 ms. Trace:
I20260709 13:03:40.356595  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:40.335824  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:40.339278  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:40.325531  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2022) took 1133 ms. Trace:
I20260709 13:03:40.414494  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:40.325266  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:40.325606  8825 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:40.337560  8754 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 6.
I20260709 13:03:40.369077 12783 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:40.417678 12818 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:40.418231  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:40.419198 12818 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:40.419929 12818 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:40.313055 12703 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:40.322491 12713 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:40.374564 12845 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 1.183s	user 0.001s	sys 0.001s
I20260709 13:03:40.419296  8675 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:40.411993  8747 rpcz_store.cc:276] 0709 13:03:39.191985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.194895 (+  2910us) service_pool.cc:224] Handling call
0709 13:03:40.324831 (+1129936us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b540018d8d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:40.325218 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":883761024}
W20260709 13:03:40.377218 12847 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.186s	user 0.004s	sys 0.000s
I20260709 13:03:40.347687  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1909) took 1646 ms. Trace:
I20260709 13:03:40.376289  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:40.344503 12640 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.083s
W20260709 13:03:40.428752  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.372838 12841 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.983s	user 0.007s	sys 0.000s
W20260709 13:03:40.381148 12869 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.396s	user 0.000s	sys 0.004s
I20260709 13:03:40.407691  9387 rpcz_store.cc:276] 0709 13:03:37.797145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.574833 (+777688us) service_pool.cc:224] Handling call
0709 13:03:38.575250 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:39.826848 (+1251598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.358785 (+531937us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54002369d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:40.359323 (+   538us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2839850240}
I20260709 13:03:40.432569 12640 consensus_queue.cc:1243] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:40.428016  8824 rpcz_store.cc:276] 0709 13:03:38.701584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.701755 (+   171us) service_pool.cc:224] Handling call
0709 13:03:40.347056 (+1645301us) spinlock_profiling.cc:243] Waited 8.19 ms on lock 0x7b540009d3d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:40.347374 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18848000}
I20260709 13:03:40.433358  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:40.433828  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 141361us
I20260709 13:03:40.434396  8675 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:40.434818  8749 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:40.437505  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:40.438724 12703 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:40.439738 12703 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:40.439296 12818 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:40.441188 12818 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:40.356024 12743 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:40.367642 12825 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:40.445832 12713 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:40.448038  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:40.449209  8747 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:03:40.449357  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1918) took 1717 ms. Trace:
I20260709 13:03:40.456143 12736 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:40.458220 12736 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:40.460105 12795 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:40.460269  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1917) took 1728 ms. Trace:
I20260709 13:03:40.450430  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:40.374564  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 64893.4us
W20260709 13:03:40.445542  8793 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 4.350s (SENT) [suppressed 26 similar messages]
W20260709 13:03:40.438169 12763 consensus_meta.cc:287] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.644s	user 0.005s	sys 0.019s
I20260709 13:03:40.478127  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2361) took 1039 ms. Trace:
W20260709 13:03:40.483597  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46261.5us
I20260709 13:03:40.484663 12640 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:40.485867  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:40.344838 12801 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.142s
I20260709 13:03:40.486953  8672 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:40.488968 12820 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:40.338371 12740 consensus_queue.cc:1059] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.153s
I20260709 13:03:40.326116 12793 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.494261 12795 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:40.465713  8822 rpcz_store.cc:276] 0709 13:03:38.731813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.813211 (+ 81398us) service_pool.cc:224] Handling call
0709 13:03:38.813574 (+   363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:39.581928 (+768354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.582006 (+    78us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:40.208437 (+626431us) spinlock_profiling.cc:243] Waited 8.87 ms on lock 0x7b2c000f2de8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e0df2a11 00007f32e0e02372 00007f32e2534a01 00007f32e24f943a 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2
0709 13:03:40.209907 (+  1470us) log.cc:844] Serialized 52 byte log entry
0709 13:03:40.253253 (+ 43346us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:40.254333 (+  1080us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.255444 (+  1111us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.255709 (+   265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.255842 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.459812 (+203970us) raft_consensus.cc:1685] finished
0709 13:03:40.459843 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.460067 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51953,"prepare.queue_time_us":739,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":97,"raft.queue_time_us":351709,"raft.run_cpu_time_us":607,"raft.run_wall_time_us":46543,"spinlock_wait_cycles":20409728,"thread_start_us":772685,"threads_started":2,"wal-append.queue_time_us":99340}
W20260709 13:03:40.495438  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 703) took 3342 ms (client timeout 1780 ms). Trace:
I20260709 13:03:40.449939  8827 rpcz_store.cc:276] 0709 13:03:38.732087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.159037 (+426950us) service_pool.cc:224] Handling call
0709 13:03:39.159492 (+   455us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:39.281311 (+121819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.281365 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:39.296061 (+ 14696us) log.cc:844] Serialized 52 byte log entry
0709 13:03:39.563171 (+267110us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:39.723280 (+160109us) log.cc:844] Serialized 14 byte log entry
0709 13:03:39.723819 (+   539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.724005 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.379945 (+655940us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:40.448713 (+ 68768us) raft_consensus.cc:1685] finished
0709 13:03:40.448847 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.449124 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":147499,"prepare.queue_time_us":202689,"prepare.run_cpu_time_us":1121,"prepare.run_wall_time_us":1239,"raft.queue_time_us":146593,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":8617,"spinlock_wait_cycles":7047040,"thread_start_us":257729,"threads_started":2,"wal-append.queue_time_us":603603}
W20260709 13:03:40.500042 12630 consensus_meta.cc:287] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.010s
I20260709 13:03:40.519874  8794 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:40.498282  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2356) took 3202 ms. Trace:
I20260709 13:03:40.526818  8747 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:40.405930  8756 rpcz_store.cc:276] 0709 13:03:36.929828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.929988 (+   160us) service_pool.cc:224] Handling call
0709 13:03:36.930422 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.572936 (+1642514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.350952 (+1778016us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540031cc54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:40.351378 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3507257216}
I20260709 13:03:40.509289  8675 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:40.528591  8826 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:40.528784 12819 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:40.486436 12831 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:40.489022 12783 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:03:40.530181  8828 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:03:40.530697  8828 pending_rounds.cc:85] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 7
I20260709 13:03:40.531003  8828 pending_rounds.cc:107] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:03:40.531288  8828 raft_consensus.cc:2889] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:40.530419 12783 consensus_queue.cc:1243] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:40.535854  8672 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:40.536357  8672 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:40.537642  8672 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:40.500492  8680 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:40.539211  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1911) took 1347 ms. Trace:
I20260709 13:03:40.539613  8680 rpcz_store.cc:276] 0709 13:03:39.192161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.194905 (+  2744us) service_pool.cc:224] Handling call
0709 13:03:40.499464 (+1304559us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:03:40.499826 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.538820 (+ 38994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.539144 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41959552}
I20260709 13:03:40.541458  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:40.501333  8673 rpcz_store.cc:276] 0709 13:03:39.438304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.213711 (+775407us) service_pool.cc:224] Handling call
0709 13:03:40.450293 (+236582us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:03:40.477980 (+ 27687us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2331520}
W20260709 13:03:40.523604  8821 rpcz_store.cc:269] 0709 13:03:37.152902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.153063 (+   161us) service_pool.cc:224] Handling call
0709 13:03:40.494918 (+3341855us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400267ad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:40.495191 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3151933184}
I20260709 13:03:40.530392  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:40.543175  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:40.543619  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:40.544041  8747 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:40.544596  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:40.545220  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:40.546032  8821 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:40.546339 12783 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:40.546850 12763 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:40.542239 12793 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:40.547474 12640 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:03:40.516939  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.572194  8821 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:40.557032  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:40.563098  9383 raft_consensus.cc:2410] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:40.558102  9352 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:40.562714  8829 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:40.584071 12871 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.584385 12630 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:40.585013 12630 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:40.584827 12871 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:40.585500 12630 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:40.585634  9389 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:40.587069 12768 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.584069  9352 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:40.546072  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:40.527405  8674 rpcz_store.cc:276] 0709 13:03:37.296268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.827673 (+531405us) service_pool.cc:224] Handling call
0709 13:03:37.827982 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.823523 (+1995541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.497748 (+674225us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b5400299d54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:40.498079 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4586801408}
I20260709 13:03:40.557005  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:40.556125  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:40.565232  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 63561.6us
I20260709 13:03:40.591470  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:40.591894  9380 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:40.593989  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.595008  8672 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:40.555665 12893 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 13:03:40.601619  8749 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:03:40.565804 12789 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:40.611744  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:40.549916 12793 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:40.559139 12740 raft_consensus.cc:3055] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:40.618011  8680 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:40.620714 12820 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:40.623145  8748 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:40.625716 12881 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:40.627015  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:40.631938  8754 raft_consensus.cc:2393] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 12: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:40.636152  8680 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:40.636500  8794 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:40.563191 12770 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:03:40.613557 12789 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.626217 12818 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:03:40.652278  8826 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:40.638116 12740 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:40.654327  8821 raft_consensus.cc:2368] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10.
I20260709 13:03:40.651376  8827 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:40.655270  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:40.653793  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1754) took 1002 ms. Trace:
I20260709 13:03:40.656436  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1744) took 1018 ms. Trace:
I20260709 13:03:40.656499  8826 rpcz_store.cc:276] 0709 13:03:39.651458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.485349 (+833891us) service_pool.cc:224] Handling call
0709 13:03:40.653724 (+168375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.657051  8821 rpcz_store.cc:276] 0709 13:03:39.637757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.602105 (+964348us) service_pool.cc:224] Handling call
0709 13:03:40.656352 (+ 54247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.587607 12768 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:40.658878  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:40.662541  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.662982 12572 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:40.663539  8680 pending_rounds.cc:85] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 9
I20260709 13:03:40.663750 12572 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:40.665135 12572 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:40.667440  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2405) took 1985 ms. Trace:
I20260709 13:03:40.667979  8751 rpcz_store.cc:276] 0709 13:03:38.682086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.704466 (+ 22380us) service_pool.cc:224] Handling call
0709 13:03:38.736793 (+ 32327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.811617 (+ 74824us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.811650 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:38.811690 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:38.811729 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.666922 (+1855193us) spinlock_profiling.cc:243] Waited 74.6 ms on lock 0x7b540038a9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:40.667005 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.667246 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171541376}
I20260709 13:03:40.669173  8796 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:40.663949  8680 pending_rounds.cc:107] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:03:40.671738  8680 raft_consensus.cc:2889] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:40.658170  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:40.663269  8821 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:03:40.670902  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:40.651099 12789 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:40.682201  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1923) took 1924 ms. Trace:
W20260709 13:03:40.684022  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 65128.5us
W20260709 13:03:40.681476  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 125487us
I20260709 13:03:40.679994  9388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 804) took 2621 ms. Trace:
I20260709 13:03:40.653098 12881 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:40.687567  8756 raft_consensus.cc:2410] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:03:40.688313  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:40.685356  9388 rpcz_store.cc:276] 0709 13:03:38.058575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.667533 (+608958us) service_pool.cc:224] Handling call
0709 13:03:38.667920 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.207369 (+539449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.207413 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:39.207476 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:39.207536 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.679473 (+1471937us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b540061c1d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:40.679547 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.679777 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1240018688}
I20260709 13:03:40.690259  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1905) took 2971 ms. Trace:
W20260709 13:03:40.692761  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.694085  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:40.682783  8821 rpcz_store.cc:276] 0709 13:03:38.751761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.662382 (+1910621us) service_pool.cc:224] Handling call
0709 13:03:40.662972 (+   590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:40.676248 (+ 13276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.676578 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.690820  8677 rpcz_store.cc:276] 0709 13:03:37.718540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.842713 (+124173us) service_pool.cc:224] Handling call
0709 13:03:37.872982 (+ 30269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.236320 (+1363338us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b54004e2120. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2470dc0 00007f32e24fb1c0 00007f32e24fc231 00007f32e24f8d8f 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c
0709 13:03:40.158144 (+921824us) spinlock_profiling.cc:243] Waited 1.25 ms on lock 0x7b54004e2120. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e246ea1d 00007f32e24551a4 00007f32e24d7447 00007f32e24effe7 00007f32e24f17de 00007f32e24fbb49 00007f32e24fc416 00007f32e24f8d8f 00007f32e24f883e 00007f32e74bd419
0709 13:03:40.651848 (+493704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:40.651901 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:40.659515 (+  7614us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:40.659608 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.671871 (+ 12263us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.690172 (+ 18301us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32971,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":9336,"spinlock_wait_cycles":1714716800}
I20260709 13:03:40.677404 12783 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:40.699520  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:40.686762 12640 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:40.705020  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1749) took 1055 ms. Trace:
I20260709 13:03:40.707507  8827 rpcz_store.cc:276] 0709 13:03:39.650089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.674656 (+1024567us) service_pool.cc:224] Handling call
0709 13:03:40.704928 (+ 30272us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:40.716939  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.709815  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:40.721982 12829 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:40.722577 12829 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.663232 12740 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:40.711627 12652 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:40.706878 12702 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:03:40.712013 12640 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:40.692353 12763 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:03:40.733492 12798 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:03:40.734184 12641 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.734766 12862 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:40.734855  8754 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:40.733526  8821 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:40.722990 12829 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:40.741142 12881 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:40.761649  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 53479.8us
I20260709 13:03:40.762056  8749 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:40.731739  8825 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:40.765076  8747 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:40.775925  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1924) took 2022 ms. Trace:
I20260709 13:03:40.781978  8821 rpcz_store.cc:276] 0709 13:03:38.752032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.710222 (+1958190us) service_pool.cc:224] Handling call
0709 13:03:40.733226 (+ 23004us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.774270 (+ 41044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.774569 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.775985 12740 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:40.776106 12652 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:40.778528  8827 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:40.778776 12641 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:40.795991  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1925) took 2043 ms. Trace:
I20260709 13:03:40.795250  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1747) took 1131 ms. Trace:
I20260709 13:03:40.796981  8825 rpcz_store.cc:276] 0709 13:03:39.649511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.204408 (+554897us) service_pool.cc:224] Handling call
0709 13:03:40.781295 (+576887us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.796877  8827 rpcz_store.cc:276] 0709 13:03:38.752315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.708248 (+1955933us) service_pool.cc:224] Handling call
0709 13:03:40.761061 (+ 52813us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:40.795602 (+ 34541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.795914 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.810360  8821 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:03:40.806974  8672 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:03:40.812350  8794 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:40.818946  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1926) took 2060 ms. Trace:
I20260709 13:03:40.819481  8821 rpcz_store.cc:276] 0709 13:03:38.752599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.798048 (+2045449us) service_pool.cc:224] Handling call
0709 13:03:40.810045 (+ 11997us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:40.813146 (+  3101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.813389 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.814167  8672 pending_rounds.cc:85] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 9
I20260709 13:03:40.791666 12687 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:40.846033  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 47429.7us
I20260709 13:03:40.789652 12756 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:40.791424 12640 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:40.829964  8827 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:40.831503  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:40.786537 12751 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:40.825011  8672 pending_rounds.cc:107] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:03:40.837251  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:40.857868  8672 raft_consensus.cc:2889] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:40.861699  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 174491us
I20260709 13:03:40.860549  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 718) took 1332 ms. Trace:
I20260709 13:03:40.862728  8827 rpcz_store.cc:276] 0709 13:03:39.522108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.824810 (+1302702us) service_pool.cc:224] Handling call
0709 13:03:40.828864 (+  4054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.853367 (+ 24503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.854657 (+  1290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.843591 12687 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:40.836083 12641 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:40.883003  8824 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:40.856696 12783 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:40.834494 12630 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:40.898733  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 399085us
I20260709 13:03:40.896786 12783 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.904171 12793 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:40.920817  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:40.903111 12687 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:40.928851  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.918663 12783 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:40.903990 12831 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:03:40.935833 12687 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:40.937995  8793 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:40.938753  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 101631us
I20260709 13:03:40.940076 12764 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:40.941388  8754 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:40.942394 12713 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:40.943660 12713 raft_consensus.cc:3055] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:40.944095 12713 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:40.945344 12713 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:40.947847 12630 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:40.934787  8827 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:40.950455  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 677) took 2844 ms. Trace:
I20260709 13:03:40.951540  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 714) took 2601 ms. Trace:
W20260709 13:03:40.956215  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.922114 12793 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:40.953949  8679 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:40.950670  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:40.959699  8681 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:40.954965  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1753) took 1303 ms. Trace:
I20260709 13:03:40.960542  8827 rpcz_store.cc:276] 0709 13:03:39.651199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.868065 (+1216866us) service_pool.cc:224] Handling call
0709 13:03:40.915075 (+ 47010us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.954472 (+ 39397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.954897 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.956475  8829 rpcz_store.cc:276] 0709 13:03:38.349796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.350044 (+   248us) service_pool.cc:224] Handling call
0709 13:03:38.350455 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.583599 (+2233144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.951175 (+367576us) spinlock_profiling.cc:243] Waited 2.17 s on lock 0x7b54003ac5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:40.951468 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4995127168}
I20260709 13:03:40.955614  8748 rpcz_store.cc:276] 0709 13:03:38.106179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.230590 (+124411us) service_pool.cc:224] Handling call
0709 13:03:38.231371 (+   781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.637199 (+2405828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.950114 (+312915us) spinlock_profiling.cc:243] Waited 2.39 s on lock 0x7b540038bdd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:40.950380 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5498568960}
I20260709 13:03:40.953301 12641 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.962182  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:40.963838 12739 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:40.964102 12641 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:40.962203 12793 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:40.976727  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.977375  8754 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:40.980641  8824 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:40.981496  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1769) took 1282 ms. Trace:
I20260709 13:03:40.981912  8824 rpcz_store.cc:276] 0709 13:03:39.699020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.831126 (+1132106us) service_pool.cc:224] Handling call
0709 13:03:40.981411 (+150285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.974946 12751 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:40.990778 12703 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:40.991709 12703 consensus_queue.cc:1243] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:03:40.992273  8681 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:40.992803  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1910) took 1801 ms. Trace:
I20260709 13:03:40.995730  9352 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:40.996361  8679 rpcz_store.cc:276] 0709 13:03:39.191842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.192352 (+   510us) service_pool.cc:224] Handling call
0709 13:03:40.952979 (+1760627us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:03:40.953333 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.959133 (+  5800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.992742 (+ 33609us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3579136}
I20260709 13:03:41.004156  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1755) took 1352 ms. Trace:
I20260709 13:03:41.004832  8827 rpcz_store.cc:276] 0709 13:03:39.651724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.961425 (+1309701us) service_pool.cc:224] Handling call
0709 13:03:40.991937 (+ 30512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.997223 (+  5286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:40.997254 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:41.003696 (+  6442us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:41.003768 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.003857 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.004061 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.006853  8754 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:40.994541 12862 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:40.990901 12751 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:41.011408 12402 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:41.008307 12718 raft_consensus.cc:3055] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:41.042093  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 104235us
I20260709 13:03:41.012017 12831 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:41.043979  8755 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:03:41.040436 12713 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:41.014665 12862 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:41.073284  8824 raft_consensus.cc:1217] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[7.4-9.5]   Dedup: 7.4->[9.5-9.5]
I20260709 13:03:41.073956  8824 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 13:03:41.041329 12718 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:41.078327 12718 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:41.078790  8755 raft_consensus.cc:1217] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Deduplicated request from leader. Original: 7.5->[9.6-9.6]   Dedup: 9.6->[]
I20260709 13:03:41.079171  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1922) took 2328 ms. Trace:
I20260709 13:03:41.079564  8755 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:41.079751 12718 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:41.079697  8826 rpcz_store.cc:276] 0709 13:03:38.751409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.674530 (+1923121us) service_pool.cc:224] Handling call
0709 13:03:40.718152 (+ 43622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:40.735943 (+ 17791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:40.736019 (+    76us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:40.736717 (+   698us) log.cc:844] Serialized 26 byte log entry
0709 13:03:40.960463 (+223746us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:40.998957 (+ 38494us) log.cc:844] Serialized 14 byte log entry
0709 13:03:41.000210 (+  1253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.000293 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:41.078911 (+ 78618us) raft_consensus.cc:1685] finished
0709 13:03:41.078932 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.079076 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37950,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":2523,"raft.queue_time_us":2808,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":7633,"thread_start_us":191617,"threads_started":1,"wal-append.queue_time_us":305826}
I20260709 13:03:41.083789  8826 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:03:41.084725  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1760) took 1396 ms. Trace:
I20260709 13:03:41.085100  8826 rpcz_store.cc:276] 0709 13:03:39.687874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.082040 (+1394166us) service_pool.cc:224] Handling call
0709 13:03:41.082597 (+   557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.084420 (+  1823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.084630 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.086815  8826 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:03:41.087273  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1929) took 1402 ms. Trace:
I20260709 13:03:41.087630  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1761) took 1399 ms. Trace:
I20260709 13:03:41.087780  8824 rpcz_store.cc:276] 0709 13:03:39.684868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.001390 (+1316522us) service_pool.cc:224] Handling call
0709 13:03:41.014955 (+ 13565us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:41.086961 (+ 72006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.087206 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.088229  8826 rpcz_store.cc:276] 0709 13:03:39.688168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.085942 (+1397774us) service_pool.cc:224] Handling call
0709 13:03:41.086402 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:41.087283 (+   881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.087537 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.089730  8824 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:03:41.099143  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.090394  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1767) took 1392 ms. Trace:
I20260709 13:03:41.100201  8824 rpcz_store.cc:276] 0709 13:03:39.698168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.088921 (+1390753us) service_pool.cc:224] Handling call
0709 13:03:41.089344 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.090129 (+   785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.090336 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:41.105002  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.070945 12831 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:41.109411 12831 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:41.128839 12829 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:41.068117 12641 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:41.136785  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:41.101742  8647 connection.cc:256] RPC call timeout handler was delayed by -1.4445793139999523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20390000000000003s for a response
I20260709 13:03:41.136839 12751 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:41.145640  8830 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 8: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:41.145721  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:41.146198 12789 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:41.146458  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1766) took 1457 ms. Trace:
I20260709 13:03:41.154543  8830 rpcz_store.cc:276] 0709 13:03:39.689475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.136277 (+1446802us) service_pool.cc:224] Handling call
0709 13:03:41.146368 (+ 10091us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.196257  8822 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 7: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:41.197700  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 717) took 1988 ms. Trace:
I20260709 13:03:41.198179  8825 rpcz_store.cc:276] 0709 13:03:39.209757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.807806 (+1598049us) service_pool.cc:224] Handling call
0709 13:03:40.854510 (+ 46704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.926762 (+ 72252us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:40.934400 (+  7638us) log.cc:844] Serialized 14 byte log entry
0709 13:03:41.197258 (+262858us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:41.197323 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:41.197383 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.197468 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.197627 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":361,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":67,"thread_start_us":240152,"threads_started":1}
I20260709 13:03:41.202452  8827 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:41.122153 12783 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:41.202994  8827 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:41.240494  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 98416.1us
I20260709 13:03:41.256230 12789 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:41.257241  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.259446  8750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2017) took 4270 ms (client timeout 1619 ms). Trace:
W20260709 13:03:41.260093  8750 rpcz_store.cc:269] 0709 13:03:36.945167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.980373 (+ 35206us) service_pool.cc:224] Handling call
0709 13:03:41.166712 (+4186339us) spinlock_profiling.cc:243] Waited 2.83 s on lock 0x7b54002fd5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:41.215292 (+ 48580us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6505096576}
I20260709 13:03:41.263083 12700 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:41.243861  8827 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:41.279326  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 111685us
W20260709 13:03:41.285728  8647 connection.cc:256] RPC call timeout handler was delayed by -1.2447793139999703s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22610000000000002s for a response
W20260709 13:03:41.287786  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.295964  8827 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:41.303349  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.303934  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.304936  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:41.308257 12718 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:41.308925 12718 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:41.309881  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 715) took 2677 ms. Trace:
I20260709 13:03:41.310477  8828 rpcz_store.cc:276] 0709 13:03:38.632696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.633233 (+   537us) service_pool.cc:224] Handling call
0709 13:03:40.529372 (+1896139us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.531866 (+  2494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.309622 (+777756us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b5400267ad0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:41.309802 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4295331712}
I20260709 13:03:41.315598  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:41.318360  8793 connection.cc:256] RPC call timeout handler was delayed by -1.2427352600000177s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
W20260709 13:03:41.324050  8793 connection.cc:256] RPC call timeout handler was delayed by -1.1409583580000344s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2629s for a response
I20260709 13:03:41.332309 12776 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:41.298491 12829 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:41.349957  8722 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:41.330837 12657 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:41.324859 12820 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:41.377288  8828 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:41.349128 12802 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.391980  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.393342  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:41.369724 12630 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.363093 12652 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:41.399904  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:41.405174  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.408792 12739 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:41.409636 12914 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:03:41.409925  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.410391 12702 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:41.411792  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.412161  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1752) took 1761 ms. Trace:
I20260709 13:03:41.413065  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:41.413753  8826 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:41.411165 12702 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:41.415958  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.412539  8821 rpcz_store.cc:276] 0709 13:03:39.650924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.833252 (+1182328us) service_pool.cc:224] Handling call
0709 13:03:40.833891 (+   639us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:40.848828 (+ 14937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:40.849140 (+   312us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:40.863467 (+ 14327us) log.cc:844] Serialized 26 byte log entry
0709 13:03:41.165713 (+302246us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:41.165755 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.165822 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:41.411901 (+246079us) raft_consensus.cc:1685] finished
0709 13:03:41.411925 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.412078 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13484,"raft.queue_time_us":19509,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":12759,"thread_start_us":254006,"threads_started":1,"wal-append.queue_time_us":337593}
I20260709 13:03:41.420244  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.424142  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:41.424422  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.424886  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1756) took 1772 ms. Trace:
I20260709 13:03:41.420921  8821 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:03:41.423835  8828 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:41.395485 12820 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.394354 12657 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:41.423168  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:41.437286  8750 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:41.434428  8830 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:41.421927  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:41.439011  8748 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:41.427465  8829 rpcz_store.cc:276] 0709 13:03:39.651963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.994156 (+1342193us) service_pool.cc:224] Handling call
0709 13:03:41.003456 (+  9300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:41.082847 (+ 79391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.082886 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:41.083542 (+   656us) log.cc:844] Serialized 26 byte log entry
0709 13:03:41.232248 (+148706us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:41.232309 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.232417 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:41.424690 (+192273us) raft_consensus.cc:1685] finished
0709 13:03:41.424711 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.424827 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":178,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":103,"thread_start_us":148186,"threads_started":1,"wal-append.queue_time_us":206613}
I20260709 13:03:41.403405 12783 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:41.423255 12915 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:03:41.447027  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.394045 12802 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:41.424628 12718 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:41.472636  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:41.473482  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.479713  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.416493 12702 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:41.483295  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 138313us
I20260709 13:03:41.482997  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.484340  8828 raft_consensus.cc:2410] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:03:41.487692  8826 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:41.488716  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.480408  8829 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:41.420258 12871 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:41.482177  8824 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:41.407126 12630 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:41.495954  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.495641 12657 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:41.497963 12919 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.235s	user 0.000s	sys 0.004s
W20260709 13:03:41.499344  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.499145  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.500016  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:41.500553  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.501276  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.506196  8821 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:41.512838  8824 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 8: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:41.516191  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:41.517855  8750 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:41.524268  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.524576  8826 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:41.522461  8722 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:41.528682  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.471995 12793 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:41.533531  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 228255us
I20260709 13:03:41.535238  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.535528  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:41.543561 12916 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:03:41.543322  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:41.544842  8828 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 12: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:41.547264  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:41.548229  8829 raft_consensus.cc:2376] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8. Re-sending same reply.
I20260709 13:03:41.549481  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.550427  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2426) took 1879 ms. Trace:
I20260709 13:03:41.550886  8752 rpcz_store.cc:276] 0709 13:03:39.670829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.806298 (+135469us) service_pool.cc:224] Handling call
0709 13:03:40.326462 (+520164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.413831 (+ 87369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.550206 (+1136375us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b540018d8d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:41.550363 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1194445184}
W20260709 13:03:41.556167 12923 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:03:41.560904  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.561847  8823 raft_consensus.cc:2376] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12. Re-sending same reply.
W20260709 13:03:41.562484  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1748) took 1912 ms (client timeout 2229 ms). Trace:
W20260709 13:03:41.562987  8823 rpcz_store.cc:269] 0709 13:03:39.649797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.560316 (+1910519us) service_pool.cc:224] Handling call
0709 13:03:41.562406 (+  2090us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.564313  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.566030  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.546164  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:41.569031 12928 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:03:41.569111  8828 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
W20260709 13:03:41.571022 12926 log.cc:927] Time spent T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
W20260709 13:03:41.572616 12917 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
W20260709 13:03:41.573479  9389 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 805) took 3466 ms (client timeout 1820 ms). Trace:
W20260709 13:03:41.574229  9389 rpcz_store.cc:269] 0709 13:03:38.107041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.365836 (+258795us) service_pool.cc:224] Handling call
0709 13:03:41.573025 (+3207189us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b54005b13d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:41.573334 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4591442688}
I20260709 13:03:41.577026  9380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 815) took 1197 ms. Trace:
I20260709 13:03:41.577391  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.577590  9380 rpcz_store.cc:276] 0709 13:03:40.379751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.379909 (+   158us) service_pool.cc:224] Handling call
0709 13:03:40.588463 (+208554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.592543 (+  4080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.576716 (+984173us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400520b50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:41.576881 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":478528768}
I20260709 13:03:41.579382  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.580021 12790 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:41.580394  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1763) took 1891 ms. Trace:
I20260709 13:03:41.581169 12790 raft_consensus.cc:3055] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:41.580909  8822 rpcz_store.cc:276] 0709 13:03:39.688695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.545663 (+856968us) service_pool.cc:224] Handling call
0709 13:03:41.580126 (+1034463us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b540002b7d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:41.580308 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376950656}
I20260709 13:03:41.581722 12790 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:41.583055 12790 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:41.584513 12790 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:41.586679  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:41.588778  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.591001  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:41.591761  8830 raft_consensus.cc:2376] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10. Re-sending same reply.
I20260709 13:03:41.593206  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:41.593652  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:41.580070  8829 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:03:41.595578  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:41.596242  8822 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:41.600399  8719 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:41.601574 12739 raft_consensus.cc:2764] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:03:41.577075  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.607239 12802 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:41.607451  8829 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:41.608894  8826 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:03:41.609717  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:41.611032  8749 raft_consensus.cc:2410] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:41.614936  8828 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:41.465169 12820 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:41.603588  8719 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:41.641438  8825 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:41.643319  8827 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:41.649897 12630 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:41.647673  8722 leader_election.cc:400] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10.
I20260709 13:03:41.647456  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1759) took 1960 ms. Trace:
I20260709 13:03:41.660255  8722 leader_election.cc:403] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:41.660663  8827 rpcz_store.cc:276] 0709 13:03:39.687514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.036279 (+1348765us) service_pool.cc:224] Handling call
0709 13:03:41.161059 (+124780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.647072 (+486013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.647373 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:41.661206 12700 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10.
I20260709 13:03:41.659376  8822 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:41.663340  8722 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:41.663280 12700 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:03:41.660348  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.658008  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 54541.3us
W20260709 13:03:41.679615  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.690408  8793 leader_election.cc:336] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.112s (SENT)
I20260709 13:03:41.646934 12819 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:03:41.690105 12862 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.603s	user 0.012s	sys 0.002s
I20260709 13:03:41.691123 12739 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:41.711112  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 175666us
I20260709 13:03:41.720714  8647 leader_election.cc:400] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
I20260709 13:03:41.721295  8647 leader_election.cc:403] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:03:41.698699 12703 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.723672  8645 leader_election.cc:400] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
W20260709 13:03:41.724125  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 77452.1us
I20260709 13:03:41.719964 12819 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.722258 12713 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.722833 12739 raft_consensus.cc:697] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:03:41.734510  8793 leader_election.cc:336] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.629s (SENT)
W20260709 13:03:41.737231  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 71861.1us
I20260709 13:03:41.737764  9382 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:41.727502 12657 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:03:41.741989 12768 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:41.743834  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:41.746448  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:41.713073 12862 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:41.743451 12790 consensus_queue.cc:1059] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
W20260709 13:03:41.757439  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 94034.6us
W20260709 13:03:41.760615  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 245639us
I20260709 13:03:41.733815 12937 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
I20260709 13:03:41.752661 12703 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:41.771587 12738 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:41.772971 12703 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:41.773494  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.773703 12934 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:03:41.774446 12934 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:41.775736 12934 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:41.778904 12793 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:41.779502 12657 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:41.779935 12790 consensus_queue.cc:1243] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:03:41.780148 12657 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:41.736253 12819 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:41.782979 12819 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:41.755997 12872 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:41.783200 12739 consensus_queue.cc:237] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:41.751302 12713 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:41.796828  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 43623.6us
I20260709 13:03:41.805984  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:41.809983  8830 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:41.810789  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:41.812587  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 78123.6us
W20260709 13:03:41.814287  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 70627.5us
I20260709 13:03:41.816529  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:41.817524  8825 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
W20260709 13:03:41.817871  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 56531.9us
I20260709 13:03:41.817548  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:41.825409  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.826820 12703 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.826793 12929 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:41.827562 12703 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:41.816731 12768 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:03:41.829896 12703 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:41.829958 12768 raft_consensus.cc:3055] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:41.830441 12768 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:41.822574 12944 raft_consensus.cc:3055] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:41.831030 12944 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:41.831701 12768 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:41.833132 12768 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:41.814845 12657 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:41.835121  9355 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:41.838177 12881 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:41.840025 12713 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:41.840636 12713 consensus_queue.cc:1059] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:41.838390 12656 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:41.841470 12713 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:41.841866 12713 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:41.842944  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:41.816505 12820 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:41.837765  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:41.849943 12656 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:41.850755 12656 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.851148 12656 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:41.853277 12831 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:41.856758 12819 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:41.859176  8645 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:41.865290 12819 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:41.871439 12713 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:41.880198 12820 consensus_queue.cc:1059] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:41.881209 12713 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:41.883196 12946 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:03:41.878115 12933 log.cc:927] Time spent T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.456s	user 0.002s	sys 0.000s
W20260709 13:03:41.887024  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 66058.1us
I20260709 13:03:41.888303  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2421) took 2730 ms. Trace:
I20260709 13:03:41.888710  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:41.879540  8748 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:41.886096 12943 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:41.907963  8794 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:41.890393  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:41.904502  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 48236.1us
I20260709 13:03:41.903025  8752 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:41.891644  8754 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:41.912652  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 1.26084e+06us
I20260709 13:03:41.894196  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:41.892056 12751 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:03:41.902428 12938 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 13:03:41.903754 12941 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
W20260709 13:03:41.941751  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.913045 12871 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:41.957437  8673 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:41.915345 12783 raft_consensus.cc:3055] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:41.956699  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:41.958650  8680 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:41.954078  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:41.890836 12703 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:41.927028 12736 raft_consensus.cc:2764] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:03:41.888722  8753 rpcz_store.cc:276] 0709 13:03:39.158004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.158167 (+   163us) service_pool.cc:224] Handling call
0709 13:03:39.158672 (+   505us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:39.187657 (+ 28985us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:40.635999 (+1448342us) spinlock_profiling.cc:243] Waited 5.11 ms on lock 0x7b18002b3c60. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c557 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:03:40.947724 (+311725us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b18002b3c60. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df409455 00007f32e5ef13c7 00007f32e25039a7 00007f32e25130bf 00007f32e2513038
0709 13:03:41.218233 (+270509us) log.cc:844] Serialized 14 byte log entry
0709 13:03:41.330571 (+112338us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:41.331379 (+   808us) log.cc:844] Serialized 26 byte log entry
0709 13:03:41.331523 (+   144us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:41.331564 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.331685 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:41.877412 (+545727us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:41.887990 (+ 10578us) raft_consensus.cc:1685] finished
0709 13:03:41.888012 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.888124 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28742,"prepare.queue_time_us":695340,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":40318,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":8048,"spinlock_wait_cycles":69765504,"thread_start_us":314979,"threads_started":2,"wal-append.queue_time_us":163804}
I20260709 13:03:41.977130  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:41.958840  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:41.930110 12937 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:41.973735  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:41.981253  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:41.930433 12738 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:41.975025  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:41.977301  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:41.983251  8679 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:41.986685  8677 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:41.987157  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:41.901547 12946 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:03:41.991560  8673 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:41.991555  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:41.890806 12700 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:03:41.990417  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:41.913304 12572 consensus_queue.cc:1059] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:03:41.994580  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:41.994781  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:41.996650  8748 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:41.998636  8674 raft_consensus.cc:2410] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:42.001654  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:41.932796 12872 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:03:41.998970  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:42.005470  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:41.990590 12702 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:42.010393  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:42.010524  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:41.933667 12831 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:03:42.009531  8672 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:03:41.971727 12783 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:42.005923  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:41.976946 12703 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:42.015121  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:42.016451  8681 raft_consensus.cc:2410] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:42.020457  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:41.981384 12937 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:42.026291  8675 raft_consensus.cc:2393] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:42.014520 12702 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.989188 12871 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:41.891040 12945 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 13:03:41.990760 12739 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.032363  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:42.033864  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:42.027601 12796 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
W20260709 13:03:42.036094  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 105635us
I20260709 13:03:42.037026  9355 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:42.036706 12783 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:42.039981 12819 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:42.040766 12776 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:42.041018  8754 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:42.041365 12776 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:42.031718 12702 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:42.031454 12937 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:42.046541  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:42.047470  8673 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.038296 12783 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.031606 12572 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:42.055009 12572 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:42.056129 12572 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:42.057739 12572 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:42.061368 12776 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.041054 12819 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:42.063257  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:42.063877 12703 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.064993 12703 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:42.065804 12862 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:42.067200 12739 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:42.067307 12871 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.069541 12739 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:42.065012  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:42.049755  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:42.071548  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:42.050451  8679 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:42.024817 12652 raft_consensus.cc:3055] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:42.073266  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:42.077054  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:42.094272  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 270202us
I20260709 13:03:42.096067  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:42.097033  8823 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:42.098712  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:42.076507 12776 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:42.103048  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:42.105683  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.107676  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:42.107884  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:42.111217  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:42.111552  8679 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:03:42.112434  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:42.113379  8748 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:42.111949  8672 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:42.108539  8827 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:42.082603 12768 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:42.138357  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 227905us
I20260709 13:03:42.118827  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:42.140973  8678 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:42.076519 12652 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:42.118417  8677 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:42.145459  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:42.071642 12656 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:42.150332  8681 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:42.196509  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:42.197538  8748 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:42.186473 12934 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.212030  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:42.223248  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 231761us
I20260709 13:03:42.224385  8677 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:03:42.221930  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 279125us
I20260709 13:03:42.227643  8794 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:42.236414  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.241151  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:42.243402  8821 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:03:42.314754 12949 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 13:03:42.318946  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 77946.9us
I20260709 13:03:42.320361  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:42.329245  8794 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:42.331033  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:42.330905 12763 consensus_queue.cc:1059] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:42.281370 12945 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:42.280424  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:42.339598 12951 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:03:42.360378 12820 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:42.334595 12763 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:42.341583 12954 raft_consensus.cc:3055] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:42.362429 12954 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:42.363852 12954 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:42.376783  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.254837 12801 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:03:42.224436 12934 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:42.383340 12934 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:42.384117 12934 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:42.384742 12934 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.387614 12656 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:42.356091  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 138916us
I20260709 13:03:42.405838  8677 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:42.406873  8674 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:42.407522  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:42.406405  8673 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:42.219298 12652 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.413331 12944 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.418367 12702 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:42.342631 12952 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:03:42.385106  8645 connection.cc:256] RPC call timeout handler was delayed by -1.0182662710000159s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
I20260709 13:03:42.437783  8678 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
W20260709 13:03:42.440656  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 120438us
I20260709 13:03:42.441677  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:42.393839 12657 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.362207 12763 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:42.490293  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.362188 12820 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.490435  8673 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.468691  8747 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:42.410876 12640 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:42.409538 12713 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.528s	user 0.011s	sys 0.004s
W20260709 13:03:42.495666  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 313437us
I20260709 13:03:42.497924  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:42.513697  8680 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.538s	user 0.014s	sys 0.000s
I20260709 13:03:42.407642 12768 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:42.515249  8680 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:42.504413  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:42.509859  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:42.517277  8827 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:42.517540  8821 raft_consensus.cc:2410] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:42.510753  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:42.499141  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:42.519089  8674 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:03:42.525961  8678 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:42.524250  9355 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:42.535073  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.493904 12739 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:42.541047  9355 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:42.500276 12872 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.547276  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:42.503741 12946 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:42.571941  9353 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.013s	sys 0.037s
W20260709 13:03:42.575237  9353 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.014s	sys 0.037s
I20260709 13:03:42.531752 12789 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:42.603003  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 182209us
I20260709 13:03:42.607331  9382 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
W20260709 13:03:42.586941  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 45974.5us
I20260709 13:03:42.591259  8673 raft_consensus.cc:2410] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:03:42.508446 12820 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.611307  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:42.615224  9385 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:42.532259 12652 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:42.593952  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 62223.5us
I20260709 13:03:42.538306 12776 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:03:42.593891  8796 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.007s	sys 0.027s
I20260709 13:03:42.467849 12937 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:42.501642 12657 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:42.602385 12713 raft_consensus.cc:3055] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:03:42.628620  8796 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.008s	sys 0.027s
I20260709 13:03:42.592325 12801 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:42.571942 12955 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:42.643216 12934 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:42.612699 12872 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:42.655143  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.658031  8754 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:42.668226  8675 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:03:42.669570  8679 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: [7.9-9.10]
I20260709 13:03:42.627416 12965 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:42.688287 12703 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.610s	user 0.010s	sys 0.003s
W20260709 13:03:42.676523  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.620296 12944 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:42.627826 12783 consensus_meta.cc:287] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.576s	user 0.011s	sys 0.002s
I20260709 13:03:42.694780  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:42.657580  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:42.563124  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0980631259999427s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2883s for a response
I20260709 13:03:42.718814  8719 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:42.721675  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:42.729979  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:42.631482 12656 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:03:42.674880 12713 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:42.690384 12630 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.722184  8645 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:42.736950  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:42.690338 12955 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.739920  8828 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:03:42.729185  8673 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:42.659762 12776 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.614837 12946 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:42.714459 12965 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.746089  8719 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:42.758095  8749 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:42.758177  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:42.761405  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:42.762902 12969 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:42.763219 12974 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:42.763581 12969 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:42.763971 12974 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.766207 12969 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:42.618335  8747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:03:42.767491  9388 raft_consensus.cc:2410] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 12 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:03:42.767839  8747 raft_consensus.cc:606] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:42.768537  8747 raft_consensus.cc:614] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:42.738354 12945 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:03:42.785701  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 87202us
I20260709 13:03:42.790449  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:42.793004  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:42.794025 12969 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:42.794587  9353 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:42.794356 12801 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.795104 12969 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.795699 12969 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.751034 12656 consensus_queue.cc:1243] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:03:42.671041 12763 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.718006 12739 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:03:42.804831  8676 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:42.806468  8681 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:42.753605 12955 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:42.737866 12953 consensus_queue.cc:1059] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:03:42.814736  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:42.757680 12965 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.830545  8681 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:03:42.754153 12703 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:03:42.733430 12768 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:42.766851 12630 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:42.806962 12702 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:42.806011 12929 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:42.754551 12971 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:03:42.814425 12756 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:42.815404 12970 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:42.881098  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 66461.3us
W20260709 13:03:42.882543  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.884554 12937 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:42.884647 12929 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:42.885622  8645 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:42.885840 12947 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:42.886703 12947 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:42.887068 12947 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.887605 12947 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:42.887331 12970 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:42.809353 12945 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.884912 12872 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:42.884874  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:42.889786 12970 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:42.833727 12713 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.890836 12972 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:42.890933  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:42.891361 12972 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.891749 12972 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:42.887176 12703 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:03:42.893697 12703 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:42.894100 12703 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.894490 12703 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:42.898129 12657 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:42.890381 12945 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:42.899050 12657 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:42.886969  8645 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:42.890368 12970 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:42.883625  9388 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
W20260709 13:03:42.923403  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 198880us
I20260709 13:03:42.906279 12934 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:42.909070 12702 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:42.944008  8822 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:42.944880  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:42.947635  8823 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:42.917013 12703 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:42.973461  8826 pending_rounds.cc:85] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 10
I20260709 13:03:42.974103  8826 pending_rounds.cc:107] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:03:42.974503  8826 raft_consensus.cc:2889] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:42.978267  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:42.979650  9381 raft_consensus.cc:2410] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:03:42.933671 12970 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:42.933630 12971 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.937950 12820 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:42.937738 12756 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:42.945935 12702 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:42.992578  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.975775 12979 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:43.002269  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:43.011960  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 317357us
W20260709 13:03:43.014765  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 254341us
I20260709 13:03:42.990305 12962 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:03:43.040777  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:43.042099  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:43.051165  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 216149us
I20260709 13:03:43.060992  8829 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:43.023900 12776 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:43.015547 12739 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:43.054162  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.994048 12971 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:43.058575  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 157882us
I20260709 13:03:43.063340  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:43.006172 12756 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.049202 12986 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:43.063611  8719 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:43.069878 12977 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:43.054811 12703 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:43.071892 12985 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.072054 12971 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:43.072589  8822 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:43.072736  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:43.073704  8823 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:43.020023 12979 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.100852  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:43.101168 12971 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:43.102288  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:43.102660 12979 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:43.056665 12983 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:03:43.079152 12630 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:43.074944  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.103478 12983 consensus_queue.cc:1243] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:43.079370 12739 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.077788  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:43.100874  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:43.054467 12972 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:43.080497  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.061362 12944 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
W20260709 13:03:43.076048  8722 leader_election.cc:336] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.528s (SENT)
I20260709 13:03:43.116206 12979 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:43.130417 12954 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:43.152669 12944 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:43.153227 12944 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:43.111836 12985 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.156267 12972 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:43.132321  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:43.158386  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:43.158855 12945 raft_consensus.cc:3055] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:43.098063 12977 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.159348 12945 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:43.160492 12945 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.159070  8827 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:43.161628 12977 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:43.121084  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.056399 12702 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.154192 12947 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.161909 12945 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:43.169104  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 68322.5us
I20260709 13:03:43.163625  8674 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:43.165393  8675 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:03:43.150535  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1926) took 1125 ms. Trace:
I20260709 13:03:43.175805  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:43.174284  8679 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:43.091079 12986 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.177109 12979 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.204392 12979 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:43.204653 12967 log.cc:927] Time spent T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.331s	user 0.002s	sys 0.000s
W20260709 13:03:43.207233  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 131224us
W20260709 13:03:43.208024  8722 leader_election.cc:336] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.609s (SENT)
I20260709 13:03:43.208449  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:43.209465 12989 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:03:43.210300  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:43.211817  8722 leader_election.cc:336] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.634s (SENT)
I20260709 13:03:43.203696 12946 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:43.196658  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:43.252806  8821 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:43.221441  8829 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:43.166980  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 739) took 1066 ms. Trace:
I20260709 13:03:43.255029  8828 rpcz_store.cc:276] 0709 13:03:42.100669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.100847 (+   178us) service_pool.cc:224] Handling call
0709 13:03:42.115465 (+ 14618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.747843 (+632378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.166681 (+418838us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54003dead4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:43.166906 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435273600}
I20260709 13:03:43.158542 12985 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:43.179816  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:43.259385  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 197588us
W20260709 13:03:43.225803  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 128733us
I20260709 13:03:43.231642  8827 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:43.220868  8676 rpcz_store.cc:276] 0709 13:03:42.025314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.025617 (+   303us) service_pool.cc:224] Handling call
0709 13:03:43.150265 (+1124648us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b5400176454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:43.150413 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1170234624}
I20260709 13:03:43.251281  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:43.258751  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46976.3us
I20260709 13:03:43.259217  8752 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:43.170768 12984 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:43.267891  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.263125  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:43.265244  8755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:03:43.274873  8755 raft_consensus.cc:606] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:43.275424  8755 raft_consensus.cc:614] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:43.265825  8826 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:43.169653 12802 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:43.268640  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:43.169409 12971 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:43.221155 12572 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.163s	user 0.019s	sys 0.000s
I20260709 13:03:43.289170  8645 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:43.296164  8825 raft_consensus.cc:2410] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 13:03:43.300422  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:43.303722  8722 leader_election.cc:336] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.530s (SENT)
I20260709 13:03:43.312628  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.273988 12790 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:43.310015  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 49075us
I20260709 13:03:43.286194 12802 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:43.308197  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 91007.5us
I20260709 13:03:43.284199 12977 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:43.272332 12989 raft_consensus.cc:2764] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:03:43.260808 12630 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:03:43.350936  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:43.314968  8722 leader_election.cc:336] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.936s (SENT)
I20260709 13:03:43.366350  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.368841  8673 raft_consensus.cc:2368] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11.
I20260709 13:03:43.154241 12944 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.372838  8673 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:43.309408 12793 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:43.378391  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:43.379982  8673 raft_consensus.cc:2376] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9. Re-sending same reply.
I20260709 13:03:43.384341  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.386008  8821 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:03:43.386101  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.388271  8673 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:43.362285 12802 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:43.306218 12972 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 13:03:43.364529 12862 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.366166 12739 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:43.405989  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:43.409209  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 59483.5us
I20260709 13:03:43.372821 12977 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:43.414995  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:43.315423 12960 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:03:43.447634  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.309288 12971 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.389781 12983 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:43.309273 12992 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.097s
W20260709 13:03:43.444067  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 175505us
I20260709 13:03:43.445864  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.446074  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:43.447233  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:43.442382  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 127454us
W20260709 13:03:43.512617  8722 leader_election.cc:336] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.021s (SENT)
I20260709 13:03:43.513250  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:43.495378  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.515352  8673 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:43.284780 12984 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:43.499070  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 183729us
I20260709 13:03:43.526139  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.526522  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:43.441866 12670 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:43.419409 12862 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.541472  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1936) took 1362 ms. Trace:
I20260709 13:03:43.319669 12703 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 13:03:43.411803 12989 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.542249  8677 rpcz_store.cc:276] 0709 13:03:42.179479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.498586 (+319107us) service_pool.cc:224] Handling call
0709 13:03:42.498903 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.508986 (+ 10083us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:42.544555 (+ 35569us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.216204 (+671649us) spinlock_profiling.cc:243] Waited 4.69 ms on lock 0x7b18001f4aa0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:03:43.541076 (+324872us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:43.541143 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:43.541202 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.541274 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.541408 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6395,"prepare.queue_time_us":35391,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":17223,"raft.queue_time_us":51774,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":43433,"spinlock_wait_cycles":10787200,"thread_start_us":318110,"threads_started":2}
I20260709 13:03:43.413409 12944 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:43.530248  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.539440  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.551337  8673 raft_consensus.cc:2410] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:43.541227  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.494553 12954 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:43.544097  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.544025  9352 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:43.548895  8647 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:43.556833  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.557866  8677 raft_consensus.cc:2410] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:03:43.524989 12796 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.532377 12790 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:43.571606  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.448113 12962 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 13:03:43.572417  8673 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:43.524642 12974 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:43.577332  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:43.588129 12652 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.960s	user 0.010s	sys 0.005s
I20260709 13:03:43.564347 12989 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.589304 12947 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:43.574783 12937 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:43.590824 12703 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:43.591361 12984 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:43.593366  8823 raft_consensus.cc:2410] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:03:43.576409 12796 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.591490 12703 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.598896 12986 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:43.600912 12703 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:43.574736 13012 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:43.603974 13014 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:03:43.605062  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 92490.1us
I20260709 13:03:43.605826 13014 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:43.582407 12756 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:43.605103 12796 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:43.607662  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.607999  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.606451 13014 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.610087 13014 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:43.615885 12703 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.580264  8677 raft_consensus.cc:2376] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9. Re-sending same reply.
I20260709 13:03:43.619745  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.621055  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.617481 12703 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:43.624445  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 85894.5us
I20260709 13:03:43.625602  8647 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:43.626863 12970 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:43.627964 12703 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:43.634517  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.635478  8673 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:43.644654  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.651345  8647 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:43.652736 12862 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:43.646313 13012 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:43.654313 13012 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.656273  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:43.657380  8672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2399) took 1308 ms (client timeout 1615 ms). Trace:
W20260709 13:03:43.658990  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:43.660586  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 245681us
I20260709 13:03:43.660871  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.662078  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:43.662218  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.662870  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.663305 12796 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:43.664312  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.664695 12989 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:43.664899  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:43.657842  8672 rpcz_store.cc:269] 0709 13:03:42.348483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.363988 (+ 15505us) service_pool.cc:224] Handling call
0709 13:03:43.656067 (+1292079us) spinlock_profiling.cc:243] Waited 38.4 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:03:43.657280 (+  1213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88433536}
I20260709 13:03:43.595237 12983 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.665666  8676 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:43.665768 12983 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.666790  8672 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:43.563347 12954 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.668860 12954 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:43.669399  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.674788 12986 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:43.652804  8647 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:43.582669 12994 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:43.700204  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 47480.9us
I20260709 13:03:43.704027  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.703977  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:43.682355  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:43.701249  8829 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:43.707976  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:43.720744  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:43.728719  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.711272 12703 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:43.738008  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:43.631436 12971 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:43.658999 12820 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:43.808961 12702 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:43.704381 12969 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:43.706502 12700 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.810866 12700 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.811986 12657 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:43.812613 12657 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.813238 12700 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:43.815819  8752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2036) took 1885 ms (client timeout 2204 ms). Trace:
W20260709 13:03:43.816255  8752 rpcz_store.cc:269] 0709 13:03:41.930119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.936176 (+  6057us) service_pool.cc:224] Handling call
0709 13:03:43.815622 (+1879446us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540008ff54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:43.815750 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2447982464}
I20260709 13:03:43.817627  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:43.821806 13008 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
I20260709 13:03:43.823305  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1938) took 1643 ms. Trace:
I20260709 13:03:43.722201  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:43.824400  8753 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:43.823772  8678 rpcz_store.cc:276] 0709 13:03:42.180049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.528243 (+348194us) service_pool.cc:224] Handling call
0709 13:03:42.528831 (+   588us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.529189 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.529222 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.156704 (+627482us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b2c000014c8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e0df2a11 00007f32e0e02372 00007f32e2534a01 00007f32e24f943a 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2
0709 13:03:43.157250 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.376738 (+219488us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:43.376802 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.376880 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.822895 (+446015us) raft_consensus.cc:1685] finished
0709 13:03:43.822930 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.823225 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23109504,"thread_start_us":217105,"threads_started":1,"wal-append.queue_time_us":385221}
I20260709 13:03:43.826998 12657 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:43.738008  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:43.768050  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 47440.5us
I20260709 13:03:43.830674  8751 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:43.831107  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:43.831995  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:43.833509  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:43.770828 13015 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:43.834919 13015 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.835561 13015 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:43.836869  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:43.838863  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:43.740635 12756 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:43.840133 12756 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:43.840534 12756 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:43.841053  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:43.841683  8829 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:43.844081  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.843924  8678 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:43.844893  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.846781  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:43.847587  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.847353 12802 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.848086  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:43.839435 12957 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:43.849637 12802 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:43.774363 13014 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.850394 12984 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.850929 12984 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.852047 13014 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:43.777297 12968 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.909s	user 0.000s	sys 0.004s
I20260709 13:03:43.852622  8677 raft_consensus.cc:2410] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:43.780385  8748 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:43.853082 12763 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:43.854120 12763 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:43.854465 12763 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:43.854652  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:43.855284  8748 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:03:43.846726  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.855491 12763 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.856500  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:43.856941 12763 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:43.770102 12974 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.790977  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:43.857985  8750 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:43.857692 12974 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.859035  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:43.860183 12974 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:43.860363  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:43.804246 12983 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:43.807626  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:43.857447  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:43.861649  8756 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:43.862337  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:43.862957  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:43.863284 12934 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:43.863849 12934 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:43.739111 12947 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:43.864620 12947 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.864910 12934 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.711056 13016 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:43.769351 12979 consensus_meta.cc:287] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.564s	user 0.007s	sys 0.007s
I20260709 13:03:43.868026 12979 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.869158 12937 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:43.869730 12937 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:43.680452 12977 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.849603 12957 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:43.787448 13003 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.237s	user 0.002s	sys 0.000s
I20260709 13:03:43.857758  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 738) took 1757 ms. Trace:
I20260709 13:03:43.883261  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:43.860486 13015 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.771157 12970 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:43.839537  8752 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:43.871966  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:43.886476  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:43.887327  8674 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:43.881978  8824 rpcz_store.cc:276] 0709 13:03:42.100281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.100440 (+   159us) service_pool.cc:224] Handling call
0709 13:03:42.119418 (+ 18978us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:42.152607 (+ 33189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.152656 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:42.201509 (+ 48853us) log.cc:844] Serialized 52 byte log entry
0709 13:03:42.501330 (+299821us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:42.841232 (+339902us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.882120 (+ 40888us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.888578 (+  6458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.888666 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.776514 (+887848us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:43.857520 (+ 81006us) raft_consensus.cc:1685] finished
0709 13:03:43.857546 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.857693 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":108933,"prepare.queue_time_us":417009,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":54414,"raft.run_cpu_time_us":419,"raft.run_wall_time_us":16738,"spinlock_wait_cycles":8381184,"thread_start_us":530271,"threads_started":2,"wal-append.queue_time_us":582386}
I20260709 13:03:43.861104  8750 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:43.796123  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:43.863843  8756 raft_consensus.cc:2410] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:03:43.870725  8748 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:43.893285  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.894021  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:43.844933 12955 consensus_meta.cc:287] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.015s	user 0.009s	sys 0.002s
I20260709 13:03:43.877568  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:43.870434 12979 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:43.896772  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:43.877568  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:43.869192 12703 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:43.906351  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:43.907258  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 362919us
I20260709 13:03:43.907773  8719 leader_election.cc:400] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11.
I20260709 13:03:43.912503  8719 leader_election.cc:403] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:43.898407 12965 consensus_meta.cc:287] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.057s	user 0.014s	sys 0.000s
W20260709 13:03:43.914966  8827 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.006s	sys 0.045s
W20260709 13:03:43.919883  8827 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.006s	sys 0.046s
I20260709 13:03:43.922058  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 736) took 1837 ms. Trace:
W20260709 13:03:43.925170  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.936380  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:43.937428  8829 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:43.922650  8830 rpcz_store.cc:276] 0709 13:03:42.084947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.085133 (+   186us) service_pool.cc:224] Handling call
0709 13:03:42.107027 (+ 21894us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.115700 (+  8673us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.115740 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:42.144177 (+ 28437us) log.cc:844] Serialized 26 byte log entry
0709 13:03:42.498710 (+354533us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:43.203769 (+705059us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:43.911250 (+707481us) spinlock_profiling.cc:243] Waited 6.14 ms on lock 0x7b1800402880. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e20a14bc 00007f32df3ef48b 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df409455 00007f32e5ef13c7 00007f32e25039a7 00007f32e25130bf
0709 13:03:43.911451 (+   201us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.921579 (+ 10128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.921713 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.921772 (+    59us) raft_consensus.cc:1685] finished
0709 13:03:43.921790 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.921972 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32088,"prepare.queue_time_us":396165,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":32040,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":9791,"spinlock_wait_cycles":25023616,"thread_start_us":542556,"threads_started":2,"wal-append.queue_time_us":587190}
W20260709 13:03:43.945024  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 401052us
I20260709 13:03:43.945186  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:43.909991 12955 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.910470 12984 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:43.955641  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:43.957221  8672 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:43.961129  8674 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:43.910976 12703 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:43.910849 12670 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:43.928731 12983 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:43.870745 12937 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:43.927870 12947 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:43.961956  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 54356.7us
I20260709 13:03:43.991092  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:43.949353 12992 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11.
I20260709 13:03:43.991912  8830 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:43.885574 12957 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:43.986096  8753 raft_consensus.cc:2410] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:03:43.991861  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:43.994635  8750 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:03:43.995688  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.011945  8829 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:43.941475 12965 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:43.994094 12945 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.735s	user 0.012s	sys 0.000s
I20260709 13:03:44.014180  8719 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:43.961691 12657 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:03:44.026516  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.027462  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:43.972360 12703 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:44.025867  8796 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:43.994268 12989 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.972616 12961 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:03:44.054291  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:43.971850 12991 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:03:44.046676  8719 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:44.065987  8576 catalog_manager.cc:5697] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 7 to 9, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 9 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:44.067909  9355 leader_election.cc:336] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.568s (SENT)
I20260709 13:03:44.030014 12947 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:44.061245 12749 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:44.081866  9355 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:44.061924 13003 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:03:44.059300 12945 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:44.075526  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:44.090690  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 44173.6us
I20260709 13:03:44.092562  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:44.093031  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:44.104570  8752 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:44.094331  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.084628  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.065279 12991 consensus_queue.cc:1243] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:44.105000  9382 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:44.107064  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:44.107682  8753 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:44.109032  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.109932  8750 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:44.110209  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:44.114739  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:44.100603 12972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.012s	sys 0.016s
I20260709 13:03:44.058676 12989 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.126961  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.097287 12749 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:44.131075  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 63216.5us
I20260709 13:03:44.100495 12943 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:44.120281  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:44.119375  8722 leader_election.cc:336] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.584s (SENT)
I20260709 13:03:44.104168 12945 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.103032 12947 raft_consensus.cc:697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:44.119479  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.140738  8752 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:03:44.140864  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1939) took 1960 ms. Trace:
I20260709 13:03:44.134716  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:44.141878  8750 raft_consensus.cc:2376] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8. Re-sending same reply.
W20260709 13:03:44.129772 12972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.013s	sys 0.016s
I20260709 13:03:44.142140  8681 raft_consensus.cc:1217] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Deduplicated request from leader. Original: 6.4->[7.5-9.6]   Dedup: 9.6->[]
W20260709 13:03:44.144775  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 170513us
I20260709 13:03:44.141256  8680 rpcz_store.cc:276] 0709 13:03:42.180328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.531635 (+351307us) service_pool.cc:224] Handling call
0709 13:03:42.576307 (+ 44672us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:42.815849 (+239542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.815909 (+    60us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:42.870993 (+ 55084us) log.cc:844] Serialized 78 byte log entry
0709 13:03:43.309715 (+438722us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:43.356724 (+ 47009us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.357023 (+   299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.357144 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.786595 (+429451us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:44.131525 (+344930us) raft_consensus.cc:1685] finished
0709 13:03:44.140563 (+  9038us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.140777 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":55272,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":173018,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":42185,"spinlock_wait_cycles":23208320,"thread_start_us":613686,"threads_started":2,"wal-append.queue_time_us":484500}
I20260709 13:03:44.148376  8681 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 13:03:44.157272  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.135747 12749 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:44.159222  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.160313  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:44.166059  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.133312 12943 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:44.144613 12945 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:44.170418  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 56337.1us
I20260709 13:03:44.150595 12972 consensus_queue.cc:1243] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:03:44.087806 12670 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.186271  8719 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:44.191291  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.191996  8750 raft_consensus.cc:2376] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11. Re-sending same reply.
I20260709 13:03:44.193660  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:44.194487  8752 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:44.147364 13015 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:44.196338  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.185344 12943 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:44.202910 12947 consensus_queue.cc:237] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.214603 12979 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:44.217103 12972 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:44.205570  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.213061 12970 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:44.213937  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 336453us
W20260709 13:03:44.212347  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 340476us
W20260709 13:03:44.219480  8825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 735) took 2143 ms (client timeout 1568 ms). Trace:
W20260709 13:03:44.244952  8825 rpcz_store.cc:269] 0709 13:03:42.075791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.075992 (+   201us) service_pool.cc:224] Handling call
0709 13:03:44.219177 (+2143185us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002684d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:44.219383 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2751268096}
I20260709 13:03:44.245517  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:44.215587  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.244446  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:44.215993 12702 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:44.217398  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.184345 12592 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.192s
I20260709 13:03:44.199921 12955 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.254988  8748 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:03:44.251634  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:44.256701 12972 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.251344  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:44.248457 13028 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:03:44.258318  8678 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:44.259591 12972 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:44.249130 13025 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:03:44.264818  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:44.262566  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:44.268296  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2393) took 1922 ms. Trace:
I20260709 13:03:44.297564  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 1960) took 1228 ms. Trace:
I20260709 13:03:44.313352  8823 rpcz_store.cc:276] 0709 13:03:43.069232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.102020 (+ 32788us) service_pool.cc:224] Handling call
0709 13:03:44.297290 (+1195270us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b540032f354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:44.297483 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1126927104}
W20260709 13:03:44.299863  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 113687us
I20260709 13:03:44.315572  8719 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:44.302749 12776 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:44.317716 12776 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.320998 12776 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.302899 12992 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:44.323764 12992 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:44.325110 12992 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.303467  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:44.303982 12702 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.307957  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:44.331849  9384 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:44.270725 13015 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:44.263778  8678 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:44.334764  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:44.335621  8678 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:44.336807  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:44.337551  8678 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:44.339308  8674 raft_consensus.cc:2393] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 14: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:44.340644  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:44.341470  8674 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:44.287402  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 746) took 1010 ms. Trace:
I20260709 13:03:44.311721  8681 rpcz_store.cc:276] 0709 13:03:42.346317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.862879 (+516562us) service_pool.cc:224] Handling call
0709 13:03:44.141984 (+1279105us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:44.148764 (+  6780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.268061 (+119297us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54002409d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:44.268209 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2899046784}
I20260709 13:03:44.347887  8672 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:44.215624 12989 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.344255  8826 rpcz_store.cc:276] 0709 13:03:43.276852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.298092 (+ 21240us) service_pool.cc:224] Handling call
0709 13:03:43.314318 (+ 16226us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.557560 (+243242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.557612 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.576467 (+ 18855us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.904995 (+328528us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:43.905062 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.286903 (+381841us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b5400267fd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:44.286958 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:44.287009 (+    51us) raft_consensus.cc:1685] finished
0709 13:03:44.287023 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.287318 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":691,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":12383,"spinlock_wait_cycles":558028544,"thread_start_us":300850,"threads_started":1,"wal-append.queue_time_us":533576}
W20260709 13:03:44.350564  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 54698.4us
W20260709 13:03:44.287360 13029 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:03:44.243116 12957 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:44.363344  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:44.343673 12944 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.789s	user 0.012s	sys 0.000s
I20260709 13:03:44.301223 12953 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:44.273368 12713 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:44.247972 13026 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:44.296237 12976 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:44.380906  8681 pending_rounds.cc:85] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 6
I20260709 13:03:44.270875 12969 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.401885  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:44.298858 12700 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:44.283932 12793 raft_consensus.cc:3055] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:03:44.246667  8756 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:44.338562  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:44.347506 12965 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.405370  8752 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:44.401703 12749 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.403277 12700 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:44.396431  8751 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:44.398345  8681 pending_rounds.cc:107] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:03:44.422461  8681 raft_consensus.cc:2889] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:44.404945  8678 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:44.406810  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:44.409543 13015 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:44.419030 12976 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.430503 13015 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.431732 12749 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:44.432617  9388 raft_consensus.cc:2368] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11.
I20260709 13:03:44.432921 13015 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:44.385772 12702 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.401726 12944 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:44.441764 12945 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.490053  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.428529 12700 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:44.442152  8675 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:44.505961  9353 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:44.411497 12819 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.456269  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.420709 12989 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:03:44.478854  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 72139.4us
I20260709 13:03:44.464102  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:44.451537 13035 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:03:44.461180  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 435455us
I20260709 13:03:44.578451  8676 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:44.578876  8796 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:44.455771  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:44.458448 12790 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.593369 12790 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.595775 12790 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:44.404163 12793 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:44.516142  8752 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:03:44.597108 12793 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.448191  8748 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7. Ops: []
I20260709 13:03:44.598501 12793 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:44.445909  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 747) took 1151 ms. Trace:
I20260709 13:03:44.600907  8827 rpcz_store.cc:276] 0709 13:03:43.294194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.300489 (+  6295us) service_pool.cc:224] Handling call
0709 13:03:43.300943 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.319011 (+ 18068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.319042 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.365749 (+ 46707us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.920537 (+554788us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:43.962164 (+ 41627us) log.cc:844] Serialized 14 byte log entry
0709 13:03:44.445186 (+483022us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b18003a5280. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:03:44.445421 (+   235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.445510 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:44.445559 (+    49us) raft_consensus.cc:1685] finished
0709 13:03:44.445656 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.445822 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9042,"prepare.queue_time_us":44941,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":492901,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":5394,"spinlock_wait_cycles":5020928,"thread_start_us":510355,"threads_started":1,"wal-append.queue_time_us":678739}
I20260709 13:03:44.603637  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.603783  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:44.460433  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.604615  8680 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:44.604736  8822 raft_consensus.cc:2376] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12. Re-sending same reply.
I20260709 13:03:44.605587  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:44.605885  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.606324  8680 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:44.459755  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:44.608111  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:44.609133  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.517892  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 217196us
I20260709 13:03:44.458997  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:44.523721  8825 raft_consensus.cc:2410] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:03:44.406572 12969 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:44.517428  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 201958us
I20260709 13:03:44.477524 12796 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:44.477581 12983 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:44.529206 13033 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:03:44.491204 12937 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:44.529524 12963 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.621649 12963 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.560869  8753 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 9: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:03:44.622716 13029 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 13:03:44.605237 12756 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.764s	user 0.013s	sys 0.001s
I20260709 13:03:44.623876 12756 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.624248 12983 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.626160 12756 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:44.610435 13015 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.605440 13043 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.580428 12989 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.483369 12703 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.577658 13035 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:44.609885 13024 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:44.626575 12983 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:44.630822  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:44.638980  8673 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:44.640270  8673 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:03:44.640607  8673 pending_rounds.cc:85] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 6
I20260709 13:03:44.640827  8673 pending_rounds.cc:107] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:03:44.641076  8673 raft_consensus.cc:2889] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:44.631418 13015 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.617383 12871 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:44.645244 12871 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.645610 13015 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.632632 12945 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:44.575119 12702 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.646867  8756 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
W20260709 13:03:44.632834 13037 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.111s	user 0.000s	sys 0.006s
I20260709 13:03:44.557089 12572 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.578451  8755 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
W20260709 13:03:44.460821  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.635203  8674 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:44.635210  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.651132  8825 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:44.584066  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:44.637099  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1956) took 1056 ms. Trace:
I20260709 13:03:44.581941  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:44.597491  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:44.652421  8677 rpcz_store.cc:276] 0709 13:03:43.580642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.644211 (+ 63569us) service_pool.cc:224] Handling call
0709 13:03:44.636899 (+992688us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54003a2ad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:44.637039 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471702912}
I20260709 13:03:44.653105  9353 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:44.653151  8752 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:03:44.637439  8823 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:44.443195  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 744) took 1370 ms. Trace:
W20260709 13:03:44.611737  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.612511  8719 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:44.614701  8722 leader_election.cc:400] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11.
I20260709 13:03:44.660069  8722 leader_election.cc:403] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:44.658813  8828 rpcz_store.cc:276] 0709 13:03:43.073265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.264569 (+191304us) service_pool.cc:224] Handling call
0709 13:03:43.294794 (+ 30225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:43.410792 (+115998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.410827 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:43.410882 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:43.410929 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.442902 (+1031973us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b540032e454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:44.442959 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.443127 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266165120}
I20260709 13:03:44.665378  8829 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:44.662540 12977 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.565181 12819 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.685896  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:44.687354  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 74954.4us
I20260709 13:03:44.616478  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:44.617442  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.688446  8719 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:44.617955  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:44.689481  8822 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:44.619513  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:44.619782 13032 log.cc:927] Time spent T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 13:03:44.654165  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:44.707846  8677 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:44.688100  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:44.710690  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 700) took 1008 ms. Trace:
I20260709 13:03:44.691565  8680 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:44.693220  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.694013 12945 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:44.694541  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:44.718626  8675 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:44.634161 12989 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:44.696393  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 81897.7us
I20260709 13:03:44.634886 12703 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.645665 12871 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:44.636641 12969 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.698642  8672 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:03:44.729950  8672 pending_rounds.cc:85] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 3
I20260709 13:03:44.730265  8672 pending_rounds.cc:107] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:03:44.730501  8672 raft_consensus.cc:2889] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:44.659876 13024 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:44.698570  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.701875 12819 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:44.711176  8754 rpcz_store.cc:276] 0709 13:03:43.704889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.705066 (+   177us) service_pool.cc:224] Handling call
0709 13:03:43.745062 (+ 39996us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.745551 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.745591 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.806792 (+ 61201us) log.cc:844] Serialized 26 byte log entry
0709 13:03:44.118938 (+312146us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:44.147406 (+ 28468us) log.cc:844] Serialized 14 byte log entry
0709 13:03:44.298207 (+150801us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b18002e6780. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:03:44.298473 (+   266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.298577 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:44.710364 (+411787us) raft_consensus.cc:1685] finished
0709 13:03:44.710476 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.710628 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4608,"prepare.queue_time_us":31731,"prepare.run_cpu_time_us":2059,"prepare.run_wall_time_us":16101,"raft.queue_time_us":155227,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":2308,"spinlock_wait_cycles":3549440,"thread_start_us":310777,"threads_started":1,"wal-append.queue_time_us":398238}
I20260709 13:03:44.660043 12572 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.714913  8680 pending_rounds.cc:85] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 8
I20260709 13:03:44.695642  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:44.743964  8680 pending_rounds.cc:107] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:03:44.744414  8680 raft_consensus.cc:2889] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:44.745193  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.722435  8823 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:44.723351  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.745690  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:44.728479  8719 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:44.634266 12700 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.729383  8822 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:44.733070  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:44.750237  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:44.750702  9382 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:44.734727  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:44.752650  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:44.738806  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.676723 12955 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:44.741122  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.647130 12702 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.763290  8719 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:44.718221 12945 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:44.753762  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:44.754856  9355 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:44.714811 12937 raft_consensus.cc:697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:44.724440  8752 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:44.765473  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.765352  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.771298  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.684580 12977 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.776187  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.778465  8826 raft_consensus.cc:2376] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9. Re-sending same reply.
I20260709 13:03:44.791708  8822 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:44.743950 12989 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.734548 13040 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:44.767735  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:44.783360  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:44.797416  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.751698 12944 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:44.801718  8754 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:03:44.802634  8754 raft_consensus.cc:606] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:44.632043 13043 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.762132 12802 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:44.766361 12971 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:44.820134  8752 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:44.746300 12994 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:03:44.812036 12796 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:44.791450 12756 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:44.831915 12756 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.832506 12572 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:44.834205 12756 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:44.834825 12977 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.790673 12955 raft_consensus.cc:697] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:44.815424 12976 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.734057 12943 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.816160 12957 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:44.816855 12702 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.842319  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:44.817471 12700 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:44.797016 12749 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:44.798759 12703 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:44.780236 12790 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:03:44.809731  8675 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:03:44.772037 13042 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:03:44.883363  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.884164  8821 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:44.884227  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:44.780576 12992 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11.
I20260709 13:03:44.771853 12937 consensus_queue.cc:237] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.885442  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.851240 12957 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.887003  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:44.828994  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.888271 12802 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:44.888609 12957 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.838217  8825 raft_consensus.cc:2368] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11.
I20260709 13:03:44.889660  8821 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:44.890956  8825 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:03:44.891417  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:44.862419  8751 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:44.892436  8825 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:44.893060  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2477) took 1016 ms. Trace:
I20260709 13:03:44.893487  8751 rpcz_store.cc:276] 0709 13:03:43.876727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.640089 (+763362us) service_pool.cc:224] Handling call
0709 13:03:44.640375 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.892170 (+251795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.892808 (+   638us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54001577d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:44.892991 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510136576}
I20260709 13:03:44.894254  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:44.863554  8675 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:03:44.894382  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.894867 12790 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:44.895166  8827 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:44.896014  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:44.896371  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.896943  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:44.897156 12934 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:44.896987 12957 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:44.867990 12994 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:44.856681 13024 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.898872 12790 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:44.899415 12790 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:44.899828 12790 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:44.904983  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.905756  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:44.893508  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:44.916659 12756 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:44.924650 13043 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.799448 12989 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:44.927263 12989 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.853302  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:44.877810 12749 raft_consensus.cc:697] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:44.927687  8793 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:44.928660 12985 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:44.929630 12985 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:44.929968 12984 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:44.930646 12984 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.931654  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:44.932047  8826 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:03:44.932512 12974 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:44.933027 12984 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:44.880762  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1947) took 1489 ms. Trace:
I20260709 13:03:44.938088  8674 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:44.876313 12955 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.835924 13040 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:44.835908 13050 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:03:44.830796 12796 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:44.836328  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:44.955955  8753 raft_consensus.cc:2393] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 11: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:44.891539  8821 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:03:44.897094  8827 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:44.864477  8678 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15. Ops: [12.11-14.12]
I20260709 13:03:44.772226 12945 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:44.897750 12934 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.999825 12934 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:44.906029  8751 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:03:45.007495  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 61347.1us
I20260709 13:03:44.871670  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:45.015535  8829 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:44.930068 12985 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:44.864789  8645 leader_election.cc:336] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.722s (SENT)
W20260709 13:03:45.019470  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 154740us
I20260709 13:03:44.880383 12943 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:44.937433  8676 rpcz_store.cc:276] 0709 13:03:43.391443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.666221 (+274778us) service_pool.cc:224] Handling call
0709 13:03:43.666491 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:44.579200 (+912709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.880406 (+301206us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b54002a99d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:44.880680 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2096680192}
I20260709 13:03:44.942533  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 693) took 1262 ms. Trace:
I20260709 13:03:44.943388  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:44.944193  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:45.057915  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 113845us
W20260709 13:03:45.058210  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 114949us
I20260709 13:03:45.057556  8755 rpcz_store.cc:276] 0709 13:03:43.680006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.680306 (+   300us) service_pool.cc:224] Handling call
0709 13:03:44.942285 (+1261979us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b5400102a54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:44.942451 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1460781568}
I20260709 13:03:45.059243  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:45.059002  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:44.878146  8672 pending_rounds.cc:85] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 5
I20260709 13:03:45.060724  8672 pending_rounds.cc:107] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:03:45.061048  8672 raft_consensus.cc:2889] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:45.061115  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:44.837827  8824 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
W20260709 13:03:44.943526 13044 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.291s	user 0.002s	sys 0.000s
W20260709 13:03:44.943691 13046 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:03:44.884303 13026 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:45.069288  8753 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:45.083096  8753 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:45.069979  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:45.070919  8827 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:45.080662  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:45.062352  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:45.080551  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:45.088836  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:45.018662 12756 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:45.080567 12802 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:45.096364 12802 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:45.096813 12802 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.097170 12802 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:45.103888  8674 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:03:45.105027  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:45.105614  8830 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:45.017885 13040 raft_consensus.cc:697] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:45.071288 12979 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:45.076589 12957 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:45.127532  8821 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:45.127679 12957 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.055752 12656 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:03:45.128350 12957 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:45.023084 12983 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.112s
I20260709 13:03:45.130401 12790 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:45.131283 12983 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:45.131894 12983 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:45.133069 12790 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:45.103091 12763 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:45.134127 12763 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.134485 12763 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:45.137179  8755 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:03:45.142599 13039 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
I20260709 13:03:45.144883  9384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:03:45.145641  9384 raft_consensus.cc:606] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:45.151602  8823 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:45.158165 13040 consensus_queue.cc:237] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.103320 12776 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:45.122220 12979 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.163686 12979 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:45.175963  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:45.178129 12702 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:45.179204 12702 consensus_queue.cc:1243] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:44.929030 12749 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:45.081511 12954 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:45.130963 12974 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:45.185798  8827 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:45.125495 12796 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:45.153652  8823 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:45.190521  9353 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:45.198427  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.207561  8827 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:45.208127  8755 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:03:45.219110  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 456301us
W20260709 13:03:45.221171 12881 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.004s	sys 0.010s
I20260709 13:03:45.231551  8821 pending_rounds.cc:85] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 5
I20260709 13:03:45.243161  8821 pending_rounds.cc:107] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:03:45.231778  8719 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:45.240999  8755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:03:45.244199  8755 raft_consensus.cc:606] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:45.245059  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:45.239259  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 704) took 1363 ms. Trace:
I20260709 13:03:45.247213  8748 rpcz_store.cc:276] 0709 13:03:43.876201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.598676 (+722475us) service_pool.cc:224] Handling call
0709 13:03:44.599097 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:44.599566 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.599608 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:44.600257 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 13:03:45.003882 (+403625us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:45.078244 (+ 74362us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.091114 (+ 12870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.091204 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.190160 (+ 98956us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b1800399340. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:03:45.229793 (+ 39633us) raft_consensus.cc:1685] finished
0709 13:03:45.238911 (+  9118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.239156 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72691,"prepare.queue_time_us":76763,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":18206,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":35920,"spinlock_wait_cycles":37791488,"thread_start_us":403156,"threads_started":1,"wal-append.queue_time_us":564136}
I20260709 13:03:45.243476  8821 raft_consensus.cc:2889] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:45.250290  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2476) took 1460 ms. Trace:
I20260709 13:03:45.250906  8750 rpcz_store.cc:276] 0709 13:03:43.789469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.261385 (+471916us) service_pool.cc:224] Handling call
0709 13:03:44.261749 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:44.310791 (+ 49042us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.310824 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:44.350152 (+ 39328us) log.cc:844] Serialized 26 byte log entry
0709 13:03:44.523210 (+173058us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:44.523253 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.523329 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:44.913751 (+390422us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:45.242123 (+328372us) raft_consensus.cc:1685] finished
0709 13:03:45.242150 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.250200 (+  8050us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48604,"raft.queue_time_us":97249,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":23096,"spinlock_wait_cycles":50498304,"thread_start_us":126750,"threads_started":1,"wal-append.queue_time_us":256389}
I20260709 13:03:45.257166  8719 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:45.133059 12983 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:45.242985 12881 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.004s	sys 0.010s
I20260709 13:03:45.260146 12983 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:45.223960 13012 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:45.269285  8821 pending_rounds.cc:107] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:03:45.271992  8821 raft_consensus.cc:2889] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:45.274173 12946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.012s	sys 0.011s
W20260709 13:03:45.278229  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 510585us
I20260709 13:03:45.221998 12702 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:03:45.257833 12943 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:45.239917 13026 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:03:45.241374 12976 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
W20260709 13:03:45.285056 12946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.013s	sys 0.011s
I20260709 13:03:45.312565  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:45.339555  8748 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:45.316457  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:45.313751 12943 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.343384 12943 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:45.343838  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:45.344548  8750 raft_consensus.cc:2376] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8. Re-sending same reply.
I20260709 13:03:45.345554  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:45.316396 12963 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:45.316378 12962 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:45.319590 13026 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:45.347790 13026 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:45.348489  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:45.348889 13026 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:45.321259 13053 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.566s	user 0.005s	sys 0.000s
I20260709 13:03:45.324548  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:45.351894  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:45.353149  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:45.305677 12986 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:45.355916 12934 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.358237 12934 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:45.308892 13061 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:45.359093  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 698) took 1657 ms. Trace:
I20260709 13:03:45.330955  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:45.333127 12871 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.587s	user 0.012s	sys 0.000s
I20260709 13:03:45.280655 12796 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.365420 12871 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:45.365576 12796 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.365891 12802 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.362468  8751 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:45.359558  8747 rpcz_store.cc:276] 0709 13:03:43.701851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.701996 (+   145us) service_pool.cc:224] Handling call
0709 13:03:43.702562 (+   566us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:43.721210 (+ 18648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.721262 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:43.748350 (+ 27088us) log.cc:844] Serialized 52 byte log entry
0709 13:03:44.609698 (+861348us) spinlock_profiling.cc:243] Waited 78.9 ms on lock 0x7b1800241780. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:03:44.748654 (+138956us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:44.781415 (+ 32761us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.033877 (+252462us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.087826 (+ 53949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.087912 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.135155 (+ 47243us) spinlock_profiling.cc:243] Waited 5.51 ms on lock 0x7b1800241780. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db
0709 13:03:45.148577 (+ 13422us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b1800241780. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:03:45.320368 (+171791us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:45.358819 (+ 38451us) raft_consensus.cc:1685] finished
0709 13:03:45.358847 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.359014 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63039,"prepare.queue_time_us":72332,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":101,"raft.queue_time_us":122799,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":30548,"spinlock_wait_cycles":214793472,"thread_start_us":360477,"threads_started":2,"wal-append.queue_time_us":962890}
I20260709 13:03:45.367698  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:45.368389  8747 raft_consensus.cc:2376] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14. Re-sending same reply.
I20260709 13:03:45.364610  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:45.346347  8750 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:45.379899  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:45.381788  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:45.384368  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:45.386416  8748 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
W20260709 13:03:45.401889  8675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.004s	sys 0.039s
W20260709 13:03:45.402660  8675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.005s	sys 0.039s
I20260709 13:03:45.404075  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:45.407287  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 96227.4us
I20260709 13:03:45.350476 13026 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:45.409523  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:45.409673  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:45.413424  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:45.417841  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:45.418027  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:45.418579  8748 raft_consensus.cc:2376] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11. Re-sending same reply.
I20260709 13:03:45.420238  8748 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:45.314443 12630 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 13:03:45.421577  8747 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:45.423166  8748 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:45.427066  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:45.385085 12763 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.384356 12976 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:45.433254  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:45.437558 12957 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.445055  8748 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:45.446144  8750 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:45.448508  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:45.451453  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:45.452625  8754 raft_consensus.cc:2410] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:03:45.451473  8748 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.465197  8748 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:45.465747  8722 leader_election.cc:336] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.157s (SENT)
I20260709 13:03:45.466226  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:45.415745 12962 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:03:45.479579  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:45.480829  8676 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:03:45.488685  8748 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:03:45.489677  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:45.532547  8748 pending_rounds.cc:85] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 6
I20260709 13:03:45.532299  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 725) took 1080 ms. Trace:
I20260709 13:03:45.532955  8748 pending_rounds.cc:107] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:03:45.533340  8748 raft_consensus.cc:2889] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:45.490417  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:45.482782  8719 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:45.463622 12984 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:45.544145  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 61463.8us
I20260709 13:03:45.545802 13060 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:45.546468 13060 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.546957 13060 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:45.458973 12976 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.492450 13061 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:03:45.528910  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 63177.2us
I20260709 13:03:45.552380 13050 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.552743 12976 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:45.553227 13050 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.530912  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:45.555367  9387 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:45.556355 13050 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:45.533847  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:45.562848  8676 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:45.490236 12763 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:45.533061  8680 rpcz_store.cc:276] 0709 13:03:44.451989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.713592 (+261603us) service_pool.cc:224] Handling call
0709 13:03:44.714412 (+   820us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:44.772497 (+ 58085us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.772544 (+    47us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:44.773284 (+   740us) log.cc:844] Serialized 78 byte log entry
0709 13:03:45.224667 (+451383us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:45.250078 (+ 25411us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.257400 (+  7322us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.531705 (+274305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.531849 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.531919 (+    70us) raft_consensus.cc:1685] finished
0709 13:03:45.531941 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.532194 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13177,"prepare.queue_time_us":32312,"raft.queue_time_us":20733,"raft.run_cpu_time_us":504,"raft.run_wall_time_us":28142,"spinlock_wait_cycles":25460608,"thread_start_us":670342,"threads_started":2,"wal-append.queue_time_us":563830}
I20260709 13:03:45.564838  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:45.565848  8680 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:45.570706  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:45.572103 13050 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:45.572687 13050 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.573128 13050 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:45.575387  8719 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:45.565945  8676 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:45.590235 13062 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:45.590947 13062 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.591426 13062 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:45.595724 13061 consensus_queue.cc:1059] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:45.597277 12953 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:45.597708 12953 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:45.598845 12953 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.600879 12953 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:45.604561  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.482066 12979 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.503437 12871 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:45.633931  8722 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.685s (SENT) [suppressed 51 similar messages]
I20260709 13:03:45.527828 12796 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:45.653656  9387 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
W20260709 13:03:45.639855 13024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.026s	sys 0.034s
I20260709 13:03:45.662333  8796 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:45.663877  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:45.738967  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 76790.4us
I20260709 13:03:45.664199  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:45.715026 12957 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:45.703559  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2480) took 1802 ms. Trace:
W20260709 13:03:45.741602 13068 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 13:03:45.740782  8756 rpcz_store.cc:276] 0709 13:03:43.900981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.745916 (+844935us) service_pool.cc:224] Handling call
0709 13:03:44.773763 (+ 27847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.774000 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:45.269976 (+495976us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.658149 (+388173us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.702976 (+ 44827us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:45.703063 (+    87us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:45.703166 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.703268 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.703464 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62269,"prepare.queue_time_us":64248,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":100,"raft.queue_time_us":52967,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":9418,"spinlock_wait_cycles":165026048,"thread_start_us":818835,"threads_started":2}
I20260709 13:03:45.749389  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1805) took 1290 ms. Trace:
W20260709 13:03:45.719934  8753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.024s	sys 0.021s
W20260709 13:03:45.750239  8753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.024s	sys 0.021s
I20260709 13:03:45.750666  8681 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:45.756381  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2422) took 1105 ms. Trace:
I20260709 13:03:45.756999  8675 rpcz_store.cc:276] 0709 13:03:44.651226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.896548 (+245322us) service_pool.cc:224] Handling call
0709 13:03:44.896940 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:44.897173 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.897201 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:44.898028 (+   827us) log.cc:844] Serialized 26 byte log entry
0709 13:03:45.403292 (+505264us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:45.403360 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.403475 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.756084 (+352609us) raft_consensus.cc:1685] finished
0709 13:03:45.756109 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.756301 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":504730,"threads_started":1,"wal-append.queue_time_us":840906}
I20260709 13:03:45.758576  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2419) took 1108 ms. Trace:
I20260709 13:03:45.759114  8675 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:03:45.759052  8672 rpcz_store.cc:276] 0709 13:03:44.650461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.877373 (+226912us) service_pool.cc:224] Handling call
0709 13:03:44.877887 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:45.061710 (+183823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:45.061755 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:45.062248 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:03:45.461522 (+399274us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:45.494659 (+ 33137us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.495300 (+   641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.495400 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.758326 (+262926us) raft_consensus.cc:1685] finished
0709 13:03:45.758348 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.758498 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36879,"raft.queue_time_us":359,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":3804,"spinlock_wait_cycles":49560192,"thread_start_us":398789,"threads_started":1,"wal-append.queue_time_us":673334}
W20260709 13:03:45.724588  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 377989us
I20260709 13:03:45.653528 12802 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:45.711572 12976 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:45.761579  8672 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:45.761482 12976 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:45.763945 12976 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:45.738590 13024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.026s	sys 0.034s
W20260709 13:03:45.718539 12945 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.654s	user 0.011s	sys 0.003s
I20260709 13:03:45.764842 12945 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:45.766927  8752 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:45.781268  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.020s	sys 0.017s
W20260709 13:03:45.781839  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.020s	sys 0.017s
W20260709 13:03:45.782518  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 693742us
I20260709 13:03:45.784439  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:45.746035 12871 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:45.803939  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:45.807503  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:45.809051  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:45.794004 12802 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:03:45.825554 13062 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.826159 12802 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:45.830386 12802 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:45.831941 12802 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:45.839461 13071 log.cc:927] Time spent T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 13:03:45.841876 13070 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 13:03:45.842829 13069 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 13:03:45.845772 13060 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.802488 12992 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:45.854010  8719 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:45.805446 12871 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:45.812947 12974 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:03:45.858400  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 224511us
W20260709 13:03:45.863070  8722 leader_election.cc:336] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.018s (SENT)
I20260709 13:03:45.823266 12954 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:45.863631  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:45.858675 12974 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.859889 13057 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:45.871868 12954 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.863608 12944 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:03:45.887312 12974 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:45.897879 12954 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:45.860582 12992 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:45.898908 13077 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:03:45.911996 13050 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.874035 12763 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
W20260709 13:03:45.903875 13074 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.170s	user 0.001s	sys 0.001s
I20260709 13:03:45.914001 12871 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:45.919528 12986 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.565s	user 0.006s	sys 0.005s
I20260709 13:03:45.943150  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1817) took 1307 ms. Trace:
I20260709 13:03:45.943689  8824 rpcz_store.cc:276] 0709 13:03:44.635365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.064265 (+428900us) service_pool.cc:224] Handling call
0709 13:03:45.064662 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:45.076008 (+ 11346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:45.076042 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:45.076570 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:03:45.458538 (+381968us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:45.458599 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.462429 (+  3830us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.942860 (+480431us) raft_consensus.cc:1685] finished
0709 13:03:45.942885 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.943070 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10917,"raft.queue_time_us":15697,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":5048,"thread_start_us":306095,"threads_started":1,"wal-append.queue_time_us":536824}
I20260709 13:03:45.947477  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 727) took 1495 ms. Trace:
I20260709 13:03:45.947975  8678 rpcz_store.cc:276] 0709 13:03:44.452493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.784224 (+331731us) service_pool.cc:224] Handling call
0709 13:03:44.864089 (+ 79865us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:44.961638 (+ 97549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.947255 (+985617us) spinlock_profiling.cc:243] Waited 79.4 ms on lock 0x7b54000dacd0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:45.947412 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182570240}
I20260709 13:03:45.950057  8826 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:45.928319 12986 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:45.951522  8827 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:45.952358 13050 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:45.952806  8827 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:45.953874 13060 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:45.950857  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1809) took 1393 ms. Trace:
I20260709 13:03:45.954857  8830 rpcz_store.cc:276] 0709 13:03:44.557639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.828337 (+270698us) service_pool.cc:224] Handling call
0709 13:03:45.950650 (+1122313us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54002288d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:45.950786 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386787072}
I20260709 13:03:45.959282  8678 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:45.974661  9388 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
W20260709 13:03:45.984694  8645 leader_election.cc:336] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.949s (SENT)
I20260709 13:03:45.985378  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:45.993036  8645 leader_election.cc:336] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 2.204s (SENT)
I20260709 13:03:45.993805  8645 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:45.996210  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 177462us
I20260709 13:03:45.974697 12976 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:45.998378  8793 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:46.000658  8678 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:46.006552  8827 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:45.973824 13080 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:45.929030 13057 consensus_queue.cc:1059] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 13:03:45.988050 12944 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:45.998092 12957 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:45.983834 13060 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:46.020745  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:46.027436 12992 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:46.023715 13080 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:45.996160 13026 consensus_queue.cc:1059] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:03:45.997663 13024 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:03:46.015508 12976 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:46.022101 12934 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.019223 13076 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:46.023947 12944 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:46.039949  8824 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:46.013063 12592 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:03:46.017647 12955 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
W20260709 13:03:46.041422  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 178377us
I20260709 13:03:46.056638  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:46.026691 12831 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
W20260709 13:03:46.064829  8722 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.618s (SENT)
I20260709 13:03:46.068776  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:46.052124 13081 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.178s	user 0.001s	sys 0.001s
I20260709 13:03:46.040526 12957 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:46.042207 13085 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 13:03:46.043665 12985 consensus_meta.cc:287] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.976s	user 0.015s	sys 0.000s
I20260709 13:03:46.041739 12965 raft_consensus.cc:3055] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:46.079274 13080 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:46.083842  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 85560.2us
I20260709 13:03:46.066000 12986 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:46.087188  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 229201us
W20260709 13:03:46.088862  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 68229.9us
I20260709 13:03:46.090234  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:46.092262  8645 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:46.093992  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.074985 12977 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.044636 12934 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:46.098682  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:46.102813  8751 raft_consensus.cc:2376] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11. Re-sending same reply.
I20260709 13:03:46.099385  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:46.099635  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:46.101223  8750 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:46.081033 12965 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:46.083038 12947 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:46.112991  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:46.113806  8722 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:46.115403  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:46.107460 13075 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:03:46.116513  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:46.096443 12957 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:46.118069 13082 log.cc:927] Time spent T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:03:46.087252 12944 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:46.136742 12944 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:46.116966 12657 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:46.139067  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2490) took 1235 ms. Trace:
I20260709 13:03:46.139515  8753 rpcz_store.cc:276] 0709 13:03:44.903764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.093390 (+189626us) service_pool.cc:224] Handling call
0709 13:03:45.093676 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:45.094828 (+  1152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:45.094860 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:45.150516 (+ 55656us) log.cc:844] Serialized 26 byte log entry
0709 13:03:45.750960 (+600444us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:45.751011 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.751088 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:46.138729 (+387641us) raft_consensus.cc:1685] finished
0709 13:03:46.138776 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.138993 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30192,"raft.queue_time_us":9683,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":19756,"thread_start_us":599405,"threads_started":1,"wal-append.queue_time_us":718994}
W20260709 13:03:46.120216 12953 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.519s	user 0.010s	sys 0.003s
I20260709 13:03:46.141911  8722 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:46.143514 13095 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:46.144359  8748 raft_consensus.cc:2410] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:46.123871 13092 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:46.114173 12943 consensus_meta.cc:287] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.768s	user 0.014s	sys 0.000s
I20260709 13:03:46.123659 12965 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:46.107704 12977 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:46.126431 12934 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:46.128180 13078 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.310s	user 0.001s	sys 0.002s
I20260709 13:03:46.128592 12976 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:46.154017 12983 consensus_meta.cc:287] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.893s	user 0.000s	sys 0.014s
I20260709 13:03:46.084326 12985 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:46.154300 12965 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:46.167644 12985 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:46.168467  8829 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:46.168570 12657 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:46.159255 12954 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:46.148655 12943 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:46.165709 12977 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:46.178965 12943 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:46.205904  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:46.206113  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:46.210752  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:46.211299 12983 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.211954 12983 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:46.212184  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:46.213407  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:46.214059  8830 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:46.214119 12983 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:46.207032  8828 raft_consensus.cc:2376] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11. Re-sending same reply.
I20260709 13:03:46.215591  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:46.215941  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:46.216714  8826 raft_consensus.cc:2368] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11.
I20260709 13:03:46.217538  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:46.217276 12974 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:46.218101  8826 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:46.218891  8822 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:03:46.219681 12974 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:46.220456  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:46.220724  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:46.232841  8826 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:46.224207 12954 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:46.225016 12986 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:46.257684  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:46.266532  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:46.268467  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:46.269680  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:46.272948 12965 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:46.273703 12965 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:46.275728  9414 heartbeater.cc:646] Failed to heartbeat to 127.7.89.190:45099 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.7.89.190:45099 timed out after 15.000s (SENT)
I20260709 13:03:46.280557  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45676 (request call id 8) took 1056 ms. Trace:
I20260709 13:03:46.281454  8830 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:46.281286  8755 rpcz_store.cc:276] 0709 13:03:45.224476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.224602 (+   126us) service_pool.cc:224] Handling call
0709 13:03:46.280098 (+1055496us) spinlock_profiling.cc:243] Waited 93.5 ms on lock 0x7b54003f6720. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2476520 00007f32e24f3652 00007f32e74c160a 00007f32e0a477d4 00007f32e0a47762 00007f32e0a47691 00007f32e0a4760c 00007f32e0a468c2 00007f32e083448c 00007f32e08338c6
0709 13:03:46.280431 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215051008}
I20260709 13:03:46.277902 12965 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:46.290519 13079 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.473s	user 0.002s	sys 0.000s
I20260709 13:03:46.293830  8826 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:46.304549  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:46.308142  8828 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:46.309206  8826 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:46.312050  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:46.313311  8755 raft_consensus.cc:2410] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:03:46.316078  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:46.317956  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:46.299503  8830 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:46.322024  8828 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:03:46.332693  8828 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:46.338858 12749 raft_consensus.cc:993] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: : Instructing follower e814ed2013244fa6be42623ea5c0c7b9 to start an election
I20260709 13:03:46.338570 12862 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:46.339419 12749 raft_consensus.cc:1081] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Signalling peer e814ed2013244fa6be42623ea5c0c7b9 to start an election
W20260709 13:03:46.337837 13042 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.453s	user 0.008s	sys 0.017s
W20260709 13:03:46.341037 13042 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.456s	user 0.009s	sys 0.017s
I20260709 13:03:46.343806 12979 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:46.343770  8823 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:55168
I20260709 13:03:46.344936  8823 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:46.345425  8823 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:46.353678 13101 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:46.371906 12831 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:46.373018 12831 consensus_queue.cc:1243] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:03:46.373631 12831 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:46.373992 12831 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:46.386445 12862 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:46.387555 12862 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:46.391417 12831 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:46.394961  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.431450 12979 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:46.395792  8749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2483) took 2469 ms (client timeout 1623 ms). Trace:
I20260709 13:03:46.399729 12945 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:46.417693  8756 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:46.419039 12763 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:46.432653  8749 rpcz_store.cc:269] 0709 13:03:43.926845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.996797 (+ 69952us) service_pool.cc:224] Handling call
0709 13:03:46.395459 (+2398662us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540018d8d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:46.395660 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2556743040}
I20260709 13:03:46.438278 12831 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:46.432137 12979 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:46.441285 12979 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:46.445096  8823 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:46.457437  8828 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:46.463718  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:46.464541  8674 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:46.442415 12763 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:46.675434 13103 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:46.717387 13103 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:46.720536 13103 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:46.738242 13111 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:03:46.819617  8674 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:46.825384  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:46.828856  8674 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:46.985553  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:47.382769  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:47.676028  8777 debug-util.cc:398] Leaking SignalData structure 0x7b08001ea2a0 after lost signal to thread 7526
W20260709 13:03:47.676716  8777 debug-util.cc:398] Leaking SignalData structure 0x7b08000a3000 after lost signal to thread 7529
I20260709 13:03:47.699167 12976 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:47.699790  8674 consensus_meta.cc:287] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.869s	user 0.012s	sys 0.000s
I20260709 13:03:47.700829  8674 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:47.702419  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1991) took 1272 ms. Trace:
I20260709 13:03:47.699960 12976 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:47.703320  8674 rpcz_store.cc:276] 0709 13:03:46.430009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.824964 (+394955us) service_pool.cc:224] Handling call
0709 13:03:47.702200 (+877236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:47.705009  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 708) took 3516 ms. Trace:
I20260709 13:03:47.705814 12976 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:47.706584 12976 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:47.707450 12976 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:47.705425  8752 rpcz_store.cc:276] 0709 13:03:44.188867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.821320 (+632453us) service_pool.cc:224] Handling call
0709 13:03:44.831100 (+  9780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:45.767479 (+936379us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.704692 (+1937213us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b5400139554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:47.704942 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2151672192}
I20260709 13:03:47.708608  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:47.715485  8674 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:47.716267 12976 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:47.718087 12976 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:47.719524 12976 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:47.726876 12976 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:47.728379 12976 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:47.729796 12976 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:47.736276 12976 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:47.738269 12976 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:47.739782 12976 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:47.750808 12976 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:47.988001  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.027101 12976 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:48.027848 12976 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.036051  8674 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:48.036903  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1987) took 1608 ms. Trace:
I20260709 13:03:48.037312  8674 rpcz_store.cc:276] 0709 13:03:46.428921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.707159 (+1278238us) service_pool.cc:224] Handling call
0709 13:03:48.036820 (+329661us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:48.038561  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:48.048089  8674 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:48.054131 12763 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:48.078279  8674 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:48.030329 12976 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:48.091297  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 1989) took 1661 ms. Trace:
I20260709 13:03:48.076092 13035 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.092151 13116 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:48.096261 13035 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:48.096243 13116 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.099056 13116 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:48.094238  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 747) took 1667 ms. Trace:
I20260709 13:03:48.100203  8677 rpcz_store.cc:276] 0709 13:03:46.427262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.093141 (+1665879us) service_pool.cc:224] Handling call
0709 13:03:48.093536 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.093777 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.093821 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:48.093890 (+    69us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:48.093960 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.094019 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.094171 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:48.102694  8674 rpcz_store.cc:276] 0709 13:03:46.429470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.038210 (+1608740us) service_pool.cc:224] Handling call
0709 13:03:48.091197 (+ 52987us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:48.104400  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 749) took 1676 ms. Trace:
I20260709 13:03:48.104776  8674 rpcz_store.cc:276] 0709 13:03:46.427793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.103515 (+1675722us) service_pool.cc:224] Handling call
0709 13:03:48.103887 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.104081 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.104106 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:48.104147 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:48.104178 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.104232 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.104339 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:48.106585  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 748) took 1679 ms. Trace:
I20260709 13:03:48.106994  8678 rpcz_store.cc:276] 0709 13:03:46.427537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.105728 (+1678191us) service_pool.cc:224] Handling call
0709 13:03:48.106025 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.106256 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.106283 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:48.106327 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:48.106365 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.106420 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.106525 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:48.107506  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 744) took 1681 ms. Trace:
I20260709 13:03:48.108780  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 751) took 1680 ms. Trace:
I20260709 13:03:48.109179  8678 rpcz_store.cc:276] 0709 13:03:46.428305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.107945 (+1679640us) service_pool.cc:224] Handling call
0709 13:03:48.108289 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.108475 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.108499 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:48.108540 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:48.108575 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.108627 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.108728 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:48.114831  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.108002  8681 rpcz_store.cc:276] 0709 13:03:46.426320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.430814 (+  4494us) service_pool.cc:224] Handling call
0709 13:03:46.431204 (+   390us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:46.431641 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:46.431983 (+   342us) log.cc:844] Serialized 14 byte log entry
0709 13:03:46.797271 (+365288us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:46.955022 (+157751us) log.cc:844] Serialized 52 byte log entry
0709 13:03:47.019495 (+ 64473us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:47.019887 (+   392us) log.cc:844] Serialized 14 byte log entry
0709 13:03:47.020520 (+   633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.020649 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:48.107251 (+1086602us) raft_consensus.cc:1685] finished
0709 13:03:48.107277 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.107440 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1080684,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":1172,"raft.queue_time_us":1932,"raft.run_cpu_time_us":450,"raft.run_wall_time_us":53899,"spinlock_wait_cycles":1920,"thread_start_us":363793,"threads_started":1,"wal-append.queue_time_us":1661218}
I20260709 13:03:48.118367  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1990) took 1688 ms. Trace:
I20260709 13:03:48.119928  8677 rpcz_store.cc:276] 0709 13:03:46.429749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.110916 (+1681167us) service_pool.cc:224] Handling call
0709 13:03:48.117674 (+  6758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.117922 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.117960 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:48.118012 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:48.118061 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.118139 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.118291 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:48.136822  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 742) took 1711 ms. Trace:
I20260709 13:03:48.137202  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 752) took 1708 ms. Trace:
I20260709 13:03:48.137423  8679 rpcz_store.cc:276] 0709 13:03:46.425490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.427005 (+  1515us) service_pool.cc:224] Handling call
0709 13:03:46.427430 (+   425us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:46.427941 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:46.428286 (+   345us) log.cc:844] Serialized 14 byte log entry
0709 13:03:46.768931 (+340645us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:46.769839 (+   908us) log.cc:844] Serialized 52 byte log entry
0709 13:03:46.770036 (+   197us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:46.770288 (+   252us) log.cc:844] Serialized 14 byte log entry
0709 13:03:46.770854 (+   566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.770965 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:48.136534 (+1365569us) raft_consensus.cc:1685] finished
0709 13:03:48.136555 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.136739 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1333169,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":90,"raft.queue_time_us":3670,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":340,"thread_start_us":339690,"threads_started":1,"wal-append.queue_time_us":1693879}
I20260709 13:03:48.137710  8678 rpcz_store.cc:276] 0709 13:03:46.428604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.110003 (+1681399us) service_pool.cc:224] Handling call
0709 13:03:48.110316 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.110545 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:48.120763 (+ 10218us) log.cc:844] Serialized 14 byte log entry
0709 13:03:48.136769 (+ 16006us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:48.136842 (+    73us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:48.136905 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.136995 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.137135 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":138,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":1538,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":108,"thread_start_us":15551,"threads_started":1}
I20260709 13:03:48.143805  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 745) took 1717 ms. Trace:
I20260709 13:03:48.144223  8675 rpcz_store.cc:276] 0709 13:03:46.426699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.430813 (+  4114us) service_pool.cc:224] Handling call
0709 13:03:48.143180 (+1712367us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:03:48.143379 (+   199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.143545 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.143561 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:48.143594 (+    33us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:48.143621 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.143660 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.143752 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3116544}
W20260709 13:03:48.163203 12831 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.724s	user 0.016s	sys 0.011s
I20260709 13:03:48.167562 12749 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.168228 12749 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:48.164279  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.061s	user 0.011s	sys 0.027s
W20260709 13:03:48.174248  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.071s	user 0.013s	sys 0.028s
W20260709 13:03:48.175395  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 2.07265e+06us
I20260709 13:03:48.177614  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:48.179003  8645 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:48.179859 12976 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:48.180299  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 765) took 2233 ms. Trace:
I20260709 13:03:48.180328  8645 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:48.180708 12976 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:48.180783  8822 rpcz_store.cc:276] 0709 13:03:45.946865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.964480 (+ 17615us) service_pool.cc:224] Handling call
0709 13:03:45.965010 (+   530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:46.219262 (+254252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.180008 (+1960746us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54002292d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:48.180227 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583202048}
I20260709 13:03:48.181433 12976 raft_consensus.cc:697] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:48.182286  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:48.182566 12976 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.184830  8645 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:48.185523 13101 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:48.186081  8645 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:48.189283  8645 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:48.186372 13101 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:48.190747 12976 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:48.207464 13116 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:48.214636  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:48.214896 12976 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:48.215679 13116 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:48.216323 13012 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.216507 12976 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:48.216810 13116 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:48.216984  8645 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:48.218106 12976 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:48.218470  8645 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:48.219452 12976 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:48.219973  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:48.220831 12976 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:48.221220  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 757) took 3604 ms. Trace:
I20260709 13:03:48.221655 12976 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:48.197067 13100 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.223624  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.224524  9388 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:48.224352 13100 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.230579 12640 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:48.231343 12640 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.234097 12640 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:48.234754  8794 connection.cc:256] RPC call timeout handler was delayed by -2.8479995469999722s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
I20260709 13:03:48.221809  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:48.221642  8829 rpcz_store.cc:276] 0709 13:03:44.616414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.016260 (+399846us) service_pool.cc:224] Handling call
0709 13:03:45.065406 (+ 49146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.168933 (+1103527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.220943 (+2052010us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003dead4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:48.221152 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2536366336}
I20260709 13:03:48.216877 13012 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:48.240190  8794 connection.cc:256] RPC call timeout handler was delayed by -2.0060005450000062s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
I20260709 13:03:48.240185 12976 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:48.243037 12763 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:48.243491 13100 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:48.240377 13116 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:03:48.244093 13100 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:48.243830 12763 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.244582 13100 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.243000 12976 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.245394 12976 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:48.248560 12640 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:48.249260 12640 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.252141 12640 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:48.253703  8796 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:45.749871  8825 rpcz_store.cc:276] 0709 13:03:44.458571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.908159 (+449588us) service_pool.cc:224] Handling call
0709 13:03:44.908558 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:44.908809 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:45.468030 (+559221us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.748794 (+280764us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:45.748907 (+   113us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:45.748975 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.749180 (+   205us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.749326 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90641,"prepare.queue_time_us":182909,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":39,"raft.queue_time_us":2613,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":675048,"threads_started":2}
I20260709 13:03:48.246315 12763 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:48.258765  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:48.261189  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:48.266984  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:48.267720 12763 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.267860  8678 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 7.
I20260709 13:03:48.268425 12763 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.271214  8679 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:48.263927 13116 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:48.275342  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.275157 13116 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.275380  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.276602 12763 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:48.264070 12640 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.277024  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.277572 13116 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:48.275621  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 2.12963e+06us
I20260709 13:03:48.278348 12976 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.279317  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:48.280095  8719 connection.cc:256] RPC call timeout handler was delayed by -1.0987854920000473s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
I20260709 13:03:48.280958  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:48.281595  8675 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:48.282174 12802 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:48.283007 12802 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:48.283632 12802 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:48.285732 12976 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:48.286183 12802 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:48.277717 12640 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:48.294461  8756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2094) took 2195 ms (client timeout 1846 ms). Trace:
I20260709 13:03:48.280920  8722 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:48.296236  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.299463  8675 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:48.324724  9388 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:03:48.307181  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:48.325695  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:48.326467  8796 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:48.307885  8677 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:48.308211  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:48.328590  8748 raft_consensus.cc:2376] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14. Re-sending same reply.
I20260709 13:03:48.329641  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:48.308888  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:48.304966  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 750) took 1876 ms. Trace:
I20260709 13:03:48.331745  8674 rpcz_store.cc:276] 0709 13:03:46.428035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.105672 (+1677637us) service_pool.cc:224] Handling call
0709 13:03:48.304180 (+198508us) spinlock_profiling.cc:243] Waited 4.79 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:03:48.304368 (+   188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.304577 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.304603 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:48.304646 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:48.304689 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.304745 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.304891 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11008896}
I20260709 13:03:48.333205  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:48.334558  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:48.327134  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:48.336016  8675 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:48.318527  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:48.323938  8756 rpcz_store.cc:269] 0709 13:03:46.098553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.098705 (+   152us) service_pool.cc:224] Handling call
0709 13:03:48.294220 (+2195515us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54001586d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:48.294375 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":698369280}
I20260709 13:03:48.319423  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:48.343305  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:48.347616 12983 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:48.348057  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:48.319756  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:48.290854 12763 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:03:48.290064  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:48.294905 12640 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:48.341503  8823 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:48.351282  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:48.319666  8681 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:48.286592 12802 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:48.297082 12796 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:03:48.344739  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:48.325565  8749 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:48.363611  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:55168 (request call id 782) took 2008 ms. Trace:
I20260709 13:03:48.292484 12944 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:03:48.309636 13101 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:48.317725  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:48.366024  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:48.366829  8750 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
W20260709 13:03:48.316378  8647 connection.cc:256] RPC call timeout handler was delayed by -1.3800997660000576s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
W20260709 13:03:48.368005  8647 connection.cc:256] RPC call timeout handler was delayed by -1.338040220000039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18460000000000001s for a response
W20260709 13:03:48.368372  8647 connection.cc:256] RPC call timeout handler was delayed by -1.2564606460000505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 13:03:48.348165 12983 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.348856  9388 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:48.349463  8752 raft_consensus.cc:2410] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:48.330240  8748 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
W20260709 13:03:48.369974  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 94931.5us
I20260709 13:03:48.350345  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:48.372991  8675 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:48.352049  8756 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:48.369947  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:48.375401  8749 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:48.352526  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:48.353737  8754 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:48.266079 13100 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:48.357023 13062 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:48.361869  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 753) took 1906 ms. Trace:
I20260709 13:03:48.362323  8751 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:48.364012  8823 rpcz_store.cc:276] 0709 13:03:46.343037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.343610 (+   573us) service_pool.cc:224] Handling call
0709 13:03:48.351730 (+2008120us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:48.311954 13087 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 2.281s	user 0.010s	sys 0.009s
I20260709 13:03:48.366024  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:48.370266  9388 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:48.381075  8755 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:48.376322  8678 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:48.381601  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.338173  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:48.378940  8681 rpcz_store.cc:276] 0709 13:03:46.455503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.117065 (+1661562us) service_pool.cc:224] Handling call
0709 13:03:48.319159 (+202094us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:03:48.319428 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.361461 (+ 42033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.361784 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5810048}
I20260709 13:03:48.382277  8825 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:48.382732  8674 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:03:48.379254  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:48.383376  8681 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:03:48.362908 12802 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:48.383782  8750 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:48.381358  9388 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.388291  9388 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:48.397135  8678 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:48.377677 12983 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:48.400463  8822 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:48.378017 13101 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.409658 13101 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:48.412917  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 1975) took 3008 ms. Trace:
I20260709 13:03:48.413447  8680 rpcz_store.cc:276] 0709 13:03:45.404706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.567073 (+162367us) service_pool.cc:224] Handling call
0709 13:03:45.567545 (+   472us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:45.568140 (+   595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:45.568186 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:45.569006 (+   820us) log.cc:844] Serialized 52 byte log entry
0709 13:03:45.868058 (+299052us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:45.868105 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.868194 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:48.311305 (+2443111us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:48.412551 (+101246us) raft_consensus.cc:1685] finished
0709 13:03:48.412584 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.412842 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":163,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":111,"spinlock_wait_cycles":3603200,"thread_start_us":298558,"threads_started":1,"wal-append.queue_time_us":418634}
I20260709 13:03:48.419039  8679 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:03:48.419174  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:48.430334  8830 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:48.422363  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.435731  9388 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:48.437712  8749 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:48.437652  8796 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:48.435715 13116 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.444700  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:48.445374  8830 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:48.446759  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.447713  8822 raft_consensus.cc:2410] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:48.448046  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.435487 13026 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.442945 12640 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:48.446445 13116 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.454769 13026 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.459452 12592 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.460067 12592 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.462093 12640 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:48.463218  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:48.464262 12640 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.475708 12955 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:03:48.466990 12763 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.467401 13100 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.468657  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:48.467202 12756 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.489480 12763 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.476002 12831 consensus_queue.cc:1059] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:03:48.490139 12756 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.490875 12831 raft_consensus.cc:3055] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:48.491271 12831 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:48.480094  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.492172 12756 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:48.492475  8823 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:48.460651 13026 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:48.493631 12955 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:48.494114 12955 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:48.484092  9387 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:48.462478 12592 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:48.495553 12955 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.497687 12955 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:48.502372  9355 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:48.492106 12831 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.492111 12763 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:48.504336  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 746) took 2077 ms. Trace:
I20260709 13:03:48.504575 13026 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:48.504510 12763 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:48.505223 13026 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.504818  8676 rpcz_store.cc:276] 0709 13:03:46.427009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.503411 (+2076402us) service_pool.cc:224] Handling call
0709 13:03:48.503794 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.503996 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.504019 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:48.504063 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:48.504094 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.504161 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.504271 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:48.505795 13026 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:48.489530 13100 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.475073 12984 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:48.527787  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.531778  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.532680  8823 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:48.536602 13100 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:48.511293 12984 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:48.543066  8793 connection.cc:256] RPC call timeout handler was delayed by -1.0158169339999858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
I20260709 13:03:48.504382 12831 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:48.538882  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.544445  8823 raft_consensus.cc:2393] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:48.547832  8830 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:03:48.546445  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.548859  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:48.549490  8823 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:48.474720 13121 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:48.545007 12756 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:48.551782  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:48.560894  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:48.466481 12983 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.554243 12756 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.564199  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:48.566285 12756 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:48.568725  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.569561 12984 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:48.569819  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:48.570489  9382 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:48.571327  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.567173 12983 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:48.567760  8822 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 15. Ops: []
I20260709 13:03:48.576736  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:48.579219  8676 raft_consensus.cc:2410] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:03:48.580569  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.580897  8680 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:48.581707  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:48.582388  8679 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:48.581009  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:48.583599  8677 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:03:48.585582  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.586504  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:48.588464  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.588860  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:48.589536  8672 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:48.590216  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:48.591328  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:48.594695 13128 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:48.595355 13128 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:48.593731  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:48.553920 13121 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.595889 13128 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:48.598886 12984 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:48.598377  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.600420  8823 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:48.601835 12945 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:48.602463 12945 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.604537 13050 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:48.604849 12945 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:48.605433 12945 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:48.606755 12943 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.607299 12943 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.607847  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:48.608752  9384 raft_consensus.cc:2410] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 13:03:48.610320  8777 debug-util.cc:398] Leaking SignalData structure 0x7b0800146120 after lost signal to thread 8576
I20260709 13:03:48.606000 12945 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.618546  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.619438  8830 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:48.611871  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:48.613982 13026 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.506440 12802 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.605254 13050 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.628507  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:48.629204  8830 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:48.629719 13050 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:48.640502  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:48.641975  8793 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:48.643325  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:48.645527  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:48.617658 13121 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:48.651634 13128 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.656378  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:03:48.657532  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.658819 13128 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:48.647826  8830 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:48.660102  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:48.659027  9353 connection.cc:256] RPC call timeout handler was delayed by -1.1771778890000633s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
I20260709 13:03:48.660758 12954 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:48.661460 12954 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:48.661991 12954 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:48.662786 12955 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.663147  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.663378 12955 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.662801  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.664970  9387 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:48.665551 12955 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:48.667683  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.668880  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.660770  8830 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:48.626794  9382 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:03:48.672480  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:48.673801  8830 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:48.669610  9384 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:48.650254  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:48.690331  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.646669 13100 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:48.647280 12763 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:48.647483 12986 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:48.693488  8796 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:48.647157 12756 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:48.680915  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.701663  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.694362  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.666227 13128 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:03:48.687489  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:48.695955  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 45799.8us
I20260709 13:03:48.679445 12976 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:03:48.648463 12796 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:03:48.756618 13035 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:48.758080  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 70703.2us
I20260709 13:03:48.752079  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.752130  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:48.762477  9382 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:48.698741 12943 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:48.766191  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 53668.4us
W20260709 13:03:48.770838  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 77453.6us
I20260709 13:03:48.751410 12756 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:48.763417 13127 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:03:48.752839 12763 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:48.769431  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:48.781315  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:48.777810 12756 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.778892 13135 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:03:48.776218  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 69636.2us
W20260709 13:03:48.770370 13126 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.070s	user 0.006s	sys 0.000s
I20260709 13:03:48.765190 12977 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:48.795192 12955 consensus_queue.cc:1059] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:48.784076 12976 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:48.805001 13035 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.805667 13035 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.806749 12763 raft_consensus.cc:697] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:48.807019 13026 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:48.807735 13026 raft_consensus.cc:3055] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:48.808218 13026 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:46.667650 12979 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:48.808921 12979 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.809247 13026 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.807837 12763 consensus_queue.cc:237] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.812017  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:48.824568  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.833545 12976 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:48.834133 13076 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:48.834564 13101 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:48.834806 13076 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.835960 13035 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:48.837097 12796 raft_consensus.cc:3055] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:48.837327 13076 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:48.837496 12796 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:48.837880  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.838979 12802 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:48.835335 13101 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:48.840413 13101 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.838557 12796 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.844702 12796 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:03:48.850883  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:48.873426  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.861331  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.874369  8821 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:48.816509 12945 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:48.894152  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 83621.2us
W20260709 13:03:48.896055  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 123026us
W20260709 13:03:48.898413  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 117224us
I20260709 13:03:48.891306 12640 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:48.908833  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:48.909168  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:48.909616  8823 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:48.911765  8647 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:48.913357 12976 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:48.918804 12954 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.921391 12954 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:48.910880 12640 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:48.922775 12640 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.924134 12640 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:48.926335  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.915838 13076 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:48.935369 13076 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:48.935981 13076 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:48.931490  8576 catalog_manager.cc:5697] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:48.896535 13139 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:03:48.911666 13135 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:48.917088  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:48.925186 13128 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:03:48.958406 13128 consensus_queue.cc:1243] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:48.959830 13135 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:48.964610  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 59644.6us
W20260709 13:03:48.965466  9353 leader_election.cc:336] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.732s (SENT)
I20260709 13:03:48.970240  8822 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:48.976723 12979 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:48.947481 13148 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:48.982237 13148 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:48.982755 13148 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:48.980039 12630 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:48.987015  8576 catalog_manager.cc:5697] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:48.988956  8576 catalog_manager.cc:5697] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:48.991122 12881 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:48.993278 12881 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:03:48.993624  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.994241 12881 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:48.996526 12881 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:48.998152 13145 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:48.944278 12977 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:48.998656 13145 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:48.999145 13042 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.999728 13042 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:48.999760 13145 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.001189 13145 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:49.003852 13057 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:49.004528 13057 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.006870 12776 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:49.007448 12776 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:49.008479 12776 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.009927 12776 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:49.011444 13095 raft_consensus.cc:3055] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:49.011916 13095 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:49.012851 13095 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.014062 13148 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:49.016136 13061 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:49.016937 13148 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:49.016722 13061 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:49.017856 12992 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:49.019003 13061 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:49.018671 12992 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:49.019697 13061 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:49.020629 13061 consensus_queue.cc:1059] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:49.021785 12992 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:49.022195 13061 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:49.022904 13061 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:49.023826 13061 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.024392 13042 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:48.996541 13040 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.025715 13061 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:49.025404 13040 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.026192 13012 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:49.018348  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 660541us
I20260709 13:03:49.076905  8755 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:49.014237 13095 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:49.017915  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 737038us
I20260709 13:03:49.087182  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:49.088508  8722 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:49.090148  8722 leader_election.cc:400] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11.
I20260709 13:03:49.027184  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:49.027189  8576 catalog_manager.cc:5697] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 7 to 11, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 11 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:03:48.945960  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:49.046648  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.091197  8827 raft_consensus.cc:2376] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10. Re-sending same reply.
I20260709 13:03:49.080034  8751 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:49.094588  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:49.093411  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:49.097311  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 151455us
I20260709 13:03:49.098474  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:49.097299  8750 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:03:49.101567  8750 pending_rounds.cc:85] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 8
I20260709 13:03:49.101946  8750 pending_rounds.cc:107] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:03:49.102054  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:49.103101  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:49.103561  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:49.102262  8750 raft_consensus.cc:2889] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:49.104527  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:49.105474  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.107227  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:49.111330 13040 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:49.097838  8751 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:49.112911  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:49.113857  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:49.114970  8823 raft_consensus.cc:2393] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:49.116946  8753 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:49.112324 13057 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:49.113109  8576 catalog_manager.cc:5697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:49.124236  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:49.120292  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:49.123204  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:49.125665  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.131563  8827 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:49.127890  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.131400  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.131345  8576 catalog_manager.cc:5697] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 7 to 8, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 8 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:49.097198 13042 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.168651 13042 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:49.137669  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:49.139180  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.141099  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.112993 12943 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:49.173344  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:49.134406  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:49.175158  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.126348  8751 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:49.115087 12979 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:49.114845 13148 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:49.118435 13050 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:49.112804 13101 raft_consensus.cc:2764] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:03:49.161890  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:49.162837  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68346.3us
I20260709 13:03:49.165160  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.164911  8576 catalog_manager.cc:5697] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:49.202112  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:49.202220  8576 catalog_manager.cc:5697] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:49.203161  8722 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:49.085455 13062 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.119469 13143 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:49.175803  8719 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:49.178560  8751 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:49.179296  9382 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:49.015429 12977 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:49.184856  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:49.208981  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 92355.3us
I20260709 13:03:49.188580  8750 raft_consensus.cc:2410] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:49.197425  8755 raft_consensus.cc:2376] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8. Re-sending same reply.
I20260709 13:03:49.209877  8647 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:49.170667 13057 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:49.122335 13076 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:49.210950  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:49.183002 12945 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:49.211751  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:49.212543  8753 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:03:49.217608  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.211530  8719 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:49.191505 13156 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:03:49.221685  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.208513 13135 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.212168 12944 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:49.222467 13135 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.208118 12992 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:49.224582  8576 catalog_manager.cc:5697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 3 to 10. New cstate: current_term: 10 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:49.218940  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:49.206537 13050 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:49.214145 13122 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:03:49.232594  8579 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43462 (request call id 267) took 40756 ms (client timeout 15000 ms). Trace:
I20260709 13:03:49.237732  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.229984  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:49.239374  8751 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:49.240384  8753 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:49.219008 13146 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:49.218722 13040 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.219142 12943 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:49.245704 13076 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:49.249478  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.255005  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:49.255836  8777 debug-util.cc:398] Leaking SignalData structure 0x7b08001ddc40 after lost signal to thread 8613
I20260709 13:03:49.261180  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:49.209697 13062 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.221297 12945 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.240698 13156 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:49.262779  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:49.317162  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 54470.3us
I20260709 13:03:49.213852 12657 consensus_queue.cc:1059] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:03:49.307628  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.322101  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.323405  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.224551 13148 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:03:49.268638 13146 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:49.309283  9382 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:49.307618  8753 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:49.312114  8576 catalog_manager.cc:5697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:03:49.326414  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 114959us
I20260709 13:03:49.257426 13135 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:49.327898  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.235262 13057 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 13:03:49.320565 12992 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:49.356488 12992 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.346554  9382 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:03:49.318658 12943 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:49.360951  8647 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:49.324940 13076 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:49.371464  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.372314  8753 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:49.314538 13062 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:49.327945 13090 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.326959 12944 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:49.377300  8576 catalog_manager.cc:5697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:49.336165 13146 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:49.307557 13040 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:49.384135 13076 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:49.389631  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.388931 12944 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.359519 12974 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.389048 13076 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:49.361522 13057 consensus_queue.cc:1243] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:03:49.390825  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 173413us
I20260709 13:03:49.391757  8751 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:49.397642  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:49.393268  8576 catalog_manager.cc:5697] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:49.395399  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.400401  8576 catalog_manager.cc:5697] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 5 to 7, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 7 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:49.395287  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.404322  8753 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
W20260709 13:03:49.405165  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 67374.5us
I20260709 13:03:49.368257 13122 raft_consensus.cc:697] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:49.408855  8824 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:49.413702  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:49.387166 12943 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:49.413758  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:49.410251  8576 catalog_manager.cc:5697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:49.235950  8579 rpcz_store.cc:269] 0709 13:03:08.427424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.431039 (+  3615us) service_pool.cc:224] Handling call
0709 13:03:49.170550 (+40739511us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:36.761832 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:36.762476 (+   644us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:36.762532 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:03:36.762551 (+    19us) tablet.cc:662] Decoding operations
0709 13:03:36.763846 (+  1295us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:36.763957 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:36.763987 (+    30us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:03:36.765290 (+  1303us) tablet.cc:701] Row locks acquired
0709 13:03:36.765341 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:03:36.765538 (+   197us) write_op.cc:270] Start()
0709 13:03:36.765684 (+   146us) write_op.cc:276] Timestamp: P: 1783602216765460 usec, L: 0
0709 13:03:36.765707 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:03:36.863525 (+ 97818us) log.cc:844] Serialized 5139 byte log entry
0709 13:03:38.434237 (+1570712us) op_driver.cc:464] REPLICATION: finished
0709 13:03:38.981066 (+546829us) write_op.cc:301] APPLY: starting
0709 13:03:39.067437 (+ 86371us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:40.017200 (+949763us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:40.017346 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:40.161799 (+144453us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:40.162526 (+   727us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 13:03:40.162639 (+   113us) write_op.cc:312] APPLY: finished
0709 13:03:40.163432 (+   793us) log.cc:844] Serialized 121 byte log entry
0709 13:03:40.247795 (+ 84363us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:40.275372 (+ 27577us) write_op.cc:454] Released schema lock
0709 13:03:40.275484 (+   112us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:40.325857 (+ 50373us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:40.325906 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10122339,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":396427,"apply.run_cpu_time_us":75578,"apply.run_wall_time_us":1370691,"cfile_cache_hit":50,"cfile_cache_hit_bytes":52200,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6215,"cfile_init":1,"lbm_read_time_us":104275,"lbm_reads_10-100_ms":5,"lbm_reads_lt_1ms":3,"num_ops":13,"prepare.queue_time_us":243821,"prepare.run_cpu_time_us":25294,"prepare.run_wall_time_us":439112,"raft.queue_time_us":669647,"raft.run_cpu_time_us":23205,"raft.run_wall_time_us":436182,"replication_time_us":1581007,"thread_start_us":846865,"threads_started":5,"wal-append.queue_time_us":791695,"wal-append.run_cpu_time_us":54243,"wal-append.run_wall_time_us":2244565}]]}
I20260709 13:03:49.415313  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:49.427471  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:49.415477  8823 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:49.415529  8824 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:49.429126  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.415884  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:49.431744  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:49.415686  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:49.432950  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:49.433447  8826 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:49.429751  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:49.435045  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:49.436685  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.438202  8753 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:49.440207  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.440963  8672 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:49.396351 13161 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:49.403579 13092 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:49.443123  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.443455  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.449312  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.451671  8677 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:49.452384  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:49.446799  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:49.447844  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.453852  8679 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:49.448535  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.449090  8678 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:49.455096  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.449306  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.455914  8679 raft_consensus.cc:2410] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:03:49.455991  8675 raft_consensus.cc:2410] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:49.445392  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:49.433452  8822 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:49.456636  8823 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:49.439249  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.458480  8753 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:49.457901  8576 catalog_manager.cc:5697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:49.402770 12974 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.415993 13156 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.464124  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.463110 13122 consensus_queue.cc:237] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.469105 13092 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.481798  8794 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:49.414198 13165 raft_consensus.cc:3055] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:49.387645 13090 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.378572 13050 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.496573  9387 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:49.493978  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:49.506100  8722 connection.cc:256] RPC call timeout handler was delayed by -2.0968374120000135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.205s for a response
I20260709 13:03:49.514351  8827 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:49.515075  8722 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:49.507172  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:49.495198 13042 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:49.509104 13134 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:49.521030  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:49.511518 13155 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 13:03:49.459390 13143 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:03:49.517843  8794 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:49.511085 12974 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:49.508673 13154 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.241s	user 0.002s	sys 0.000s
I20260709 13:03:49.528265  8794 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:49.502156 13165 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:03:49.539134  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:49.540026  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.543815 12630 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:49.544104 13116 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:49.544481 12630 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.544955 12630 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:49.519886 12749 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:49.496636 12881 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:03:49.519443 12984 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:49.558683  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 1990) took 1174 ms. Trace:
I20260709 13:03:49.559309  8829 rpcz_store.cc:276] 0709 13:03:48.385758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.385884 (+   126us) service_pool.cc:224] Handling call
0709 13:03:48.386172 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.386542 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.386571 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:48.387121 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:03:48.961400 (+574279us) spinlock_profiling.cc:243] Waited 6.01 ms on lock 0x7b18003d5e80. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:03:49.134979 (+173579us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:49.135017 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.135160 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.549281 (+414121us) raft_consensus.cc:1685] finished
0709 13:03:49.549310 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.558546 (+  9236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":117,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":4329,"spinlock_wait_cycles":13812992,"thread_start_us":173338,"threads_started":1,"wal-append.queue_time_us":821518}
I20260709 13:03:49.529663 12640 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:49.568173 13170 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:49.568989 13170 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.552476 13134 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:49.571868 13042 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:49.572506 13042 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:49.548964 13168 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:49.573259 13168 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.495687 13012 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:03:49.574203  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:49.571127  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 56150.6us
I20260709 13:03:49.554284  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:49.554724  8794 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:49.548938  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 781) took 3206 ms. Trace:
I20260709 13:03:49.578107  8828 rpcz_store.cc:276] 0709 13:03:46.342667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.344874 (+  2207us) service_pool.cc:224] Handling call
0709 13:03:46.345157 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.457895 (+112738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.548639 (+3090744us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540009c9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:49.548862 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257763712}
W20260709 13:03:49.616922  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50377us
I20260709 13:03:49.618402  8719 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:49.618943  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 64351.7us
I20260709 13:03:49.619880  8794 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:49.620805  8826 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:49.503784  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.562899 13166 raft_consensus.cc:993] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: : Instructing follower 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:03:49.577486  8681 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:03:49.627882  8794 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:49.629655  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.631757  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:49.626183 13061 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:49.659906  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 166066us
I20260709 13:03:49.653723 13170 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:49.650408 13169 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.651852  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2116) took 1330 ms. Trace:
W20260709 13:03:49.664767  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.663633  8754 rpcz_store.cc:276] 0709 13:03:48.321906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.376549 (+ 54643us) service_pool.cc:224] Handling call
0709 13:03:48.376884 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:49.020294 (+643410us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:49.208822 (+188528us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.385571 (+176749us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.392469 (+  6898us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:49.403115 (+ 10646us) log.cc:844] Serialized 26 byte log entry
0709 13:03:49.403305 (+   190us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:49.403339 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.403426 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.651558 (+248132us) raft_consensus.cc:1685] finished
0709 13:03:49.651585 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.651772 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":768505,"prepare.queue_time_us":223947,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":6497,"raft.queue_time_us":761741,"raft.run_cpu_time_us":424,"raft.run_wall_time_us":33080,"thread_start_us":350598,"threads_started":3,"wal-append.queue_time_us":194307}
I20260709 13:03:49.626798 13167 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:49.661464  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:49.626952 12657 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:49.548496 12943 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.573606 13042 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:49.657155  8751 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:49.627859 13134 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:49.624706 13057 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:49.633764 12640 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:49.681005  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.704954  9353 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:49.721338  9353 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:49.674459  8794 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:49.662943 13090 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:49.747025  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.747979  8751 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
W20260709 13:03:49.722774  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 48419.4us
I20260709 13:03:49.676378 12657 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:49.676667 12776 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.666s	user 0.015s	sys 0.003s
I20260709 13:03:49.675590 13167 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.679292 13135 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:49.751971  8794 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:49.714319 13076 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:49.698777 13134 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:49.670369 13061 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.744344 12984 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:49.773109  8749 raft_consensus.cc:2410] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:03:49.696271 13161 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:49.772992  8777 debug-util.cc:398] Leaking SignalData structure 0x7b08000cd080 after lost signal to thread 8705
W20260709 13:03:49.786706  8777 debug-util.cc:398] Leaking SignalData structure 0x7b08001c29a0 after lost signal to thread 8706
I20260709 13:03:49.781351 12984 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:49.659111 13165 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:49.784061  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 208586us
W20260709 13:03:49.769953 13095 consensus_meta.cc:287] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.657s	user 0.003s	sys 0.010s
W20260709 13:03:49.788602  8777 debug-util.cc:398] Leaking SignalData structure 0x7b0800377ba0 after lost signal to thread 8780
I20260709 13:03:49.768412 12944 raft_consensus.cc:1081] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Signalling peer 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:03:49.768918 12657 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:49.783708  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.783316 13134 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:49.769030 12977 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:49.779399  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:49.773389 13167 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:49.810410 12630 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:49.810806 12977 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:49.813352 12977 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:49.675225 13169 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:49.816735 13169 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:49.792268  8777 debug-util.cc:398] Leaking SignalData structure 0x7b080039bb40 after lost signal to thread 8781
W20260709 13:03:49.820114  8777 debug-util.cc:398] Leaking SignalData structure 0x7b08003d4780 after lost signal to thread 8854
W20260709 13:03:49.820725  8777 debug-util.cc:398] Leaking SignalData structure 0x7b08003e8ce0 after lost signal to thread 8855
I20260709 13:03:49.789937  8722 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:49.772742 13135 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:49.742484 13057 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:49.796705 13145 consensus_meta.cc:287] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.795s	user 0.017s	sys 0.000s
I20260709 13:03:49.794762 13012 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:49.791836 12984 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:49.780628 12776 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:03:49.789531 12640 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:49.802662 13177 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:49.780435 13076 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:49.806389 13061 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:49.827366 12749 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:49.807761 13168 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:49.820005 12630 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:49.770766 12979 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:49.792819 13164 consensus_queue.cc:1059] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:03:49.855180 12640 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:49.846114  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 215543us
I20260709 13:03:49.851708  8722 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:49.889030  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.851768  8825 raft_consensus.cc:2410] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:49.789919 13176 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.891927  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:49.867079  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 123804us
I20260709 13:03:49.846170 13090 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:49.824970 13135 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:49.836357 13169 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:49.871852 12962 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.826253 13180 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:49.890041  8675 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:49.893381  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:49.893548  8826 raft_consensus.cc:1217] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[7.4-11.6]   Dedup: 7.4->[9.5-11.6]
I20260709 13:03:49.915679  8826 pending_rounds.cc:85] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 4
I20260709 13:03:49.916020  8826 pending_rounds.cc:107] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:03:49.916323  8826 raft_consensus.cc:2889] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:49.894433  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:49.895183  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.896037  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:49.865617 12977 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.868738 12749 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.905720  8830 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:03:49.906541  8680 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:53352
I20260709 13:03:49.909780  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:49.924180  8829 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:03:49.862071 12944 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:03:49.927506 12944 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:49.927889 12944 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:49.928035 13076 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:49.928774 12944 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:49.901453 12640 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:49.902989 13095 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:49.938997  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:49.939491  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.940508  8719 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:49.933064  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.932830  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.936060  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.943307  8675 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:49.930060 12944 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:49.943868  8679 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
W20260709 13:03:49.946928  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.946895  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:49.947603  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.947683  8679 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
W20260709 13:03:49.948067  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.881129 12979 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:49.950829  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 54928.6us
I20260709 13:03:49.951687  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:49.952910  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:49.960666  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:49.961460  8752 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:49.930578 12749 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:49.913435 13156 consensus_queue.cc:1059] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:03:49.977669  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:49.974694  8752 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:49.920806 13168 consensus_queue.cc:1059] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:03:49.972496  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:49.980232 13171 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:03:49.979449  8679 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:49.983618  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:49.983870  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:49.984330  8675 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:03:49.948706 13190 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:49.991963 12992 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:49.930359 12977 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.981276 13095 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:49.909368 13176 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:49.988579 13181 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:49.879865 13161 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.009473 13195 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:50.009663  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:50.010453  8673 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:03:50.010165 13195 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.012279  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:50.012755 13195 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:50.020337 13178 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:03:49.922399 12962 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.001170 12992 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:50.028411 12992 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.029088 12992 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:50.029683 12962 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:49.982182 13169 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:50.035195 12945 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.037683 13167 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.044024 12979 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:50.047569 13057 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.049180 13026 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:49.972828 13146 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.568s	user 0.013s	sys 0.000s
I20260709 13:03:50.051566 13146 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.056144  8672 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:50.056759  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:50.007196  8752 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:50.062119  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:50.057986  8675 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:03:50.063741  8677 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:03:50.008148  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 44289.7us
W20260709 13:03:50.068157  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 64261.1us
I20260709 13:03:50.069725  8722 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:50.078368  8679 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:03:50.086123  8722 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:50.091291  8722 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:50.099505  8675 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:03:50.101706 12945 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:50.102273 12945 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.102672 12945 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:50.114960 13146 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:50.126935  8777 debug-util.cc:398] Leaking SignalData structure 0x7b08003ed220 after lost signal to thread 9413
W20260709 13:03:50.128470  8777 debug-util.cc:398] Leaking SignalData structure 0x7b0800330720 after lost signal to thread 13065
W20260709 13:03:50.129252  8777 debug-util.cc:398] Leaking SignalData structure 0x7b08003d5cc0 after lost signal to thread 13083
W20260709 13:03:50.129720  8777 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6e40 after lost signal to thread 13089
I20260709 13:03:49.999809 13190 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.136881  8680 raft_consensus.cc:476] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Not starting forced leader election -- already a leader
I20260709 13:03:50.134925  9352 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:50.119697 13095 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:50.119894 13169 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:50.128347 12979 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:50.130134 13122 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:50.145656  9382 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:50.084373 12657 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.082386 13182 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:03:50.155014  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:50.158881  9352 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:50.164388 13024 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:50.155462 13161 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.165691 13161 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.155479 12945 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.168195 13161 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.168884 13190 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.136164 13187 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:50.169785 13187 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.172190 13187 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:50.172659 12657 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:50.173385 13026 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:50.173853 13026 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:50.174266 13026 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:50.166103 13189 raft_consensus.cc:2764] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:03:50.137427 13197 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:50.177611  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:50.178305 13197 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:50.183741 13157 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.856s	user 0.007s	sys 0.000s
I20260709 13:03:50.160812 12979 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.187631  8681 raft_consensus.cc:1275] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:50.188858 12979 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.190434  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:50.191713  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:50.192330 13168 consensus_queue.cc:1048] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:50.195602  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:50.165030 13024 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:50.199676 12984 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.201215 13176 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.202258 12992 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.202699 12984 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:50.228458  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:50.228477  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:50.229334  9381 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:03:50.202867 13167 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.205083  9384 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:50.205838  8679 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:50.208781  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:50.238265  8677 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:50.210716  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:50.240506  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:50.211970  8830 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:50.243088  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2115) took 1921 ms. Trace:
I20260709 13:03:50.243548  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2495) took 1160 ms. Trace:
I20260709 13:03:50.243772  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:50.243497  8748 rpcz_store.cc:276] 0709 13:03:48.321617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.370818 (+ 49201us) service_pool.cc:224] Handling call
0709 13:03:48.371191 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:49.018896 (+647705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.018935 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:49.019570 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 13:03:49.175063 (+155493us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:49.175109 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.176022 (+   913us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:50.183046 (+1007024us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:50.220185 (+ 37139us) raft_consensus.cc:1685] finished
0709 13:03:50.220209 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.243021 (+ 22812us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":647256,"raft.queue_time_us":650097,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":84,"spinlock_wait_cycles":50138368,"thread_start_us":155036,"threads_started":1,"wal-append.queue_time_us":214457}
I20260709 13:03:50.217438  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:50.217770  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:50.215035  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:50.245708  9388 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:50.246251  8828 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:50.232312  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 73515.9us
I20260709 13:03:50.233954  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2014) took 1330 ms. Trace:
I20260709 13:03:50.186025 12640 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.219298  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:50.247159  9352 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:50.242378  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:50.247535  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:50.246759  8821 rpcz_store.cc:276] 0709 13:03:48.903356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.903541 (+   185us) service_pool.cc:224] Handling call
0709 13:03:48.914378 (+ 10837us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:48.957019 (+ 42641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.957045 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:48.958779 (+  1734us) log.cc:844] Serialized 52 byte log entry
0709 13:03:50.043685 (+1084906us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b18002c8600. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:03:50.136362 (+ 92677us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:50.210753 (+ 74391us) log.cc:844] Serialized 14 byte log entry
0709 13:03:50.211081 (+   328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.211250 (+   169us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:50.211326 (+    76us) raft_consensus.cc:1685] finished
0709 13:03:50.233713 (+ 22387us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.233890 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66878,"raft.queue_time_us":58233,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":27177,"spinlock_wait_cycles":13576576,"thread_start_us":154254,"threads_started":2,"wal-append.queue_time_us":1180778}
I20260709 13:03:50.249212  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:50.249604  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:50.249840  8679 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
W20260709 13:03:50.250521  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 169575us
I20260709 13:03:50.253940  8750 rpcz_store.cc:276] 0709 13:03:49.082817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.218528 (+135711us) service_pool.cc:224] Handling call
0709 13:03:49.218866 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.312982 (+ 94116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.313032 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:49.313104 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:49.313156 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.243273 (+930117us) spinlock_profiling.cc:243] Waited 93.9 ms on lock 0x7b54002abf54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:50.243335 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.243482 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215888128}
W20260709 13:03:50.255085  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.254966  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:50.255666  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.256774  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:50.255223  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:50.245167  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:50.245893  8824 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:03:50.260874  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:50.271581  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:50.272997  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.271581  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:50.274073  8722 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:50.157205 13169 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.227008 12974 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:50.288405  9386 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:50.181675 13197 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:50.288244 13184 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:03:50.297817  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:50.302269  8825 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:50.305675  8751 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:50.300702  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:50.304179  9387 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:50.274266  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:50.272590 13012 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:50.316150  8793 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:50.321376  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:50.323956  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:50.326205  8821 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:50.307935  8796 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:50.244916 13187 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:03:50.301106 13167 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:50.308785 13134 consensus_meta.cc:287] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.003s
I20260709 13:03:50.343066 12977 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.276274 13024 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.339747 13026 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.340003 12796 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:50.317600 13189 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:50.348606 12749 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.362450 12640 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:50.363376 12749 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:50.340518 13012 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:50.357723 13189 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.365329 13189 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:50.315169 13100 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:03:50.376655 13122 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:50.377403 13122 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:50.378710 13026 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.353648 13134 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.383069  9385 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:50.358354 12796 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.356380 13204 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:50.391372 12945 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.393162 13134 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:50.393333 12796 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:50.387039 13135 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.397627 13135 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:50.394018 13090 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:50.365381 13012 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.299789  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:50.459031  8822 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:50.464556  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2513) took 1152 ms. Trace:
I20260709 13:03:50.468732  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:50.474982 12976 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:03:50.476058  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.477061 12977 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:03:50.488998  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 151027us
I20260709 13:03:50.489269 12977 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:50.492455  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 192581us
W20260709 13:03:50.493348 13218 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:03:50.493798 12977 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.480345  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:50.455816  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:50.498986  8749 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:50.506345  8674 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:50.506677  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 53249.6us
I20260709 13:03:50.514436  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:50.480973 13057 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:50.499943 13050 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:50.511185 13101 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
W20260709 13:03:50.524966  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.525853  8679 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:50.528353 13134 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:03:50.533007 13204 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:50.536484  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:50.537560  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:50.537825  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1840) took 1130 ms. Trace:
I20260709 13:03:50.538353  8827 rpcz_store.cc:276] 0709 13:03:49.407706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.407872 (+   166us) service_pool.cc:224] Handling call
0709 13:03:49.417104 (+  9232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.514976 (+ 97872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.537179 (+1022203us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b54003acad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:50.537589 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222974720}
I20260709 13:03:50.543166  9382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 682) took 1130 ms. Trace:
I20260709 13:03:50.543812  9382 rpcz_store.cc:276] 0709 13:03:49.412990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.413388 (+   398us) service_pool.cc:224] Handling call
0709 13:03:50.542628 (+1129240us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b54003a6454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:50.542869 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1597886848}
I20260709 13:03:50.547991 13187 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:50.548784 13187 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:50.549441 13187 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:50.556931 12776 raft_consensus.cc:3055] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:50.557690 12776 raft_consensus.cc:740] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:03:50.557745  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 47419.3us
I20260709 13:03:50.563395  8645 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:50.564954 13024 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.567413 12976 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.568492 12992 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.571209  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:50.572379  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:50.573370  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:50.574183 12974 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:50.574656 12974 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.575042 12974 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:50.577580 13040 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:50.482923  8754 rpcz_store.cc:276] 0709 13:03:49.311698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.720630 (+408932us) service_pool.cc:224] Handling call
0709 13:03:49.768667 (+ 48037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.768919 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:49.787673 (+ 18754us) log.cc:844] Serialized 14 byte log entry
0709 13:03:50.053465 (+265792us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b1800471820. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:03:50.463405 (+409940us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:50.463584 (+   179us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:50.463701 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.463846 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.464212 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15947,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":164216,"raft.run_cpu_time_us":448,"raft.run_wall_time_us":449,"spinlock_wait_cycles":5520640,"thread_start_us":253397,"threads_started":1,"wal-append.queue_time_us":572529}
W20260709 13:03:50.586021 13160 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.262s	user 0.010s	sys 0.000s
W20260709 13:03:50.588109 13220 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:03:50.588987  8722 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:50.565857 13024 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.601053  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2118) took 2278 ms. Trace:
I20260709 13:03:50.601778  8756 rpcz_store.cc:276] 0709 13:03:48.322524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.378195 (+ 55671us) service_pool.cc:224] Handling call
0709 13:03:48.392072 (+ 13877us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:49.023045 (+630973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.023080 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:49.023617 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:03:49.209059 (+185442us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:49.209108 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.209199 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:50.584756 (+1375557us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:50.600339 (+ 15583us) raft_consensus.cc:1685] finished
0709 13:03:50.600370 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.600781 (+   411us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":630539,"raft.queue_time_us":632103,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"spinlock_wait_cycles":50854656,"thread_start_us":185044,"threads_started":1,"wal-append.queue_time_us":197655}
I20260709 13:03:50.604720  9384 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:50.606110 13092 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.607065  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:50.530076  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:50.611781  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.612345 13204 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:50.612691  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50633.3us
I20260709 13:03:50.613322 13204 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.615000 13090 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.615962 13090 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.570262 12976 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.583509 12776 consensus_queue.cc:260] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.592981 13141 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:50.645318  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 65678.8us
W20260709 13:03:50.666949  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 103803us
I20260709 13:03:50.668694  8645 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:50.667177 13161 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:50.668808 13165 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:50.676908  8674 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:50.656653 13040 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:50.656883 13101 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:50.688719  8748 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:03:50.689772  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:50.694922 12976 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:50.698603  8755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 719) took 1624 ms (client timeout 1732 ms). Trace:
I20260709 13:03:50.703428 13224 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:03:50.674449 13161 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:50.707877  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:50.719231  8827 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:03:50.703441  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:50.719650  8827 pending_rounds.cc:85] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 6
I20260709 13:03:50.720047  8827 pending_rounds.cc:107] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:03:50.720314  8827 raft_consensus.cc:2889] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:50.721230 12776 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:50.721796 13121 consensus_queue.cc:1059] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:03:50.723049  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:50.712986 13090 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:50.726961 13141 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.728760  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:50.729385  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 53218us
I20260709 13:03:50.729722  8826 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:50.730086 13141 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:50.737465  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2004) took 1115 ms. Trace:
I20260709 13:03:50.739784 13175 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:50.740353 13175 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:50.740739 13175 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:50.742882  8825 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:50.706372 13169 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:50.714118  8755 rpcz_store.cc:269] 0709 13:03:49.073628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.106912 (+ 33284us) service_pool.cc:224] Handling call
0709 13:03:50.698188 (+1591276us) spinlock_profiling.cc:243] Waited 89.2 ms on lock 0x7b5400102a54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:50.698376 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205242624}
I20260709 13:03:50.738003  8672 rpcz_store.cc:276] 0709 13:03:49.621929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.655883 (+ 33954us) service_pool.cc:224] Handling call
0709 13:03:49.656383 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.056889 (+400506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.736823 (+679934us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b54002409d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:50.737215 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":918811520}
I20260709 13:03:50.751375  8679 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
W20260709 13:03:50.753086  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 164351us
I20260709 13:03:50.753738 13197 consensus_queue.cc:1059] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:50.755048 13197 raft_consensus.cc:3055] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:50.755561  8830 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:03:50.755532 13197 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:50.757015 13197 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.718376 13161 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:50.758373  8793 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.651s (SENT) [suppressed 16 similar messages]
W20260709 13:03:50.760669  8677 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.014s
W20260709 13:03:50.761224  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.761691 13050 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:50.762460 13050 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.765488 13050 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:50.768472  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 78969.2us
I20260709 13:03:50.769731  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:50.770888 13101 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:50.771385 13101 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:50.771824 13101 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:50.772373  9352 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:50.773700 13202 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:50.774297 13202 raft_consensus.cc:697] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:50.775399  8673 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:50.775998 13204 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:50.776368  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:50.778564 13202 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.782881 13121 consensus_queue.cc:1243] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:50.785933 12984 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:50.788089 13024 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.788515 13100 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.789307 13100 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.790122 12984 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.792840 12984 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:50.761168  8677 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:50.793267  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:50.794245  9384 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:50.797537 12974 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.799676 13100 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:50.801265  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 61680.6us
I20260709 13:03:50.758775 13197 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:50.754417  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:50.831523  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:50.830542  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:50.829581  8796 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:50.839917  8673 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:03:50.843714  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 74164.9us
I20260709 13:03:50.855646  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:50.839136  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:50.835141 12976 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.867973 13040 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.872455  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:50.873799 13145 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:03:50.875051  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:50.876055  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:50.876706  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:50.876765 12763 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:03:50.880110  8645 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:50.868539 13169 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:50.876094  8755 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:50.851655 12974 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:50.905345  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:50.882892 13189 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:50.921092  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 82133.7us
I20260709 13:03:50.924948  8749 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:50.925100  8753 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:03:50.925704 13213 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:50.926246 13213 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.926288 13189 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.926769 13213 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:50.927340 13196 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.928879  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:50.871206 13187 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.905998 13195 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:03:50.928769  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 174614us
I20260709 13:03:50.939810  9389 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:50.891258 13040 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.936048 13061 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:50.941154  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:50.947471  8825 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
W20260709 13:03:50.965855  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 85915.3us
I20260709 13:03:50.967132  8645 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:50.967285 13143 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:50.960188 13100 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:50.968354 13143 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:50.969326  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.970908  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:50.971704  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:50.917675 12976 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.975448 12976 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:50.976859  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:50.977723  8827 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:50.928217 13196 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:50.978820 12945 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:50.980424 13184 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:50.980970 13184 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:50.982244 13184 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.983875 13184 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:50.986867  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 169140us
I20260709 13:03:50.988039 13191 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:50.988691 13076 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:50.988842 13191 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:50.989226 13076 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:50.996043 13234 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.138s	user 0.003s	sys 0.000s
I20260709 13:03:50.968595 13100 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:50.940995 13195 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:50.950397 12640 raft_consensus.cc:3055] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:03:51.051755  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 84869.2us
I20260709 13:03:51.071372  8681 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: []
W20260709 13:03:51.072404  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 85194us
I20260709 13:03:51.073475 13122 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.074275 13187 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:51.071979 13101 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:51.087654  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 110283us
I20260709 13:03:51.102854  8794 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:50.944862 12796 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.079746 13195 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:51.079994 12640 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:51.119302  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 45930.5us
W20260709 13:03:51.142089  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 88334.1us
I20260709 13:03:51.091569 13121 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:51.101456 13024 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.139717  8645 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:50.990254 13191 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:51.166091  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:51.130548 13175 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.091459 13161 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.177848  8751 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:50.990397 13076 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.090981 13061 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:51.177394  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:51.191653  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:51.228382  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:51.230024  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 75863.9us
W20260709 13:03:51.232411  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 67987.8us
I20260709 13:03:51.234177  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:51.210757 13024 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:51.217578 12881 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.217975 13187 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:51.244133  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 133638us
I20260709 13:03:51.240765  8749 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:51.250401  9387 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:51.244453  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:51.255368  8752 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:03:51.250921  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:51.250742  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:51.211874 13224 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:51.271662  8680 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:51.271850  8748 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:51.277453  8752 pending_rounds.cc:85] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 5
I20260709 13:03:51.269563  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:51.288332  8752 pending_rounds.cc:107] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:03:51.288758  8752 raft_consensus.cc:2889] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:51.263433 13238 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.328s	user 0.001s	sys 0.001s
I20260709 13:03:51.272214  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:51.293804  8826 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.003s
I20260709 13:03:51.164664 13169 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.112s
I20260709 13:03:51.273741 13076 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:51.298472  8826 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:03:51.296428  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:51.302186  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:51.246469 13024 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:51.219413 13121 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:51.305799 13100 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:51.306288 13040 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:51.257527 13122 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:51.256503 12976 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.255504 13241 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:51.312506  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 349791us
I20260709 13:03:51.316159  8673 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:51.312311  8754 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:51.320886  8674 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:51.248509 13182 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:03:51.290643 13165 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:03:51.344274  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:51.248543 12881 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:51.356065  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 59466.4us
I20260709 13:03:51.357034  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:51.358088 12881 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:51.267491 12640 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:51.359928 12640 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:51.362725 13176 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:03:51.364846 13176 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:51.367977 13061 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:51.370908 13042 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:51.317881 13050 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.372799 13050 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:51.379825  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.382125 13121 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:51.385457 12974 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:51.386091 12974 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:51.386137 12776 consensus_meta.cc:287] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.664s	user 0.011s	sys 0.001s
I20260709 13:03:51.341773 13101 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:51.349781 13143 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:51.413774  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:51.351140  8826 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:51.352088 13175 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:51.352421  8677 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:51.352881 12796 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:51.353478 12976 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.354385  8749 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:51.355115  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:51.355453  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:51.386726  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.421295  8754 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:03:51.388232 13161 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:51.349150 12943 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.215s
I20260709 13:03:51.387291 12974 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:51.350309  8827 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:51.358601  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:51.365356 13176 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:51.443362 13176 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:51.420512  8749 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:03:51.429883  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 239020us
I20260709 13:03:51.402670 13042 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:51.447201  9353 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:51.433486  8821 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:51.437075 12976 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:51.439247 12974 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.440656 13185 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:51.449831  8749 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:51.415462  8748 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:51.458240  8673 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:03:51.416208 12776 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:51.416872 13122 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:51.472749  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.480175  8677 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:51.486799  9353 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:51.492661  8722 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:51.500836 13248 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:03:51.502713 13239 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.355s	user 0.003s	sys 0.000s
W20260709 13:03:51.507129  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 101859us
I20260709 13:03:51.508622  8749 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:51.508309 13042 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:51.511160 13197 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.700s	user 0.012s	sys 0.003s
I20260709 13:03:51.475067 13076 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:51.521522  8679 raft_consensus.cc:2410] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:03:51.522301  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 163800us
I20260709 13:03:51.473169 12776 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:51.498236 12881 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:51.545141  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.523463 13185 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:51.547355  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:51.552510  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2512) took 2234 ms. Trace:
I20260709 13:03:51.539734 12881 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:51.562738 12881 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:51.564438 13202 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:51.565062 13202 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:51.562773  8825 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:03:51.571303  8825 pending_rounds.cc:85] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 8
I20260709 13:03:51.571628  8825 pending_rounds.cc:107] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:03:51.571888  8825 raft_consensus.cc:2889] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:51.553191  8753 rpcz_store.cc:276] 0709 13:03:49.308147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.460154 (+152007us) service_pool.cc:224] Handling call
0709 13:03:49.460590 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:50.925767 (+1465177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.535136 (+609369us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b540031bd54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:51.542728 (+  7592us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3344548736}
W20260709 13:03:51.537006 13213 consensus_meta.cc:287] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.005s
I20260709 13:03:51.517045 13122 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.542086 13076 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:03:51.593952  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 124076us
I20260709 13:03:51.595384 12943 raft_consensus.cc:3055] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:51.596027 12943 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:51.597119 12943 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.598605  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:51.599687  8828 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:51.601084  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:51.601925  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:51.602360  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:51.603005  8821 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
W20260709 13:03:51.603964  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 62885.1us
I20260709 13:03:51.608788  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:51.609246  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:51.609622  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:51.609289  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:51.610977  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:51.611778  8756 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:51.609633  8751 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:51.555058 13185 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:51.610450  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:51.571743 13121 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.619187  8826 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:03:51.570786 13062 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.623720 13195 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:51.625638 13255 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:51.627295  8823 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:51.629359  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2531) took 1801 ms. Trace:
I20260709 13:03:51.629828  8752 rpcz_store.cc:276] 0709 13:03:49.827867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.088167 (+260300us) service_pool.cc:224] Handling call
0709 13:03:50.088621 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.307610 (+1218989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.629048 (+321438us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540018d8d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:51.629288 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2595657088}
I20260709 13:03:51.631551  8672 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:51.634742  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.637501  8755 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:51.631201 13195 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:51.598497 12943 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:51.648289  8751 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:51.649210  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:51.652398 13253 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:51.653149 13253 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.654929  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:51.656904  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:51.658715  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:51.660106 13122 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:51.661005  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:51.661383 13230 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.897s	user 0.007s	sys 0.000s
I20260709 13:03:51.622500 13062 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:51.645619 13121 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.669128  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:51.670351  9389 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:51.669237 13121 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:51.672884 13196 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:51.672726 13076 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.618296 13185 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:51.675325  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:51.679191  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:51.683802  8823 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:51.683943  8829 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:51.566362 13202 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:51.686106  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:51.588222 13213 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:51.651786 13187 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:51.684659 12881 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:51.681734 13143 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:51.686964  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 200343us
W20260709 13:03:51.707988  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 199878us
I20260709 13:03:51.687486  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:51.685334  8749 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:03:51.692878  8673 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:51.710186  8749 pending_rounds.cc:85] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 10
I20260709 13:03:51.710886  8749 pending_rounds.cc:107] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:03:51.711270  8749 raft_consensus.cc:2889] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:51.704958  8827 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:51.707221  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:51.690667  8722 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:51.694968  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:51.699062  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:51.687261  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:51.699388  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:51.716761  8672 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:03:51.717736  8675 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:51.726081  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:51.727455 13121 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:51.728204 13121 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.688552 13122 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:51.731330 13121 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:51.731163 13122 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.733930 13122 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:51.738245  9382 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:51.720381  8676 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:51.706040  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:51.695947 13253 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:51.749701  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:51.689013 13062 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:51.755489 13187 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:51.756963 13187 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:51.757362 13187 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:51.698702 12881 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:51.696949 13213 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:51.765017  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.681449 12776 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:51.780994  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 71912.7us
I20260709 13:03:51.723873 13176 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:51.793892  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 107462us
I20260709 13:03:51.795562  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 774) took 1173 ms. Trace:
I20260709 13:03:51.795994  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:51.791056  8826 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:51.758381 13187 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:51.816037  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.796058  8681 rpcz_store.cc:276] 0709 13:03:50.621905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.664225 (+ 42320us) service_pool.cc:224] Handling call
0709 13:03:50.664668 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.072036 (+407368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.791528 (+719492us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b5400353c54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:51.795482 (+  3954us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":682294528}
W20260709 13:03:51.818396  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.825246  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:51.826771  8830 raft_consensus.cc:2410] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:51.876924  8674 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:51.779047 13169 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:51.886305  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:51.896095  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 129380us
I20260709 13:03:51.897080  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:51.896435 12974 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:51.874837 13213 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:51.902406 12974 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.903182 13213 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.905666 13213 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:51.907084 12974 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:51.908179 13176 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:51.875916 13040 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:51.909255 13040 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:51.910396 13040 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.912745 13012 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.207s
I20260709 13:03:51.915617 13187 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:51.923940 13252 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:51.924283  8719 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:51.924486 13062 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:03:51.925161 13260 log.cc:927] Time spent T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 13:03:51.922287  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 150019us
I20260709 13:03:51.925815  8719 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:51.926193 13062 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:51.926643  8645 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:51.926842 13042 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:51.927539 13176 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:51.928022 13176 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:51.928562  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:51.925920 13076 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.930015 13057 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:51.929773 13169 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:51.930449 13057 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.930473 13176 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:51.930862 13062 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:03:51.930853  8793 leader_election.cc:336] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.531s (SENT)
W20260709 13:03:51.934998  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 171824us
W20260709 13:03:51.995056  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.096532  8793 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:51.942251  8645 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:52.099334  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.930780 12776 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:51.941684  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 145763us
I20260709 13:03:52.097326  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:52.105778  8722 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:52.107175  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:52.110684 13167 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.962s	user 0.024s	sys 0.019s
W20260709 13:03:52.114351 13167 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.025s	sys 0.019s
I20260709 13:03:52.122293 12945 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:52.003713 13057 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:52.105051 13042 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:52.103645 13050 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:52.168274  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 352306us
W20260709 13:03:52.166915  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 236096us
W20260709 13:03:52.169560  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 240334us
I20260709 13:03:52.170060  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:51.929801 13076 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:52.175290  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.176533  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:52.177274  9387 raft_consensus.cc:2368] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
W20260709 13:03:52.178690  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 466613us
I20260709 13:03:52.183709  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:52.157019 12974 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:52.187811  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:52.189409  8722 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:52.123075 12945 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.198490 13258 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:52.185379  8645 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:52.164712 13143 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:52.198585 13165 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:52.198010 13256 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:52.206228  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:52.206810  8647 leader_election.cc:400] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:03:52.207161  8647 leader_election.cc:403] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:03:52.168344 13253 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:03:52.211817  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:52.194921 13050 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:52.221426  8826 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:52.206050 13269 raft_consensus.cc:3055] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:52.231935  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:52.217641 13258 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:52.214144 13184 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:52.243083  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 57770.7us
I20260709 13:03:52.244566  8823 raft_consensus.cc:2368] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:03:52.230736 13197 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:03:52.259410 13076 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:52.238385 13195 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:52.253074 12943 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.602s	user 0.005s	sys 0.007s
I20260709 13:03:52.233067 13269 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:52.219430 13062 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.308380 13090 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:52.308805  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:52.312987  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:52.317770 13269 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.319641 13269 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:52.326099 13266 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:03:52.255817 13184 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.329814 13184 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:52.223994 13256 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.339905 13256 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:52.340301 12945 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:52.340059  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:52.341324  9382 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:52.313895 13090 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:52.353461  8794 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:52.357010  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 81646.4us
W20260709 13:03:52.362524  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.382165  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:52.363126 12974 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:52.358881 12640 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:52.399169  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 218586us
I20260709 13:03:52.372128 13270 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:52.390914 13134 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:52.397634 12974 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.406592 13262 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:03:52.414095  8647 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:52.414815  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 44451.6us
I20260709 13:03:52.398437 13282 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:52.418565 13175 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:52.391332 13204 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:03:52.428031  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:52.416174 12974 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:52.419428 13195 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:52.414824 13169 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:52.418051 13134 raft_consensus.cc:697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:52.452441  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:52.448137  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:52.450085  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:52.452224 13195 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:52.459203  8647 leader_election.cc:400] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
W20260709 13:03:52.461036  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 47741.3us
I20260709 13:03:52.461894  9352 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:52.435259 13175 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:52.438742 13187 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:52.463331 13195 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:52.438795 13121 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.410517 13267 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:03:52.469625 13293 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:52.469309 13121 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.420662 13282 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:52.479293  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 280868us
W20260709 13:03:52.510789  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.474169  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 58632.9us
W20260709 13:03:52.513513  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.463625 13289 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:52.515897  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 50324us
I20260709 13:03:52.516228  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 819) took 1121 ms. Trace:
I20260709 13:03:52.514930 13289 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.523684 13282 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:52.464152 13175 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:52.525274 13176 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.526409 13165 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:52.526907 13165 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:52.527842 13165 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.535055  8749 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:52.535800  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:52.516597  8825 rpcz_store.cc:276] 0709 13:03:51.395302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.400256 (+  4954us) service_pool.cc:224] Handling call
0709 13:03:51.400697 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.573063 (+172366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.515971 (+942908us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b5400267fd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:52.516146 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":371628416}
I20260709 13:03:52.529362  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:52.504503 13281 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 13:03:52.514139 13185 consensus_meta.cc:287] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.839s	user 0.015s	sys 0.000s
I20260709 13:03:52.475323 13161 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:52.525571 13175 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.537827  8826 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:52.515316 13293 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.532369 13287 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:52.545420  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:52.554219  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:52.559294  8753 raft_consensus.cc:2410] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:03:52.532037 13271 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.195s	user 0.001s	sys 0.001s
I20260709 13:03:52.531204 12776 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:52.563174  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:52.538290 13204 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:52.529090 13165 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:52.528740 13176 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:52.520153 13062 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:52.547290  9355 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:52.565071  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:52.554104  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:52.566257 12776 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.474289 13057 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.551559 13197 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:03:52.553643 13268 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.284s	user 0.004s	sys 0.000s
I20260709 13:03:52.569247  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:52.529961 13282 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:52.571851  8754 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:52.571848  8752 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:52.565241 13076 consensus_queue.cc:1059] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.011s
W20260709 13:03:52.576339  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.611378  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.550868 13249 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:03:52.595212 13213 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.612298 13213 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.612769 13061 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:52.613117  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2545) took 1009 ms. Trace:
I20260709 13:03:52.613430 13061 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:52.596943 13273 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.347s	user 0.001s	sys 0.001s
I20260709 13:03:52.492961 13134 consensus_queue.cc:237] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.590035  8749 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:03:52.530048 13289 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:52.568856 13287 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:52.575420 13284 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:03:52.575188 12776 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:52.589896 13143 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:52.597983  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 48129.7us
I20260709 13:03:52.637985  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:52.591952 13262 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:52.594990 13264 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.352s	user 0.000s	sys 0.002s
I20260709 13:03:52.547219 13185 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.600539  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:52.551393 13293 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:52.613580  8748 rpcz_store.cc:276] 0709 13:03:51.603358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.636329 (+ 32971us) service_pool.cc:224] Handling call
0709 13:03:51.682643 (+ 46314us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:51.755708 (+ 73065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.755748 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:51.756474 (+   726us) log.cc:844] Serialized 52 byte log entry
0709 13:03:51.942762 (+186288us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:52.111507 (+168745us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.158018 (+ 46511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.158172 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.493430 (+335258us) spinlock_profiling.cc:243] Waited 89.4 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:52.607221 (+113791us) raft_consensus.cc:1685] finished
0709 13:03:52.612881 (+  5660us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.613041 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104036,"prepare.queue_time_us":171015,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":85401,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":10675,"spinlock_wait_cycles":205724544,"thread_start_us":185749,"threads_started":1,"wal-append.queue_time_us":452000}
I20260709 13:03:52.598839 13270 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:52.630853  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 777) took 1355 ms. Trace:
I20260709 13:03:52.638875  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:52.645193  8826 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:52.647667  8672 rpcz_store.cc:276] 0709 13:03:51.269084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.271806 (+  2722us) service_pool.cc:224] Handling call
0709 13:03:52.624561 (+1352755us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54002a8fd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:52.624758 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766889216}
I20260709 13:03:52.648429  8828 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:52.648353 13092 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:52.638077 13176 raft_consensus.cc:3055] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:52.613318 13057 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:52.650262  8645 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:52.654002  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:52.650662 13176 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:52.601506 12640 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:03:52.617893 13213 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:52.657568  9385 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:52.591037 13175 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:52.646281 13187 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.649410 13270 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:52.658197 13262 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:52.586102 13062 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.625339 13197 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.624346 13282 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.650400 13092 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.700384 13092 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:52.678138 13187 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.704499  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1874) took 1024 ms. Trace:
I20260709 13:03:52.708767 13189 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:52.709254 13195 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.711787 13185 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:52.712337  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:52.669174 13143 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:52.714596 13143 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.715063 13143 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:52.721683 13276 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:03:52.726161 13187 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:52.760892 13187 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.685178 13061 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:52.685603  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 724) took 3576 ms. Trace:
I20260709 13:03:52.763041  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:52.762518  8747 rpcz_store.cc:276] 0709 13:03:49.081371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.199842 (+118471us) service_pool.cc:224] Handling call
0709 13:03:50.699295 (+1499453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:50.800515 (+101220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.800547 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:50.801109 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:03:50.892967 (+ 91858us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:50.893016 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.377640 (+484624us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400102a54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:51.377705 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.420313 (+ 42608us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:51.504057 (+ 83744us) raft_consensus.cc:1685] finished
0709 13:03:51.504081 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.655512 (+1151431us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400102a50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:52.657240 (+  1728us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1298,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":2640,"spinlock_wait_cycles":3682992128,"thread_start_us":91360,"threads_started":1,"wal-append.queue_time_us":314459,"wal-append.run_cpu_time_us":5197,"wal-append.run_wall_time_us":1386189}
I20260709 13:03:52.761559 13187 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.767436 13187 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:52.692391 13161 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:52.669766 13213 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.771342 13213 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.773797 13213 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:52.783180  8825 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:52.687522 13262 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.786988 13269 consensus_queue.cc:1059] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:52.788468 13277 raft_consensus.cc:3055] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:52.788868 13277 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:52.698101 13121 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:52.727068 13176 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.730979 13202 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:52.805826  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 182493us
I20260709 13:03:52.807045  8672 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:52.732156 12640 consensus_queue.cc:1243] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:52.732455 12974 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:52.736035 13263 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.421s	user 0.003s	sys 0.000s
W20260709 13:03:52.739800 13275 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.504s	user 0.002s	sys 0.000s
I20260709 13:03:52.740918 13062 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:52.742527 13256 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:52.744258 13282 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:52.683907  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:52.745765 13167 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:52.746727 12945 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:52.748162  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 783) took 1231 ms. Trace:
I20260709 13:03:52.827759  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:52.749111  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:52.829003  8672 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:52.751299  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2039) took 1070 ms. Trace:
I20260709 13:03:52.751408 13197 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:52.752015  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 70048.3us
I20260709 13:03:52.835000  8823 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:52.835256  9353 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:52.752074  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:52.753273  8826 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:52.753340 13185 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:52.704944  8821 rpcz_store.cc:276] 0709 13:03:51.680248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.680381 (+   133us) service_pool.cc:224] Handling call
0709 13:03:51.687325 (+  6944us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:51.698889 (+ 11564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.698932 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:51.699708 (+   776us) log.cc:844] Serialized 26 byte log entry
0709 13:03:51.795921 (+ 96213us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:51.880703 (+ 84782us) log.cc:844] Serialized 14 byte log entry
0709 13:03:51.889763 (+  9060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.889860 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.594289 (+704429us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:52.704151 (+109862us) raft_consensus.cc:1685] finished
0709 13:03:52.704267 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.704423 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4364,"prepare.queue_time_us":89676,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":11859,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":224,"spinlock_wait_cycles":243272576,"thread_start_us":95158,"threads_started":1,"wal-append.queue_time_us":474537}
I20260709 13:03:52.755056  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:52.755905  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:52.757365  8793 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.017s	sys 0.010s
W20260709 13:03:52.838250  8793 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.018s	sys 0.011s
W20260709 13:03:52.838652  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 642775us
W20260709 13:03:52.839264  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.767987 13261 raft_consensus.cc:3055] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:52.800967  8679 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:52.810667  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:52.842535  8673 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:03:52.823560  9389 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:52.749915  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:52.821167  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:52.830287  8676 rpcz_store.cc:276] 0709 13:03:51.680719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.680834 (+   115us) service_pool.cc:224] Handling call
0709 13:03:52.751123 (+1070289us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b54002576d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:52.751235 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24564608}
I20260709 13:03:52.808736 13161 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.785441 13143 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.828161  8674 rpcz_store.cc:276] 0709 13:03:51.517220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.526892 (+  9672us) service_pool.cc:224] Handling call
0709 13:03:51.546267 (+ 19375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.877642 (+331375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.747886 (+870244us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54002571d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:52.748097 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":759758720}
I20260709 13:03:52.762414 13024 consensus_queue.cc:1059] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:03:52.697243 13287 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:52.859208  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:52.850741  8829 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 10: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:52.851938  8680 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:52.835139 13176 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:52.863592  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:52.883116  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:52.883901  8676 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:52.845541 13189 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:52.897926  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:52.841785 13261 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:52.900805  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:52.901110  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:52.901572  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:52.848124 13121 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.902472  9380 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:52.847211 13175 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:52.921566  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:52.915879  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.926226  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:52.927016  8821 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:52.926226  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:52.845196 13289 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.932053  8824 raft_consensus.cc:1275] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:52.933393  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:52.848383 13136 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:03:52.846446 13256 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.935581  8829 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:52.938153  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:52.941455  8827 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:03:52.957186  8676 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:03:52.951891  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.957616  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:52.944597  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:52.837723 12945 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.860205 13161 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:52.958525  8752 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:52.957306  8827 pending_rounds.cc:85] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 3
I20260709 13:03:52.969156  8827 pending_rounds.cc:107] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:03:52.969540  8827 raft_consensus.cc:2889] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:52.964864 13298 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:52.964929  8680 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:52.921825 13165 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:52.838183 13277 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.889508 13204 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:03:52.985091  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:52.861677 13270 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.987042  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:52.973426 13062 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.968082 13195 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:52.995800  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 343965us
I20260709 13:03:52.962957 13293 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.887807 13167 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:52.996861  8680 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:52.963863 13175 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:53.001163  8794 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:52.947953 13121 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:53.019483  8828 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:53.025997  9353 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:53.019003  8794 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:53.028746  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:52.992646 13165 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:53.038838  8826 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:53.051381  9353 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:53.022810  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:53.054845  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:53.056721  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:53.057904 13180 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:53.058404 13180 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:53.058952 13180 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:53.030751 13249 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:53.075021 13300 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:53.075489  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:53.075930 13300 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:53.076524 13300 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:53.018438 13062 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:53.079308 13270 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:53.080363 13293 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:53.080802 13161 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:53.081427 12640 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:53.082023 12640 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:53.082449 12640 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:53.063445 13276 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:53.083089 12945 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:53.083933 13278 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.822s	user 0.003s	sys 0.002s
I20260709 13:03:53.085163 13256 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:53.020944 13185 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:03:53.087083  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:53.088488  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:53.088992 13167 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:53.090022  8680 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:53.090858 13143 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:53.092116  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 408346us
I20260709 13:03:53.092680  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:53.093137  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:53.093361 13121 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:53.093936 13143 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:53.094331  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:53.095021  8673 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:53.095832 13175 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:53.096374  8681 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:53.100394  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:53.101245  8827 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:53.104704  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:53.106204  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:53.033990 12974 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:53.066283 13296 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:03:53.107064  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:53.110828  8823 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:53.111567 13277 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:53.002723 13261 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:53.113564 13165 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:53.114815  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:53.115640  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:53.124284  8793 leader_election.cc:336] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.633s (SENT)
I20260709 13:03:53.195835  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:53.064818  8828 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:53.129575 13299 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:53.129949  8645 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:53.093956  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:53.135957 13062 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:53.143447  8796 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:53.135936  8827 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
W20260709 13:03:53.138420  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.162840  8719 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:53.035418 13223 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.015s	sys 0.004s
I20260709 13:03:53.093521  8676 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:53.028836 13213 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:03:53.219869  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 57242.4us
W20260709 13:03:53.189697  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:53.190478  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 45062.2us
I20260709 13:03:53.149706  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:53.192003  9385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 882) took 1495 ms. Trace:
I20260709 13:03:53.192828  8821 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:53.191669  8824 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:53.196810  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:53.233587  8752 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:53.234905  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 85319.4us
I20260709 13:03:53.222023  9385 rpcz_store.cc:276] 0709 13:03:51.677624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.684531 (+  6907us) service_pool.cc:224] Handling call
0709 13:03:51.692420 (+  7889us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.659926 (+967506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.172377 (+512451us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b540061b554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:53.172583 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2219142016}
I20260709 13:03:53.084291 13289 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:53.193897 13090 raft_consensus.cc:3055] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:53.231871  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 88515.1us
I20260709 13:03:53.222986  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:53.246332  8828 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:53.243108  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:53.192071 13169 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.247538  8719 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:53.207824 13223 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.016s	sys 0.005s
I20260709 13:03:53.259763  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:53.260676  8673 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:03:53.264473  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:53.200388 13076 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:53.265671  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:53.269615  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:53.268338  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 236594us
I20260709 13:03:53.200668 13299 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:53.213029 12881 raft_consensus.cc:3055] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:53.276455  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:53.232801 13175 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:53.198429 13185 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:03:53.219491 13213 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:53.274324  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:53.270648  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:53.277283  8674 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:53.244454 12945 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:53.255427 13062 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:53.257709 13090 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:53.299186 13315 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.299913 13315 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:53.300242 13090 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:53.301822 13090 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:53.266049 13297 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:53.284440 13187 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:53.302847 13297 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.306334 13297 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:53.279737 12881 raft_consensus.cc:740] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:53.293299 13213 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:53.303349  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 173561us
I20260709 13:03:53.307548 12881 consensus_queue.cc:260] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:53.308756 12881 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:53.308805  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:53.273487 13287 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:03:53.256493 13306 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:53.291891  8676 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:53.270042 13134 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:53.264559 13305 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:53.293422 13076 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:53.269872 13256 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:53.304199  8674 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:53.328006  8673 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:53.258402 13191 consensus_queue.cc:1059] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:03:53.267086 13169 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:53.356727  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 232525us
I20260709 13:03:53.302584 13315 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:53.296305 12945 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:53.266453 13061 consensus_queue.cc:1048] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
W20260709 13:03:53.367802  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 103431us
I20260709 13:03:53.339149 13175 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:53.377290  8793 leader_election.cc:336] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.655s (SENT)
I20260709 13:03:53.382395  8719 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:53.364989 13300 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:53.383602  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:53.387208  8793 leader_election.cc:336] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.651s (SENT)
I20260709 13:03:53.362887 13305 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:53.297001 13165 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 13:03:53.381098 13299 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:53.395363 13299 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:53.374958 12945 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:03:53.428143 13122 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:53.428819 13122 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.429276 13122 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:53.341712 13256 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:53.450810  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 184276us
I20260709 13:03:53.452242  8828 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:53.454378  9355 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:53.455818  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 147149us
I20260709 13:03:53.385500 13300 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:53.460784  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 186836us
W20260709 13:03:53.459444  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 181929us
W20260709 13:03:53.470081  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 224271us
W20260709 13:03:53.470919  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 83766.1us
I20260709 13:03:53.459662  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:53.426379 13061 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:53.492022  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:53.492933  8793 leader_election.cc:336] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.779s (SENT)
W20260709 13:03:53.494272  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.354823 13306 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:53.455775 13256 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:53.487154 13180 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:53.517221  8793 leader_election.cc:336] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.800s (SENT)
I20260709 13:03:53.481688 13321 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:53.462122 13324 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:53.480580 13175 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:53.503553 13061 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:03:53.529135  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 145603us
I20260709 13:03:53.528499  9380 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:53.524333  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:53.483618 13287 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 13:03:53.530313 13202 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.716s	user 0.008s	sys 0.005s
I20260709 13:03:53.539108  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:53.539851  8756 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:53.542181  9387 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:53.543107  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:53.543933  8755 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:53.531864 13061 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:53.548424  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:53.527562 13324 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:53.515798 13300 consensus_queue.cc:1059] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:03:53.552637  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:53.554535  8673 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:53.522663 13312 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:03:53.593827  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:53.596856  8681 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:53.542007 13012 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:53.599439  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 107540us
I20260709 13:03:53.535312 13175 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:53.552682 13076 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:53.656876  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 56342.7us
I20260709 13:03:53.604579  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:53.783396  8752 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.549s	user 0.009s	sys 0.004s
I20260709 13:03:53.549829 13325 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:53.686375  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:53.691969  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:53.644820  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 127633us
W20260709 13:03:53.816601  8793 leader_election.cc:336] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.838s (SENT)
I20260709 13:03:53.816504  8752 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:03:53.601830  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:53.560009 13324 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:53.817023  8824 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:53.643263 13180 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:53.829509  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:53.830158  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:53.684496 13326 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:03:53.618554  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:53.815825  8722 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:53.650142  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:53.836560  8673 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:53.808629  8753 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:53.596138 13287 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.629976 13322 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:53.604404 13312 consensus_queue.cc:1243] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:53.643213  8748 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:03:53.845651  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.846388  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:53.836817  8674 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:03:53.682560 13176 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.839740 13287 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:53.649099 13306 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:53.813578  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 280434us
I20260709 13:03:53.604115 13012 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:53.620858 13335 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:53.602032 13309 log.cc:927] Time spent T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:03:53.640126 13169 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:53.847834  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:53.629509 12640 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.004s
I20260709 13:03:53.840020  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:53.873486  8673 raft_consensus.cc:2368] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9.
I20260709 13:03:53.601284 13321 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:53.685418  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:53.854012  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:53.854578 13345 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:53.854110 13176 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:53.857108 13322 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:53.858742  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.859695  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:53.847913 13293 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:53.864814  8828 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:53.880174 13302 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:03:53.876088  8756 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:53.656950 13204 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:53.880390  9386 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:53.880398  8681 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:03:53.884964  8753 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:03:53.880818  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:53.700093  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:53.684985 13305 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:53.888275  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:53.830397 13267 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:53.813354 13040 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:03:53.906745  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:53.848187 13334 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.908648  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:53.869289 13175 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:53.828222 13325 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:53.974139  8828 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:53.974983  8674 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:53.888690 13169 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.910763 13322 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:53.912107  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2044) took 1085 ms. Trace:
I20260709 13:03:53.909982 13321 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:53.993995  8679 rpcz_store.cc:276] 0709 13:03:52.826821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.842462 (+ 15641us) service_pool.cc:224] Handling call
0709 13:03:52.842894 (+   432us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:52.843243 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.843274 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:52.860714 (+ 17440us) log.cc:844] Serialized 52 byte log entry
0709 13:03:53.138486 (+277772us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:53.138536 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.138645 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:53.911833 (+773188us) raft_consensus.cc:1685] finished
0709 13:03:53.911859 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.912042 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":131772,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":7395,"thread_start_us":213246,"threads_started":1,"wal-append.queue_time_us":397580}
I20260709 13:03:54.020941  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:53.788288 13331 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.208s
I20260709 13:03:54.026124  8823 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:53.916744  8673 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: [10.5-12.6]
I20260709 13:03:53.904990  8829 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:53.924216 13293 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:54.029755  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:54.030802  8825 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
W20260709 13:03:53.907615 12881 consensus_meta.cc:287] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.594s	user 0.010s	sys 0.003s
I20260709 13:03:53.974241  8830 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:54.039209  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:53.977890  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:53.992635  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:53.916354 13287 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:53.922271 13342 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:54.024994  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:54.063048  8824 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
W20260709 13:03:54.037500  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 184104us
I20260709 13:03:53.992478 13267 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.064417  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:54.065517  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:54.066831 13213 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.760s	user 0.011s	sys 0.003s
I20260709 13:03:54.066939  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.067392  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:54.067844  8679 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:54.068205  8829 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 12: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:54.068674  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.060479 13305 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:03:54.069435  8679 raft_consensus.cc:2393] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:54.069901 13305 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:53.952921 13306 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.910596 13344 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:53.884406 12640 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.070295 13305 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:54.079759  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:54.081139  8719 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:54.070827 13306 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.023520 13169 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.071758  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:54.097651  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 281074us
I20260709 13:03:54.101008  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:54.101812  8674 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:54.102165  8828 raft_consensus.cc:2393] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:54.104168  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:54.105054  8829 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:54.106660  8681 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
W20260709 13:03:54.113866  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.115950  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:54.117846  8826 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:54.060313 13348 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:54.114537  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.123066  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.124018  8674 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:54.125204  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:54.132472  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:54.133431  8793 leader_election.cc:336] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.726s (SENT)
I20260709 13:03:54.133944  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:54.138132  8681 raft_consensus.cc:2376] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12. Re-sending same reply.
I20260709 13:03:54.140162 13349 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.140007 13288 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:54.141152 13169 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:54.140863 13349 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.143663  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:54.144061 13303 log.cc:927] Time spent T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Append to log took a long time: real 0.841s	user 0.004s	sys 0.005s
I20260709 13:03:54.144681  8824 raft_consensus.cc:2393] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:54.076124 13267 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:54.140075  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:54.149017 13305 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.023794 13334 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:54.158825  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.118875 13297 consensus_meta.cc:287] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.804s	user 0.014s	sys 0.005s
W20260709 13:03:54.188495  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 79199.6us
I20260709 13:03:54.194025  8679 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:03:54.067420 13213 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.197614  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.201494  8679 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:54.196692  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.208711  8674 raft_consensus.cc:2376] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15. Re-sending same reply.
W20260709 13:03:54.209947  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 76558.3us
I20260709 13:03:54.210952 13350 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:54.211860 13343 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:03:54.211961 13350 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:54.213330  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:54.213624 13062 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.904s	user 0.008s	sys 0.005s
I20260709 13:03:54.214210  8830 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:54.158389 13176 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:54.154074 13322 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.212617  8827 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:54.214601  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.216809  8679 raft_consensus.cc:2410] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:54.217669  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.214071 13062 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:54.218361  8679 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:54.215492  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:54.214886 13325 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:54.215320 13305 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:54.220679  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.221472  8830 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:03:54.222675  8679 raft_consensus.cc:2410] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:03:54.219149 13349 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:54.219543 13062 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:54.219667  8754 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:54.227527 13062 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.230865 13062 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:54.231838 13169 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:54.233604 13169 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:54.234102 13169 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:54.121762 13076 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:54.226354  8679 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:03:54.239665  8793 leader_election.cc:336] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.097s (SENT)
I20260709 13:03:54.240376  8793 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:54.257402  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.259450  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.260313  8681 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:03:54.189302 13134 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:54.279901  8679 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:54.281657  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 192309us
I20260709 13:03:54.283336  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:54.273799  9352 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:54.267632  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.287537  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.288269  8681 raft_consensus.cc:2376] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15. Re-sending same reply.
W20260709 13:03:54.288399  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.288690  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:54.084301 13344 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.305991  8719 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:54.286973 12945 consensus_meta.cc:287] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.883s	user 0.015s	sys 0.003s
I20260709 13:03:54.136706 13348 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:54.293589 13122 consensus_meta.cc:287] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.864s	user 0.017s	sys 0.000s
I20260709 13:03:54.298629 13062 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:54.308635 13256 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.785s	user 0.011s	sys 0.003s
W20260709 13:03:54.315609 13090 consensus_meta.cc:287] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.013s	user 0.012s	sys 0.004s
I20260709 13:03:54.318401  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.329679  8681 raft_consensus.cc:2376] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9. Re-sending same reply.
I20260709 13:03:54.205823 13297 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.305450 13267 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:54.336336  8679 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:54.338828  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.317543 13213 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:54.290213 13167 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:03:54.414718  8824 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:54.436408  8821 raft_consensus.cc:2368] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11.
I20260709 13:03:54.415309  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:54.420006 13364 log.cc:927] Time spent T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:03:54.420181  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.446731  8679 raft_consensus.cc:2368] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9.
I20260709 13:03:54.420388 13189 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.420727 13360 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:54.421552 12640 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:54.415589 13361 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:54.326843 12945 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.333114 13062 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.422987 13344 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:54.330329 13122 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.423815 13261 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:54.277828 13293 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.329512 13306 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:54.338812 13267 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:03:54.445019  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 205394us
I20260709 13:03:54.496959  8747 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 5. Current term is 6. Ops: []
W20260709 13:03:54.497123  8793 leader_election.cc:336] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.100s (SENT)
I20260709 13:03:54.447543  8681 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:54.414431 13362 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:54.439628 13167 consensus_queue.cc:1243] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:54.291981 13076 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.338128 13256 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.467010  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.509666  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:54.472247  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.455597 13297 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:54.466490 13336 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:54.500022 13335 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.611s	user 0.014s	sys 0.001s
I20260709 13:03:54.509445  8679 raft_consensus.cc:2376] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12. Re-sending same reply.
W20260709 13:03:54.515758  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 205447us
I20260709 13:03:54.493418 13322 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:54.520886 13358 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:54.500160 13267 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:54.536512 13313 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 1.233s	user 0.000s	sys 0.004s
I20260709 13:03:54.538797 13306 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.539446 13306 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.542160 13306 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:54.544054  8750 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:54.527884  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.495883 12640 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:54.465644 13189 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.499857 13362 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:54.587047 13321 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:54.535826 13357 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.225s	user 0.000s	sys 0.000s
I20260709 13:03:54.549623  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1882) took 1650 ms. Trace:
I20260709 13:03:54.550496 13076 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:54.557809  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 60770.6us
I20260709 13:03:54.545228 13335 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:54.554800 13359 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:54.560544 13351 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:54.530598 13336 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.567162  9384 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:54.556682 12945 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:54.555043 13062 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:54.538371 13322 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.553788 13012 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:54.553316 13331 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:03:54.555506 13061 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.966s	user 0.011s	sys 0.002s
I20260709 13:03:54.551203 13122 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:54.581823  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.584438  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 800) took 1331 ms. Trace:
I20260709 13:03:54.551144 13256 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:54.556540  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.560513  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 757) took 1409 ms. Trace:
I20260709 13:03:54.599615  8681 raft_consensus.cc:2410] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:54.601629 13321 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:54.632140  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.638788  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.660835  8722 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:54.612543  8822 rpcz_store.cc:276] 0709 13:03:52.899766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.899907 (+   141us) service_pool.cc:224] Handling call
0709 13:03:52.927070 (+ 27163us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:52.927443 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:52.965986 (+ 38543us) log.cc:844] Serialized 14 byte log entry
0709 13:03:53.197700 (+231714us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:53.202239 (+  4539us) log.cc:844] Serialized 26 byte log entry
0709 13:03:53.202474 (+   235us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:53.202530 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.202665 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:54.535835 (+1333170us) spinlock_profiling.cc:243] Waited 7.81 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:54.543795 (+  7960us) raft_consensus.cc:1685] finished
0709 13:03:54.549329 (+  5534us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.549534 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39679,"prepare.queue_time_us":38489,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":38848,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":6041,"spinlock_wait_cycles":20110720,"thread_start_us":229488,"threads_started":1,"wal-append.queue_time_us":307676}
I20260709 13:03:54.608857 13276 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:54.668618 13299 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.273s	user 0.016s	sys 0.000s
I20260709 13:03:54.557562 13334 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:54.675436  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.703052  8674 raft_consensus.cc:2410] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:54.552130 13348 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:54.609285 13362 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:54.663865 13371 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.080s	user 0.000s	sys 0.003s
W20260709 13:03:54.663506 13363 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:03:54.682847  8680 rpcz_store.cc:276] 0709 13:03:53.253378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.274945 (+ 21567us) service_pool.cc:224] Handling call
0709 13:03:53.289537 (+ 14592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.456785 (+167248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.456835 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.456896 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:53.456945 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.583962 (+1127017us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54001f9d54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:54.584053 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.584313 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383956864}
I20260709 13:03:54.712060  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:54.712953  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.715061  8753 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:54.714473  8756 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:54.682166  8673 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:54.696159  8679 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:54.695143  8751 rpcz_store.cc:276] 0709 13:03:53.151375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.152705 (+  1330us) service_pool.cc:224] Handling call
0709 13:03:53.198987 (+ 46282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.199277 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:53.343551 (+144274us) log.cc:844] Serialized 14 byte log entry
0709 13:03:54.559853 (+1216302us) spinlock_profiling.cc:243] Waited 3.46 ms on lock 0x7b18003e63e0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e20a14bc 00007f32df3ef48b 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287
0709 13:03:54.559999 (+   146us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:54.560069 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:54.560144 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.560245 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.560425 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50065,"prepare.queue_time_us":193962,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":51,"raft.queue_time_us":139176,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":16691,"spinlock_wait_cycles":209980928,"thread_start_us":549594,"threads_started":3,"wal-append.queue_time_us":216339}
I20260709 13:03:54.700773  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:54.587915 12640 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:54.698348 13308 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.390s	user 0.012s	sys 0.020s
I20260709 13:03:54.727514  8752 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:03:54.727972  8752 pending_rounds.cc:85] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 6
I20260709 13:03:54.728331  8752 pending_rounds.cc:107] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:03:54.728586  8752 raft_consensus.cc:2889] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:54.727396  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:54.727279  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:54.729342  8749 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:54.723090 13334 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:54.711737 12945 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:54.734910  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.738303  8674 raft_consensus.cc:2376] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12. Re-sending same reply.
I20260709 13:03:54.667737 13359 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.742170  8751 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:54.712026 13335 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:54.749713  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 85571.8us
I20260709 13:03:54.744324  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 841) took 1045 ms. Trace:
I20260709 13:03:54.693302 13061 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.751293  8821 rpcz_store.cc:276] 0709 13:03:53.699360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.699688 (+   328us) service_pool.cc:224] Handling call
0709 13:03:54.744023 (+1044335us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b540032ee54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:54.744230 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1032477056}
I20260709 13:03:54.754958  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.752553  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.724427 13348 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.693044 13331 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:54.759096  8793 leader_election.cc:336] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.035s (SENT)
I20260709 13:03:54.703969 13276 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.759603  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:54.762880  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:54.762880  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:54.763720  8752 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:54.737129 12945 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:54.768682  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.774528 13331 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:54.763118 13339 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:54.782344  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.783650  8680 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:54.761400 13358 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.788738  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 792) took 1990 ms. Trace:
I20260709 13:03:54.789508  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.791011  8674 raft_consensus.cc:2410] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:54.768164 12945 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:54.792969 13322 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:54.794162 13336 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:54.794312 13358 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.789238  8677 rpcz_store.cc:276] 0709 13:03:52.798332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.799010 (+   678us) service_pool.cc:224] Handling call
0709 13:03:52.816131 (+ 17121us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:52.816354 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.816379 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:52.837054 (+ 20675us) log.cc:844] Serialized 26 byte log entry
0709 13:03:53.187311 (+350257us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:53.187352 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.187432 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:54.566644 (+1379212us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:54.782522 (+215878us) raft_consensus.cc:1685] finished
0709 13:03:54.782548 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.788650 (+  6102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5736832,"thread_start_us":301124,"threads_started":1,"wal-append.queue_time_us":456422}
I20260709 13:03:54.796134  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.797883  8677 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:54.800997  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:54.789661 13305 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.569s	user 0.016s	sys 0.000s
I20260709 13:03:54.806711  8752 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:54.807837  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:54.754482 13335 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:54.794222 13169 consensus_meta.cc:287] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.560s	user 0.013s	sys 0.000s
I20260709 13:03:54.811069  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.814527  8680 raft_consensus.cc:2410] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:54.808857  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.762364 13348 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.813329  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:54.811636  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.819340  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.820447  8674 raft_consensus.cc:2410] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:03:54.821586  8677 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:54.823711  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.831406  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.832075  8677 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:54.833982  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.819623 13358 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:54.834743 13324 consensus_meta.cc:287] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.991s	user 0.003s	sys 0.012s
I20260709 13:03:54.840181 13348 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:54.841347  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 802) took 1352 ms. Trace:
I20260709 13:03:54.817009 13169 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.855793  8677 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:54.857553  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:54.858211  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.845201  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.859071  8748 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:54.859941  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.845594  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:54.860975  8680 raft_consensus.cc:2410] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:03:54.857136  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 60284.2us
I20260709 13:03:54.856424  8672 rpcz_store.cc:276] 0709 13:03:53.489439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.489685 (+   246us) service_pool.cc:224] Handling call
0709 13:03:53.512840 (+ 23155us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:03:53.529165 (+ 16325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.529208 (+    43us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:03:53.549015 (+ 19807us) log.cc:844] Serialized 104 byte log entry
0709 13:03:53.794203 (+245188us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:53.849991 (+ 55788us) log.cc:844] Serialized 14 byte log entry
0709 13:03:54.655882 (+805891us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b1800449140. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:03:54.840396 (+184514us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b1800449140. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e5ef13c7 00007f32e25039a7 00007f32e25130bf 00007f32e2513038
0709 13:03:54.840677 (+   281us) log.cc:844] Serialized 14 byte log entry
0709 13:03:54.840956 (+   279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.841050 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:54.841099 (+    49us) raft_consensus.cc:1685] finished
0709 13:03:54.841113 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.841278 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":172842,"prepare.queue_time_us":1052296,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":100,"raft.queue_time_us":11232,"raft.run_cpu_time_us":523,"raft.run_wall_time_us":28915,"spinlock_wait_cycles":119036928,"thread_start_us":233978,"threads_started":1,"wal-append.queue_time_us":399812}
I20260709 13:03:54.851046  8749 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:54.834100 13321 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:54.864406  8677 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:03:54.865347  8645 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:54.865583  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.866357  8677 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:54.866513  8645 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:54.867285  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.842527 13061 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:54.870164  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:54.871847  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:54.871989  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.851620 13359 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:54.872869  8680 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:03:54.868553  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.877959  8677 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:54.844120 13267 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:54.881788  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.885154  8672 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:54.856724 13324 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.911638  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.896953  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.886412  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:54.913903  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:54.869251 13305 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:54.870755 13076 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:54.885313  8748 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:54.915987  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.887938 13360 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:54.920615 13329 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.285s	user 0.006s	sys 0.004s
I20260709 13:03:54.874847 13187 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
W20260709 13:03:54.928766 13333 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.300s	user 0.000s	sys 0.009s
I20260709 13:03:54.910912 13358 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:03:54.930368 13358 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:54.930707  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2056) took 1083 ms. Trace:
I20260709 13:03:54.927496 13076 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:54.930795 13358 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:54.931246  8823 rpcz_store.cc:276] 0709 13:03:53.847399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.847529 (+   130us) service_pool.cc:224] Handling call
0709 13:03:54.930431 (+1082902us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54002283d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:54.930588 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352082688}
I20260709 13:03:54.932293 13358 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.932763  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 794) took 2133 ms. Trace:
I20260709 13:03:54.874765 13306 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:54.934499  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:54.904027 13335 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:54.897719 13267 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:54.938539  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.933853  8675 rpcz_store.cc:276] 0709 13:03:52.798907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.799103 (+   196us) service_pool.cc:224] Handling call
0709 13:03:52.819855 (+ 20752us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.231533 (+411678us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.231952 (+   419us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:53.279192 (+ 47240us) log.cc:844] Serialized 26 byte log entry
0709 13:03:53.516761 (+237569us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:53.516807 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.516882 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:54.928141 (+1411259us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:54.932404 (+  4263us) raft_consensus.cc:1685] finished
0709 13:03:54.932431 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.932675 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":50604,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":1406,"spinlock_wait_cycles":267297024,"thread_start_us":616161,"threads_started":2,"wal-append.queue_time_us":244813}
I20260709 13:03:54.943162  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.943079  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.947734  8672 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 9: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:54.874687 13348 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:03:54.947024  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.949014  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:54.923597 13305 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:54.951069  8752 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:54.925088 13382 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:54.950886  8677 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:54.938273 13358 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:54.898887 13362 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.959172  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:54.951586 13315 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:54.964169  8829 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:54.964395  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:54.961068  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:54.965005  8827 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:54.961585  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.966001  8672 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:54.963001  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.963507  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:54.966902  8826 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:54.965764  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:54.967247  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 53422.3us
I20260709 13:03:54.962147  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.967650  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:54.966537  8674 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:54.968065  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:54.969399  8680 raft_consensus.cc:2410] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:54.969317  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:54.970063  8822 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:54.970157  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:54.953763 13305 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:54.971625  8675 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:54.971314  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:54.978443  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.972348  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:54.981509  8752 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:54.950098 12945 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:54.987985  8680 raft_consensus.cc:2410] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:54.989229  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:54.989463  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:54.990538  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:54.991503  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:54.991835  8829 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:54.992400  8672 raft_consensus.cc:2410] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:03:54.997864  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:55.000674  8827 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:55.001971  8677 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:55.004199  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:54.968348 13382 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:54.971333 13315 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:54.980553 13076 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:55.015185 13382 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.016064  8677 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:55.018349  8672 raft_consensus.cc:1217] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Deduplicated request from leader. Original: 11.10->[12.11-15.13]   Dedup: 15.13->[]
I20260709 13:03:55.019137  8672 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: [12.11-15.13]
I20260709 13:03:55.007162  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.010432  8753 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:55.011927 13383 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:55.003094 13187 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:55.013521 13324 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.006417 13061 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:55.019977  8675 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:55.020921  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.012559 13315 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:55.032963  8752 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
W20260709 13:03:55.020272 13323 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.372s	user 0.018s	sys 0.016s
W20260709 13:03:55.031175  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.027236  8680 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:55.034121 13061 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.039201 13061 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:55.043143  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:55.043789 13359 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:55.021073 13383 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:55.047438 13383 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:55.055136  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:55.056046  8752 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:55.021023 13340 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:55.064760  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.064538  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:55.065954 13340 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:55.068351  8675 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:55.069140 13340 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.069741  8675 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:55.070269 13340 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:55.072064 13384 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:55.081436  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:55.072933 13362 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.084092  8822 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:55.003732 13306 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.054s
W20260709 13:03:55.034850 13323 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.401s	user 0.018s	sys 0.016s
I20260709 13:03:55.087589  8823 raft_consensus.cc:2393] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:55.037585 13187 raft_consensus.cc:697] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:55.091642 13345 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.092085  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.094234  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2565) took 1556 ms. Trace:
I20260709 13:03:55.097608  8747 rpcz_store.cc:276] 0709 13:03:53.537875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.538001 (+   126us) service_pool.cc:224] Handling call
0709 13:03:53.548576 (+ 10575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.501437 (+952861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.093986 (+592549us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b540034bf54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:55.094170 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2180646656}
I20260709 13:03:55.098345  8674 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:55.090018 13360 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:55.110718  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 104124us
I20260709 13:03:55.111562  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:55.114653  8672 raft_consensus.cc:2393] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:55.104043 13362 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:55.117552 13362 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.120620 13362 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:55.114938 13359 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:55.121987 13359 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.114595  8645 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:55.126058  8793 leader_election.cc:336] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.523s (SENT)
I20260709 13:03:55.126214  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:55.128232  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.100685 13361 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:55.129662  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:55.091146 13384 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.130690 13306 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:55.135443  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:55.149835  8747 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:55.150429  8672 pending_rounds.cc:85] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 6
I20260709 13:03:55.151118  8672 pending_rounds.cc:107] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:03:55.151420  8672 raft_consensus.cc:2889] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:55.154081 13359 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:55.101545 13345 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:55.140736 12945 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.143031 13386 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:55.144589 13289 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:55.104322 13324 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.146155  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.146454  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:03:55.156105  9386 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 621) took 2292 ms (client timeout 2205 ms). Trace:
W20260709 13:03:55.161532  9384 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 894) took 1942 ms (client timeout 2484 ms). Trace:
I20260709 13:03:55.187942  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:55.188803  8829 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:55.164575  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1902) took 1060 ms. Trace:
I20260709 13:03:55.163358  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:55.177986  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:55.177271  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:55.191452  8822 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:03:55.191797  8823 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:55.154978  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:03:55.187793  9386 rpcz_store.cc:269] 0709 13:03:52.864201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.885775 (+ 21574us) service_pool.cc:224] Handling call
0709 13:03:55.155880 (+2270105us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b54001a3bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:55.156002 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1939655040}
I20260709 13:03:55.192938  8830 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:55.162433  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.194680  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:55.188524  9384 rpcz_store.cc:269] 0709 13:03:53.219251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.220991 (+  1740us) service_pool.cc:224] Handling call
0709 13:03:54.509479 (+1288488us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:03:55.161274 (+651795us) spinlock_profiling.cc:243] Waited 56.8 ms on lock 0x7b54003a6454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:55.161444 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142175360}
I20260709 13:03:55.189522  8828 rpcz_store.cc:276] 0709 13:03:54.104617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.111575 (+  6958us) service_pool.cc:224] Handling call
0709 13:03:54.112060 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:54.418468 (+306408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:54.418520 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:54.434430 (+ 15910us) log.cc:844] Serialized 26 byte log entry
0709 13:03:54.661295 (+226865us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:54.661358 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.164008 (+502650us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b540016ab54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:55.164084 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.164153 (+    69us) raft_consensus.cc:1685] finished
0709 13:03:55.164174 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.164508 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1179,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":121,"spinlock_wait_cycles":679410944,"thread_start_us":173284,"threads_started":1,"wal-append.queue_time_us":274959}
I20260709 13:03:55.195329  8829 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:03:55.204651  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.204854  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:55.205693  8749 raft_consensus.cc:2368] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
I20260709 13:03:55.206704  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:55.207717  8827 raft_consensus.cc:2393] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:03:55.208921  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.187376 13289 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:55.213965  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.161903 13024 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.157842 13306 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:55.158188 13276 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:55.193493 13386 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.225862 13386 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:55.155037 13384 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:55.199185 13324 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:55.229907 13324 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.230082  8756 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:55.205449 13338 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:03:55.230747  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:55.231912 13319 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.926s	user 0.005s	sys 0.000s
I20260709 13:03:55.232179 13061 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:55.233647  8680 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:55.233778  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 790) took 2436 ms. Trace:
I20260709 13:03:55.148859 13189 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.135492 12640 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:55.124640 13360 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:55.215766  8755 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:55.224391  9355 leader_election.cc:400] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11.
I20260709 13:03:55.223685 13024 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.235116  8747 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:55.236537  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:55.254194  8827 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:55.237377  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.237741  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:55.255882  8755 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:55.246738  8756 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:55.238174  8752 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:55.249600  8748 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:55.239014  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:55.260171  8749 raft_consensus.cc:2368] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11.
I20260709 13:03:55.261267  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.240525  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:55.263005  8828 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:55.262908  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:55.238807  9355 leader_election.cc:403] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:03:55.230769 13338 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.255929  8822 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:55.155117 13361 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.224280 13306 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:55.261268  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.272529  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:55.262018  8749 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:55.270406 13338 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.155095 13187 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.234162  8678 rpcz_store.cc:276] 0709 13:03:52.797620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.797748 (+   128us) service_pool.cc:224] Handling call
0709 13:03:52.815677 (+ 17929us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.035839 (+220162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.035880 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:53.055127 (+ 19247us) log.cc:844] Serialized 26 byte log entry
0709 13:03:53.282067 (+226940us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:53.282124 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.283133 (+  1009us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.193275 (+1910142us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:03:55.233269 (+ 39994us) raft_consensus.cc:1685] finished
0709 13:03:55.233291 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.233582 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7531,"raft.queue_time_us":80195,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":28086,"spinlock_wait_cycles":12376832,"thread_start_us":284225,"threads_started":2,"wal-append.queue_time_us":111626}
I20260709 13:03:55.237980 13360 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.217955 13185 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:55.278319  8755 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:55.281922  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.293359  8748 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:55.218179 13267 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:55.234582 13061 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.311264  8822 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:03:55.279186 13276 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:55.266275 13289 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:55.324741 13289 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:55.280166 13305 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:55.334190  8645 leader_election.cc:400] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
I20260709 13:03:55.336903  8678 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:55.316546 13024 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.341442  8645 leader_election.cc:403] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:55.312263 13185 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.348814  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.318465 13267 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.353209  8645 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:55.356546  9353 leader_election.cc:400] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11.
I20260709 13:03:55.367211  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.368111  8678 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:03:55.339744 13383 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.354578 13395 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:55.379586  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.380048  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.352134 12945 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:55.386777  8830 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:55.367324 12640 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:55.395635  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:55.375741 13384 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:55.396481  8822 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:55.410187  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.401221  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.416152  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:55.417754  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.391593 12945 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.420213 13267 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.421473 12945 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:55.391608 13061 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:03:55.422326 13385 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:03:55.422529 13061 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:55.423998 13393 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:03:55.385022 13345 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.424619 13383 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:55.425027 13061 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:55.425879 13185 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.400602 13398 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:55.426889 13398 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.429095 13398 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:55.431879 13305 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.395663 13395 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:55.432960  8647 connection.cc:256] RPC call timeout handler was delayed by -1.0071393069999885s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
W20260709 13:03:55.433923  8793 leader_election.cc:336] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.323s (SENT)
I20260709 13:03:55.434980  8678 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:55.435660 13395 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.436290  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.437172  8678 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:55.438015  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.438174 13279 raft_consensus.cc:3055] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:55.438843 13279 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:03:55.421617 13390 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 13:03:55.444412  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:55.444368  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:55.446774  8678 raft_consensus.cc:2410] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:03:55.447108  9384 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:55.409843 13384 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.449364 13169 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.449848 13384 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:55.399627 12640 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:55.453044  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1905) took 1347 ms. Trace:
I20260709 13:03:55.453512  8824 rpcz_store.cc:276] 0709 13:03:54.105421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.145423 (+ 40002us) service_pool.cc:224] Handling call
0709 13:03:54.145821 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.435721 (+289900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.452745 (+1017024us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54003dd1d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:55.452975 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427341184}
I20260709 13:03:55.453604 12640 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:55.456761  8750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2568) took 1915 ms (client timeout 2484 ms). Trace:
W20260709 13:03:55.482949  8750 rpcz_store.cc:269] 0709 13:03:53.545796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.552314 (+  6518us) service_pool.cc:224] Handling call
0709 13:03:55.456534 (+1904220us) spinlock_profiling.cc:243] Waited 983 ms on lock 0x7b5400139554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:55.456691 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2260967808}
I20260709 13:03:55.484743  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.485527  8750 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:03:55.487722  9353 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:55.455550 13279 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.489332 13279 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:55.493216 13361 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:55.494750  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.495692 13040 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:55.495590  8750 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:55.496282 13040 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:55.497572  9353 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:55.498762 12656 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:55.499254 12656 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.499630 12656 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:55.476171  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:55.503268 13061 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:55.503875 13061 raft_consensus.cc:697] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:55.504158  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.504783 13061 consensus_queue.cc:237] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:55.503182  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:55.506161 13345 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:55.507148  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:55.506546 13040 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:55.508066 13040 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:55.508355  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:55.506592 13345 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:55.509187 13345 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:55.498500  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.510187  8755 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:55.510564  8722 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:55.471344  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.478261 13386 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:55.511894  8681 raft_consensus.cc:2410] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:55.505299  8748 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:55.515136  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:55.516044  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 162917us
I20260709 13:03:55.518329  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:55.521278  9353 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:55.524958  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:55.526393  8647 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:55.528273  8793 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:55.519971 13167 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:55.536765  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.537763  8755 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:03:55.508116 13324 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:55.541028  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.526345  8722 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:55.541862  8750 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:55.535564  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:55.545516  8645 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:55.546923  9353 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:55.546854  8645 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:55.551910  8681 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:03:55.552795  9355 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:55.554733  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:55.517328 13395 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.556039  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:55.543977  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:55.557557  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:55.559598  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:55.445765 13399 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
I20260709 13:03:55.584052  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:55.562193  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:55.562832  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.538519 13305 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:55.585139  8747 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:55.574684  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:55.533102 13277 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:55.604876  8821 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:03:55.513156 13401 raft_consensus.cc:2764] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:03:55.583748  9387 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:03:55.600008  9380 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:55.533476 13394 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:55.538843 13361 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:55.539041 13382 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:55.624467  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:55.601483  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:55.602447  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55632us
I20260709 13:03:55.626494  8755 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:55.627043  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:55.526851 13327 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:55.550374 13169 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:55.558764 13187 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:55.606797 13012 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:55.607935  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 55233.7us
I20260709 13:03:55.611919  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:55.613687  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:55.550563 13167 raft_consensus.cc:697] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:55.615694  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:55.616032  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.585932  8749 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:55.591272  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:55.545215 13340 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:55.541518 13324 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.669073 13324 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:55.583993 13383 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:55.582916 13384 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:55.560652 13398 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:55.634895 13189 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:55.618263 13395 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.639599  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:55.639361  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:55.619635 13269 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:55.641088  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:55.586097 12945 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:03:55.641479  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:55.642755  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:55.644011  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:55.644524  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:55.689910  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.690236  9388 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:55.690547  9383 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:55.645010  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:55.607972 13323 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:55.599702 13399 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:55.609660 13277 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.609932 13276 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11.
I20260709 13:03:55.654595  9355 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:55.657637  8747 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
W20260709 13:03:55.660683  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.683275 13189 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.708320  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:55.709713  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:55.673447 13383 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.710912 13383 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:55.691715 13269 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:55.712456 13175 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:55.712795 13269 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:55.712917 13175 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:55.715127  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:55.716274 13395 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:55.716630  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:55.718329  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.699136 13277 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:55.708369 13189 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:55.679416 13340 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:55.721750 13340 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:55.723515  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.724992  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.725660  8673 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:03:55.662781  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:55.624094 13293 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:55.624599 13382 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.550259 13400 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.110s
I20260709 13:03:55.563148 13306 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:55.675129  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 83988.3us
I20260709 13:03:55.635764 13169 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.631423 13327 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:55.743760  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:55.682444  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:55.651881 13305 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.635196 13361 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:55.655072 13403 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:55.690763  8678 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:03:55.731084 13395 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:55.759337  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 175381us
I20260709 13:03:55.695708  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.739156 13382 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:55.726589  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:55.727447  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:55.728739  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 74273.8us
I20260709 13:03:55.683019 13315 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:55.732633  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:55.739902  8793 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:55.740242  8719 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:55.741952  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2478) took 1673 ms. Trace:
I20260709 13:03:55.767976  8677 rpcz_store.cc:276] 0709 13:03:54.068691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.005123 (+936432us) service_pool.cc:224] Handling call
0709 13:03:55.005471 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.016506 (+ 11035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.741689 (+725183us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b5400299d54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:55.741886 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23832960}
W20260709 13:03:55.745394  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 63066.2us
I20260709 13:03:55.700115 13399 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.769814  8719 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:55.749161  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:55.805090  8747 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:55.666707 13012 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.651467 13394 raft_consensus.cc:697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:55.731303 13293 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.758884  8676 raft_consensus.cc:2410] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:55.751189 13289 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:55.668949 13167 consensus_queue.cc:237] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:55.760962  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:55.762059  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 69911.3us
I20260709 13:03:55.764560  8673 raft_consensus.cc:2410] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:55.745854 13169 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:55.748703 13327 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:55.841217  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:55.843029  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 103167us
I20260709 13:03:55.873064  9380 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:03:55.874336  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:55.771452  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:55.844540  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 74793.2us
I20260709 13:03:55.845736  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2564) took 2308 ms. Trace:
I20260709 13:03:55.846549  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:55.681978 13398 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:03:55.856614  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 124041us
I20260709 13:03:55.876339  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:55.876094  8754 rpcz_store.cc:276] 0709 13:03:53.536985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.537117 (+   132us) service_pool.cc:224] Handling call
0709 13:03:53.537443 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.227056 (+689613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.845330 (+1618274us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54001022d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:55.845643 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1568516864}
I20260709 13:03:55.877242  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:55.858451  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.753386 13403 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:55.862949  8796 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:55.862906  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.870132  8749 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:55.877702  8719 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:55.908195  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 45360.9us
I20260709 13:03:55.878633  8674 raft_consensus.cc:2393] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:55.848261 13187 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.848153 13360 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:55.909813  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.856163 13293 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:55.908205  8677 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:55.912341  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:03:55.932118  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:55.933183  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.918737 13270 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.848666 13401 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:55.844076 13399 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:55.854172 12640 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:55.857740 13406 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:03:55.855867 13359 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:55.756876 13305 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.854882 13400 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:55.935599 13394 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.937023 13407 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:55.939146  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2073) took 1228 ms. Trace:
I20260709 13:03:55.978793  8675 rpcz_store.cc:276] 0709 13:03:54.710282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.277353 (+567071us) service_pool.cc:224] Handling call
0709 13:03:55.300556 (+ 23203us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:55.348082 (+ 47526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.348119 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:55.356961 (+  8842us) log.cc:844] Serialized 52 byte log entry
0709 13:03:55.759210 (+402249us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:55.759259 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.759342 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.938741 (+179399us) raft_consensus.cc:1685] finished
0709 13:03:55.938775 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.939046 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42050,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":156,"thread_start_us":398630,"threads_started":2,"wal-append.queue_time_us":558901}
I20260709 13:03:55.953891 13361 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:55.958137  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2069) took 1359 ms. Trace:
I20260709 13:03:55.920706 12945 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:55.960573  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 48338.3us
I20260709 13:03:55.861987 13362 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:55.861389 13289 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:55.870726 13386 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.968029  9387 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:03:55.817899 13395 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:55.972576  8747 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:55.973943  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 208688us
I20260709 13:03:55.906344 13398 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:55.977049  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 205678us
I20260709 13:03:55.959581 13076 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:55.982476  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:55.985230  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 108069us
I20260709 13:03:55.981406  8672 rpcz_store.cc:276] 0709 13:03:54.599099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.116170 (+517071us) service_pool.cc:224] Handling call
0709 13:03:55.149606 (+ 33436us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:55.288510 (+138904us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.288552 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:55.469927 (+181375us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b2c000014c8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e0df2a11 00007f32e0e02372 00007f32e2534a01 00007f32e24f943a 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2
0709 13:03:55.511329 (+ 41402us) log.cc:844] Serialized 52 byte log entry
0709 13:03:55.727777 (+216448us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:55.727822 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.727899 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.957661 (+229762us) raft_consensus.cc:1685] finished
0709 13:03:55.957701 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.958050 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6083,"raft.queue_time_us":177631,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":11255,"spinlock_wait_cycles":3478912,"thread_start_us":310971,"threads_started":2,"wal-append.queue_time_us":335263}
I20260709 13:03:56.089527  8677 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:55.989902  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.921232 13360 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.920253 13315 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:55.909842 13403 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:55.926941 13411 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:56.044461 13400 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.047364 13306 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:56.050817  9383 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:55.978078 13401 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.051631 13189 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.114064  8677 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:55.923707 12656 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.056504 13012 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:55.961205 13348 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:56.056900  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:55.992137 13407 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.059415  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:56.061715  8756 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:56.062572  8750 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:56.081013  8749 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:56.082144  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:56.086175  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:56.046415 13279 consensus_meta.cc:287] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.556s	user 0.012s	sys 0.000s
I20260709 13:03:55.991420 12945 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.148161  8673 raft_consensus.cc:2410] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:03:56.149125  8676 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:03:56.151763 13395 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.153548 13383 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:56.099507  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 332092us
I20260709 13:03:55.929970 13187 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.081633 13289 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:56.092312 13406 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:03:56.111589  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2486) took 1985 ms. Trace:
W20260709 13:03:56.111393  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 350521us
I20260709 13:03:55.974485 13417 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:56.094254 13360 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:56.155160  9389 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:03:56.122494  8672 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:56.094751 13315 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.975505 13270 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.092435 13076 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:56.106078 13362 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:56.131466  8796 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.132:41749 timed out after 1.648s (SENT) [suppressed 28 similar messages]
W20260709 13:03:56.137645  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 78330.3us
I20260709 13:03:56.136593  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:56.140726  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 54650.8us
I20260709 13:03:56.239442  8674 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:03:56.143251  8824 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.007s	sys 0.018s
I20260709 13:03:56.156231  8747 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:03:56.240897  8824 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.007s	sys 0.018s
I20260709 13:03:56.157110  9385 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:56.155670 12945 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:56.160878  8755 consensus_meta.cc:287] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.003s
I20260709 13:03:56.247536  8755 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:56.160570  9381 raft_consensus.cc:2368] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13.
I20260709 13:03:56.162146  8680 rpcz_store.cc:276] 0709 13:03:54.126141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.064112 (+937971us) service_pool.cc:224] Handling call
0709 13:03:55.233458 (+169346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.234137 (+   679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.111334 (+877197us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54001ff9d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:56.111518 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338274560}
W20260709 13:03:56.164626  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.162015  9352 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:56.271512  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 109574us
I20260709 13:03:56.275008  9383 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:03:56.159613  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 282006us
I20260709 13:03:56.162649  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:56.183372  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.138170 13418 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:56.146806 13411 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:56.156620 13012 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:56.237672 13076 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:56.111023 13338 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:56.245736  8672 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:03:56.276638  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 114101us
I20260709 13:03:56.290025  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:56.273701  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.278105  8719 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:56.234230 13315 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:56.296881 13416 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:56.297690 13416 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.300779 13416 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.301636 13416 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:56.296144  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:56.288336  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2161) took 1438 ms. Trace:
I20260709 13:03:56.243054 13417 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:56.268945 12640 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:03:56.305402  8752 rpcz_store.cc:276] 0709 13:03:54.850118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.747876 (+897758us) service_pool.cc:224] Handling call
0709 13:03:55.762571 (+ 14695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.762867 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:55.872421 (+109554us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.288022 (+415601us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:56.288074 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:56.288129 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.288190 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:56.288279 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7347,"prepare.queue_time_us":68716,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":50,"thread_start_us":313835,"threads_started":1}
I20260709 13:03:56.269354 13306 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:56.367880  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 762) took 1149 ms. Trace:
W20260709 13:03:56.311744 13345 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.802s	user 0.013s	sys 0.000s
I20260709 13:03:56.302500 13416 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:56.378769  8751 rpcz_store.cc:276] 0709 13:03:55.218723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.959840 (+741117us) service_pool.cc:224] Handling call
0709 13:03:55.960397 (+   557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.960714 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:56.106508 (+145794us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.367415 (+260907us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:56.367491 (+    76us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:56.367569 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.367650 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:56.367811 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21078,"prepare.queue_time_us":146006,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":65,"thread_start_us":384695,"threads_started":2}
I20260709 13:03:56.284003 13136 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:56.283434 13411 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:56.326642 13386 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:56.286324 13359 raft_consensus.cc:697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:56.282940 13418 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:56.325681 13061 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.339552 13419 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:56.389094  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2151) took 1542 ms. Trace:
I20260709 13:03:56.403535  8753 rpcz_store.cc:276] 0709 13:03:54.846948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.851267 (+  4319us) service_pool.cc:224] Handling call
0709 13:03:54.851606 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.019809 (+168203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.388813 (+1369004us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b540034b054. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:56.389031 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364744960}
I20260709 13:03:56.285656 13012 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:56.392923  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 153361us
I20260709 13:03:56.408020  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:56.407995  8793 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:56.397439  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 266025us
I20260709 13:03:56.407589  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:56.387298 13277 consensus_meta.cc:287] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.668s	user 0.014s	sys 0.001s
I20260709 13:03:56.412364  8823 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:56.368171 13395 consensus_queue.cc:1059] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:03:56.367794 13305 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.395984 13411 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:56.417923  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:56.402978 13323 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.686s	user 0.012s	sys 0.000s
I20260709 13:03:56.382709 13306 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.421691  8822 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
W20260709 13:03:56.423048  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 117776us
I20260709 13:03:56.410826 13012 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:56.430863 13323 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:56.431870 13187 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.431320 13359 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.427548  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:56.405344 13348 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:56.403723 13418 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:56.421056 13277 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.391413 13345 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.443822 13306 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:56.444667 13435 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:56.421322 13269 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.708s	user 0.007s	sys 0.007s
I20260709 13:03:56.448602 12640 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:56.415021 13299 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:56.449323 13293 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.412012 13428 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:56.450439 13417 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.475762  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:56.477764 13315 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:56.453737  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 175730us
W20260709 13:03:56.451037 13324 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.767s	user 0.009s	sys 0.003s
I20260709 13:03:56.420396 13189 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:56.392159 13416 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.462332  9382 raft_consensus.cc:2368] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10.
I20260709 13:03:56.485244 13324 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.487197 13416 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.487689 13324 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:56.408809 13061 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.490069  8719 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:56.442454 13433 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.462500 13428 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.500373  8719 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:56.459074 12640 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.500164  9384 raft_consensus.cc:2368] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9.
I20260709 13:03:56.457216 13400 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:56.499508  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 220992us
I20260709 13:03:56.504076 13270 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:56.504906 13424 log.cc:927] Time spent T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:03:56.461915 13383 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:56.462949 13401 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.420744 13434 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
W20260709 13:03:56.531368  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:56.532172  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.536453  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:56.500522 13361 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:56.457481 13348 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.527129  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:56.476337 13299 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.540112 13289 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.524348 13437 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:03:56.579864 13040 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.071s	user 0.012s	sys 0.003s
I20260709 13:03:56.582196 13299 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.542780 13441 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:56.463135 13269 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:56.547583 13431 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.130s	user 0.001s	sys 0.001s
I20260709 13:03:56.548733 13061 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.548394 12945 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.510927 12640 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:56.550007 13305 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:56.550695  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:56.519156  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:56.516896 13407 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.511227 13428 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:56.608924  8722 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:56.607587 13348 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.616340 13289 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.616997  8677 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:56.619925 13345 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.620844 13277 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:03:56.621762 13327 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.003s
I20260709 13:03:56.621454 13411 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:56.623663 13169 consensus_meta.cc:287] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.714s	user 0.010s	sys 0.004s
W20260709 13:03:56.624217 13382 consensus_meta.cc:287] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.775s	user 0.012s	sys 0.000s
W20260709 13:03:56.625214 13403 consensus_meta.cc:287] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.517s	user 0.007s	sys 0.004s
I20260709 13:03:56.566241 13012 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.626150 13315 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.510358 13433 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.643529  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 903) took 1008 ms. Trace:
I20260709 13:03:56.689177  8822 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:56.689102  9381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 709) took 1044 ms. Trace:
I20260709 13:03:56.636492  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:56.693919  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 85093.8us
I20260709 13:03:56.694098  9382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 713) took 1048 ms. Trace:
I20260709 13:03:56.657963  9385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 705) took 1017 ms. Trace:
I20260709 13:03:56.695363  9385 rpcz_store.cc:276] 0709 13:03:55.640978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.641212 (+   234us) service_pool.cc:224] Handling call
0709 13:03:56.657644 (+1016432us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b54002369d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:56.657864 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1074086144}
I20260709 13:03:56.661162  9387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 901) took 1048 ms. Trace:
I20260709 13:03:56.668618  8678 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:56.675661  9352 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:56.698300  9382 rpcz_store.cc:276] 0709 13:03:55.645454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.636136 (+990682us) service_pool.cc:224] Handling call
0709 13:03:56.694004 (+ 57868us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:56.694664  9384 rpcz_store.cc:276] 0709 13:03:55.635029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.635306 (+   277us) service_pool.cc:224] Handling call
0709 13:03:56.643199 (+1007893us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b54003b2ed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:56.643434 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1858914560}
I20260709 13:03:56.679786  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 809) took 1046 ms. Trace:
I20260709 13:03:56.700599  8674 rpcz_store.cc:276] 0709 13:03:55.633567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.090960 (+457393us) service_pool.cc:224] Handling call
0709 13:03:56.108056 (+ 17096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.239895 (+131839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.679338 (+439443us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54002a8fd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:56.679678 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301237632}
I20260709 13:03:56.701077  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:56.660341  9380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 902) took 1036 ms. Trace:
W20260709 13:03:56.710068  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 190967us
I20260709 13:03:56.712388  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:56.715587  9389 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:56.693508  9381 rpcz_store.cc:276] 0709 13:03:55.644240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.644468 (+   228us) service_pool.cc:224] Handling call
0709 13:03:56.688865 (+1044397us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b5400018bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:56.689038 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1056022272}
I20260709 13:03:56.716264  9389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 714) took 1070 ms. Trace:
I20260709 13:03:56.708427  9380 rpcz_store.cc:276] 0709 13:03:55.624053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.624208 (+   155us) service_pool.cc:224] Handling call
0709 13:03:56.659958 (+1035750us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b540064e954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:56.660246 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":426105600}
W20260709 13:03:56.582813 13422 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
W20260709 13:03:56.582535 13399 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.591s	user 0.008s	sys 0.000s
I20260709 13:03:56.697314  9387 rpcz_store.cc:276] 0709 13:03:55.613124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.613325 (+   201us) service_pool.cc:224] Handling call
0709 13:03:56.660954 (+1047629us) spinlock_profiling.cc:243] Waited 90.2 ms on lock 0x7b54006e1754. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:56.661089 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207345024}
I20260709 13:03:56.719228  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:56.723734  9385 raft_consensus.cc:2376] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11. Re-sending same reply.
I20260709 13:03:56.718760  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:56.724066  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:56.716650  9389 rpcz_store.cc:276] 0709 13:03:55.645733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.550312 (+904579us) service_pool.cc:224] Handling call
0709 13:03:56.716197 (+165885us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:56.626469 13398 consensus_meta.cc:287] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.527s	user 0.003s	sys 0.005s
I20260709 13:03:56.724793  9382 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:03:56.729792  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:56.730950  9382 raft_consensus.cc:2368] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10.
W20260709 13:03:56.634707 13404 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.903s	user 0.000s	sys 0.006s
I20260709 13:03:56.735980  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:56.628051 13362 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:56.738281  8825 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:56.735327  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:56.718868 13350 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:56.611160 13040 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:56.638804 13269 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:56.740137  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:56.792924  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 857) took 1204 ms. Trace:
I20260709 13:03:56.745096  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:56.631875 13293 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:56.691751 13305 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:56.739866  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:56.739375 13437 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:56.744398  8822 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:56.715677 13122 raft_consensus.cc:2764] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:03:56.727313 13338 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:56.626727 13076 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.725239 13061 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:56.727836 13348 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.726168 13401 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:56.741757 13324 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:56.719715 13169 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.584630 13360 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.651582 13418 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.736296 13291 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:03:56.791442 13279 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:03:56.842801 13350 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.611357 13441 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.720085 13382 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.801244 13040 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.739595 13345 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:03:56.861716 13401 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:56.720304 13403 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:56.851615  8754 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.032s	sys 0.027s
I20260709 13:03:56.744375 13249 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:03:56.853823  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:56.735363 13398 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.794747 13323 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:56.688196 13175 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.795398  8824 rpcz_store.cc:276] 0709 13:03:55.588461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.588644 (+   183us) service_pool.cc:224] Handling call
0709 13:03:55.589094 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.589447 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:55.621223 (+ 31776us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.728634 (+1107411us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.792505 (+ 63871us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:56.792565 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:56.792619 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.792693 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:56.792851 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51093,"prepare.queue_time_us":32141,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":12556,"raft.queue_time_us":198375,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":5709,"thread_start_us":1103647,"threads_started":3,"wal-append.queue_time_us":712770}
I20260709 13:03:56.896503  8679 raft_consensus.cc:2368] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13.
I20260709 13:03:56.850281 13122 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:56.898216 13122 raft_consensus.cc:697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:56.899124 13040 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.899336 13122 consensus_queue.cc:237] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.901309 13360 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:56.856827 13422 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:03:56.850840 13338 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:56.903254 13338 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:56.904433 13433 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.736876 13399 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.907058 13175 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:56.907420 13452 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:03:56.718784 13327 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.910400 13327 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:56.911353  9388 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:56.911687 13441 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:56.912972 12945 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:56.913460 13169 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:56.914795  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:56.915799  9382 raft_consensus.cc:2368] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13.
I20260709 13:03:56.915990  8678 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:56.916662 13411 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:56.916805  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:56.891683 13305 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.888063  8822 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
W20260709 13:03:56.892953  8754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.032s	sys 0.028s
I20260709 13:03:56.920205  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2575) took 1819 ms. Trace:
I20260709 13:03:56.846412  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:56.870553 13348 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:56.920807  8754 rpcz_store.cc:276] 0709 13:03:55.100520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.877884 (+777364us) service_pool.cc:224] Handling call
0709 13:03:55.878302 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.878529 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:56.193567 (+315038us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.919803 (+726236us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:56.919858 (+    55us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:56.919906 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.919993 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:56.920130 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34773,"prepare.queue_time_us":214316,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":111083,"spinlock_wait_cycles":7806336,"thread_start_us":1005067,"threads_started":3,"wal-append.queue_time_us":201842}
I20260709 13:03:56.923350  9384 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:56.923357  9382 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:56.939124  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:56.940270  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:56.918385 13418 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:56.944249  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:56.948042  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:56.948814  8824 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:03:56.918706 13305 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:56.920128 13351 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:56.950727  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 213222us
I20260709 13:03:56.952688  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:56.951141  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:56.946664 12945 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:56.885025 13350 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.960755  8719 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:56.961856 13360 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:56.966975  8822 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:56.968334  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:56.969205  8822 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:56.970741  9382 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:56.970950 13434 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:56.971634 13434 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.973307 13289 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.973920 13289 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.976886 13289 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:56.974920 13394 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.978579 13419 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.978659 13394 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.979276 13419 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.979876 13403 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:56.981182 13394 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:56.981283  8793 leader_election.cc:336] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.990s (SENT)
I20260709 13:03:56.981544 13419 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:56.981858  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:56.970293  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:56.983367  8822 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:56.987365  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:56.988747  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:56.997011  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:56.998124  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:56.998885  8672 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:56.918443 13401 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:56.955475 13012 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:57.023294  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:57.024295  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:03:57.034354  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.957482 13418 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:56.974493 13434 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:56.947392  9384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:03:57.312893  9384 raft_consensus.cc:606] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:57.313341  9384 raft_consensus.cc:614] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:57.000932 13433 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:57.089102  8674 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:03:57.079458  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:57.325124  8822 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
W20260709 13:03:57.091410  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 104140us
W20260709 13:03:57.092154  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 90188.4us
W20260709 13:03:57.087230  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 124985us
W20260709 13:03:57.179271  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:57.313995  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.336249 13394 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.336823 13315 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:57.336934 13394 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.337486  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:57.089807 13411 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:57.338155  8821 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:57.339661 13299 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:57.340126 13299 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:57.341010 13299 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.342474  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:57.342733  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:33196 (request call id 7) took 1581 ms. Trace:
I20260709 13:03:57.343333  9384 rpcz_store.cc:276] 0709 13:03:55.739561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.943243 (+1203682us) service_pool.cc:224] Handling call
0709 13:03:57.313747 (+370504us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:57.345142  9384 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:57.347163  8719 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:57.358829 13448 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:57.359777 13448 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:57.362812 13458 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:57.366209 13407 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.366873 13407 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.368377 13457 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:57.003525 13384 consensus_queue.cc:1059] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.313s
I20260709 13:03:57.371490 13384 raft_consensus.cc:3055] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:57.371878 13384 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:57.372730 13384 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.374042 13384 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:57.374470 13457 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.375147 13457 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.378129 13457 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:57.382434  8821 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:03:57.391464  8821 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:57.395145 13382 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:57.100288 13441 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.296s
I20260709 13:03:57.397540 13441 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:57.398043 13441 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:57.399132 13345 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.399952 13345 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:57.406387  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:57.407367  8823 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:03:57.311347 13453 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.353s	user 0.001s	sys 0.000s
I20260709 13:03:57.315120 13360 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:57.091163 12945 raft_consensus.cc:697] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:57.338464 13411 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.445791 13175 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.454479  8830 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:57.463735  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2076) took 1044 ms. Trace:
I20260709 13:03:57.458349  8794 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:57.466204  8672 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:03:57.464295  8830 rpcz_store.cc:276] 0709 13:03:56.478397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.453694 (+975297us) service_pool.cc:224] Handling call
0709 13:03:57.454207 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:57.463288 (+  9081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.463643 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:57.469447  8794 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:57.474313  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:57.488070  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 813) took 1790 ms. Trace:
W20260709 13:03:57.490125  8824 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.001s
W20260709 13:03:57.491035  8681 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.023s	sys 0.030s
I20260709 13:03:57.468765 13360 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:57.491755  9384 raft_consensus.cc:2393] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:03:57.491340  8828 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.007s	sys 0.050s
W20260709 13:03:57.492519  8681 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.023s	sys 0.030s
I20260709 13:03:57.495573  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:57.504796  8828 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.008s	sys 0.050s
I20260709 13:03:57.492295  8824 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
W20260709 13:03:57.502420  8826 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.027s	sys 0.030s
W20260709 13:03:57.507889  8826 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.027s	sys 0.030s
I20260709 13:03:57.477687 13435 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:57.508986  8794 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:57.491912  8677 rpcz_store.cc:276] 0709 13:03:55.697373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.115314 (+417941us) service_pool.cc:224] Handling call
0709 13:03:56.115758 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.692924 (+577166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.487756 (+794832us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b5400298954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:57.487985 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1152237440}
I20260709 13:03:57.476644 13448 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:57.502488  8829 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.021s	sys 0.040s
I20260709 13:03:57.515774  8794 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:57.472051 13323 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 13:03:57.525415  8829 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.022s	sys 0.040s
I20260709 13:03:57.468289 13175 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:57.485313 13324 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.469089 13411 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.513990 13433 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:57.460837 13441 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.564738 13350 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:57.564869  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.566134  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:57.566761 12945 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.571864 13360 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:57.573019 13405 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.819s	user 0.003s	sys 0.000s
I20260709 13:03:57.532920 13435 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:57.548430 13442 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:57.535642 13448 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:03:57.559644 13324 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.575361 13433 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:57.584185  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:57.602679  8677 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:57.585201  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:57.576858 13428 consensus_meta.cc:287] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.849s	user 0.012s	sys 0.000s
W20260709 13:03:57.559309 12640 consensus_meta.cc:287] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.831s	user 0.011s	sys 0.002s
I20260709 13:03:57.591559  9384 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:57.590555 13327 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:03:57.571719 13323 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:57.533098 13348 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:57.596402  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:57.596333  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:57.603721  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:57.607458  9385 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:57.618115  8672 raft_consensus.cc:2410] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:03:57.608778  8678 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:57.618101  8677 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:03:57.603672 13441 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:57.609207 13416 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:03:57.627080  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:57.654924  8680 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:57.660744  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:57.606258 13428 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.622603 13348 raft_consensus.cc:3055] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:03:57.658643 13338 consensus_meta.cc:287] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.755s	user 0.010s	sys 0.004s
W20260709 13:03:57.682839  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 355621us
I20260709 13:03:57.618213 12640 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.659288 13323 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:57.631963 13122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.016s	sys 0.040s
I20260709 13:03:57.667411 13345 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:57.700963  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:57.701436  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:57.701905  8754 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:57.703074  8823 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
W20260709 13:03:57.705610  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.706776 13323 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:57.707095 13345 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:57.698129 13442 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:03:57.685524 13348 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:57.709021  8827 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:03:57.710170  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:57.666960 13416 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.710565  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:57.619256 13435 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.713495  8753 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
W20260709 13:03:57.702298 13122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.017s	sys 0.040s
I20260709 13:03:57.714210  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:57.694242 13338 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:57.741117  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:57.765146  8750 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:57.761096  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:57.718466  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:57.720561  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:57.777076  8825 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:57.738996  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:57.778743  8755 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:57.783768  8753 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:57.784410  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:57.717252  8823 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:57.790268  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:57.790540  8749 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:57.790412  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:57.791136  8752 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 12: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:03:57.793076  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 613552us
I20260709 13:03:57.795104  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:57.795949  8748 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:57.791719  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:57.798576  8747 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:57.801857  8751 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:03:57.803879  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 1.10752e+06us
I20260709 13:03:57.712875 13345 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.817469  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:57.819305  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:57.820194  8830 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:57.821134  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.824071  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:57.710824 13433 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.834017  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:57.839632 13407 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:57.815559 13175 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:57.844089 13323 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:57.851931 12640 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:57.808104 13348 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.852970 13345 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:57.857465  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:57.857547  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:57.840399 13407 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:57.816490 13411 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:57.860580  8680 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:03:57.875051  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:57.875612  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:57.823205 13434 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:57.871896  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:57.876246  8680 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:57.861958  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 346200us
I20260709 13:03:57.875772  8755 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:57.879721  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:57.880388  8756 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:57.872222  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1941) took 1183 ms. Trace:
I20260709 13:03:57.879595  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:57.882519  8753 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:57.882862  8830 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:03:57.886315  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:57.891609  8756 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:03:57.832531 13324 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:57.902966  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:57.904206  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:57.904842  9386 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:57.898731  8753 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:03:57.903774  8747 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:57.896800  8751 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:57.878275  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:57.908776  8755 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:03:57.866551 13175 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.922590 13324 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.922737 13122 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.922879 13175 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.923317 13324 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.923476 13122 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:57.925315 13175 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:57.925460 13324 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:57.925669 13122 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:57.926396  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:57.927882  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:57.929271  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:57.933722  8750 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:57.934564  8749 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:57.909696  8793 leader_election.cc:336] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.127s (SENT)
I20260709 13:03:57.935580  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:57.894721 13407 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:57.910506 13468 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:57.935904  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:57.919255 13416 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:57.921548 13338 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:57.866887 13345 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.888789 13469 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:57.930522  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:57.953809  8752 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:57.938424  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 60188.4us
I20260709 13:03:57.960449  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:57.938881  8754 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:03:57.939237  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:57.964147  8755 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:57.913494 13464 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:03:57.944829  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 237633us
I20260709 13:03:57.945881  8750 raft_consensus.cc:2368] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12.
W20260709 13:03:57.994211  8722 leader_election.cc:336] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Timed out: RequestConsensusVote RPC to 127.7.89.132:41749 timed out after 2.138s (SENT)
I20260709 13:03:57.994338  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:57.995566  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:57.996169  8750 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:03:57.952483  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:57.997018  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:57.997552  8751 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:57.903787 13428 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:57.942725  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 182223us
I20260709 13:03:57.985971 13416 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:57.945500 13463 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:03:58.012933  9386 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:03:57.909004 13348 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:57.999609  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.000174  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:58.016513  8754 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:58.017663  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:58.018348  8754 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:58.019443  8754 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:57.948623 13465 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:58.008626  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:57.890415 13434 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:57.990497  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:58.058338  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.092296  8823 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
W20260709 13:03:58.060375  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.949465 13466 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:58.157961  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 167575us
I20260709 13:03:58.161661  8828 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:03:58.003008 13411 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:03:58.029215 13384 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.655s	user 0.011s	sys 0.006s
I20260709 13:03:58.161963  8751 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:57.973093  8680 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:03:58.178685  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:57.987983 13345 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.193113  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1945) took 1502 ms. Trace:
I20260709 13:03:58.006942 13175 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.163s
I20260709 13:03:58.195307  8827 rpcz_store.cc:276] 0709 13:03:56.690546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.692505 (+  1959us) service_pool.cc:224] Handling call
0709 13:03:56.692822 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:57.711846 (+1019024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.192646 (+480800us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003dd1d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:58.193009 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2336208768}
I20260709 13:03:57.955826 13468 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:58.198603  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 284182us
I20260709 13:03:58.199460  8722 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:58.196111  8752 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:03:58.201002  8755 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:03:58.201740  8750 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 13. Ops: []
I20260709 13:03:58.200026  8794 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:58.161057 13465 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:57.999543 13469 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:58.211285  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:58.212121  8680 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:03:58.213250  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:58.172427 13474 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:58.214593  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:58.207877  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 145290us
I20260709 13:03:58.208143  8722 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:58.207759 13167 consensus_queue.cc:1059] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:03:58.225555 13481 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.234819 13481 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.236125  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:58.228204 13480 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:58.235657  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:58.237852 13480 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:58.239449  8680 raft_consensus.cc:2410] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:03:58.254997  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55114.5us
I20260709 13:03:58.202128 13435 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:58.254187  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:58.207285 13360 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:58.273181 13360 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:58.273519  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:58.273923 13480 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:58.274349 13360 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.274865  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:58.275516  8678 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:58.275794 13434 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:58.276737 13465 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:57.880846  8829 rpcz_store.cc:276] 0709 13:03:56.689239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.690687 (+  1448us) service_pool.cc:224] Handling call
0709 13:03:56.691053 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.691249 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:57.532188 (+840939us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.871672 (+339484us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:57.871762 (+    90us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:57.871848 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.871953 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.872137 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8359,"prepare.queue_time_us":248532,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":17556,"spinlock_wait_cycles":85120,"thread_start_us":1170976,"threads_started":2}
I20260709 13:03:58.279884 13345 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:58.280112 13469 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:58.281055  8719 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:58.281458 13442 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:58.282838 13435 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.284013 13428 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:58.284893 13468 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:58.285198 13435 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:58.285487  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:58.285674 13448 raft_consensus.cc:3055] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:58.286324 13448 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:58.286940  8719 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:58.287772 12656 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:58.287909 13428 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:58.288551 13428 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:03:58.234818 13467 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:03:58.212970 13411 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:58.282048 13338 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:58.293431  8719 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:58.289547 13428 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:58.323657  8796 leader_election.cc:336] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Timed out: RequestConsensusVote RPC to 127.7.89.132:41749 timed out after 1.593s (SENT)
I20260709 13:03:58.206892 13441 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:03:58.271775 13458 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:58.318023 13469 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:58.330541  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 103913us
W20260709 13:03:58.346118  8796 leader_election.cc:336] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Timed out: RequestConsensusVote RPC to 127.7.89.132:41749 timed out after 1.648s (SENT)
W20260709 13:03:58.334266  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.297978 13442 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:58.294605  8680 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
W20260709 13:03:58.348004  8673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2536) took 2809 ms (client timeout 2127 ms). Trace:
I20260709 13:03:58.352793  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:03:58.351876  8673 rpcz_store.cc:269] 0709 13:03:55.535350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.863568 (+328218us) service_pool.cc:224] Handling call
0709 13:03:58.341462 (+2477894us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54000dacd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:58.344544 (+  3082us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508599168}
I20260709 13:03:58.334462 12640 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:58.354404  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:03:58.347581  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 142155us
I20260709 13:03:58.341810 13468 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:58.341281 13486 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:58.358888  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:58.360641  8747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:03:58.347600 13469 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.360913  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:58.361146  8680 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:58.361353  8747 raft_consensus.cc:606] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:58.334044 13411 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.364317  8673 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 14: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:58.365736  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:58.367930  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:58.369370  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:58.359615 12640 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.352375 13458 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:58.374280  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2543) took 1386 ms. Trace:
I20260709 13:03:58.359364 13468 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.359599 13428 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:58.362699  8722 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:58.378051  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 121337us
I20260709 13:03:58.374718  8676 rpcz_store.cc:276] 0709 13:03:56.987624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.988609 (+   985us) service_pool.cc:224] Handling call
0709 13:03:57.039940 (+ 51331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:57.040671 (+   731us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:57.040717 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:57.087728 (+ 47011us) log.cc:844] Serialized 26 byte log entry
0709 13:03:57.712025 (+624297us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:57.712073 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.712145 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.374024 (+661879us) raft_consensus.cc:1685] finished
0709 13:03:58.374049 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.374199 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":175,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":141459,"thread_start_us":375333,"threads_started":1,"wal-append.queue_time_us":643470}
I20260709 13:03:58.361354 13486 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.350234 13421 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:58.362635 13489 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:03:58.365147 13469 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:58.395522 13458 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.396140 13399 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:58.394340 13442 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.397517 13407 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:58.419673  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 57081.3us
W20260709 13:03:58.391356  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.422142 13428 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.432775 13428 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:58.397521 13481 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:58.436357 12640 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:58.440286 13411 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:58.418569 13486 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:58.420727  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:58.419406  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:58.373873 13167 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:58.445829  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 210205us
I20260709 13:03:58.448035  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:58.448880  8676 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:58.449597  8794 leader_election.cc:400] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12.
I20260709 13:03:58.450019  8794 leader_election.cc:403] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:58.380043 13468 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:58.453280  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 100793us
I20260709 13:03:58.454046  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:58.454906 13476 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:58.455425 13476 raft_consensus.cc:697] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:58.456485  9353 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:58.456365 13476 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:58.457297 13338 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:58.457739 13338 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.458091 13338 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:58.459086  9383 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 715) took 2812 ms (client timeout 2138 ms). Trace:
W20260709 13:03:58.459542  9383 rpcz_store.cc:269] 0709 13:03:55.646188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.116651 (+470463us) service_pool.cc:224] Handling call
0709 13:03:58.458826 (+2342175us) spinlock_profiling.cc:243] Waited 35.5 ms on lock 0x7b54001a3bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:58.458993 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81617664}
I20260709 13:03:58.467242 13338 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:58.469753  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.471436  9385 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:58.473160  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:58.470323  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1938) took 1914 ms. Trace:
I20260709 13:03:58.474342  9387 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:58.474153  8826 rpcz_store.cc:276] 0709 13:03:56.555977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.607169 (+ 51192us) service_pool.cc:224] Handling call
0709 13:03:56.613230 (+  6061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.895180 (+281950us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:57.513292 (+618112us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.813062 (+299770us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.816126 (+  3064us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.816315 (+   189us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:57.818428 (+  2113us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:57.818487 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.394353 (+575866us) spinlock_profiling.cc:243] Waited 9.65 ms on lock 0x7b18004164a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:03:58.469356 (+ 75003us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b18004164a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419
0709 13:03:58.469978 (+   622us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b5400127354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:58.470035 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.470234 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":49702,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":113,"raft.queue_time_us":50665,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":6679,"spinlock_wait_cycles":695732224,"thread_start_us":867002,"threads_started":2,"wal-append.queue_time_us":710058}
I20260709 13:03:58.478250  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.484179  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.488577 13338 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:58.468768  9353 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:58.488708 13488 raft_consensus.cc:3055] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:58.491276  9380 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:58.491698 13488 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:58.468330  8576 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40096 (request call id 141) took 27542 ms (client timeout 15000 ms). Trace:
I20260709 13:03:58.492700 13488 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.493674  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.494139 13488 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:58.495436  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.496824  9353 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:58.501286 13299 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:58.497439  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.498569  9383 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:58.503960  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.505065 13338 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:58.505262 13276 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:58.505797 13276 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:58.506999  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.507839  9383 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:58.507647  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:58.508724 13276 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:58.515149  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.519542 13480 raft_consensus.cc:3055] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:58.519863  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.520072 13480 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:58.521042 13480 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:58.522042  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.522889  9387 raft_consensus.cc:2368] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9.
I20260709 13:03:58.525851  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.526834  9383 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:58.522377 13480 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:58.528108 13484 raft_consensus.cc:3055] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:58.528584 13484 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:58.529582 13484 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:58.532591 13482 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.305s	user 0.002s	sys 0.010s
I20260709 13:03:58.505615 13338 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:58.540853 13338 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:58.544065 13448 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.545822 13398 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:58.569316  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 117252us
W20260709 13:03:58.575842  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 106487us
W20260709 13:03:58.576812  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.579092  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:58.580549  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.584439  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.585752  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.586501  8821 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:58.587739  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:58.419749 13421 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:58.426954 13324 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:58.589687  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.578338  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:58.591990  8829 raft_consensus.cc:2393] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:03:58.492556  8576 rpcz_store.cc:269] 0709 13:03:30.949605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.246942 (+297337us) service_pool.cc:224] Handling call
0709 13:03:58.468028 (+27221086us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:50.153593 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:50.172946 (+ 19353us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:50.173007 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:03:50.173029 (+    22us) tablet.cc:662] Decoding operations
0709 13:03:50.198186 (+ 25157us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:50.198306 (+   120us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:50.198353 (+    47us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:03:50.199798 (+  1445us) tablet.cc:701] Row locks acquired
0709 13:03:50.199858 (+    60us) write_op.cc:260] PREPARE: finished
0709 13:03:50.200062 (+   204us) write_op.cc:270] Start()
0709 13:03:50.200238 (+   176us) write_op.cc:276] Timestamp: P: 1783602230199982 usec, L: 0
0709 13:03:50.200262 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:03:50.201047 (+   785us) log.cc:844] Serialized 6354 byte log entry
0709 13:03:50.414086 (+213039us) op_driver.cc:464] REPLICATION: finished
0709 13:03:50.710962 (+296876us) write_op.cc:301] APPLY: starting
0709 13:03:50.733048 (+ 22086us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:51.012158 (+279110us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:51.101179 (+ 89021us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:52.313275 (+1212096us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:52.313758 (+   483us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 13:03:52.313829 (+    71us) write_op.cc:312] APPLY: finished
0709 13:03:52.314484 (+   655us) log.cc:844] Serialized 148 byte log entry
0709 13:03:52.769896 (+455412us) spinlock_profiling.cc:243] Waited 7.92 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32df3ef0ac 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b
0709 13:03:53.285530 (+515634us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:53.298758 (+ 13228us) write_op.cc:454] Released schema lock
0709 13:03:53.298841 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:53.299507 (+   666us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:53.299542 (+    35us) write_op.cc:454] Released schema lock
0709 13:03:54.143085 (+843543us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
Metrics: {"dns_us":264,"mutex_wait_us":6233454,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":113575,"apply.run_cpu_time_us":61095,"apply.run_wall_time_us":2654851,"cfile_cache_hit":66,"cfile_cache_hit_bytes":70943,"num_ops":16,"prepare.queue_time_us":255790,"prepare.run_cpu_time_us":24291,"prepare.run_wall_time_us":215355,"raft.queue_time_us":27862,"raft.run_cpu_time_us":2433,"raft.run_wall_time_us":93227,"replication_time_us":213618,"spinlock_wait_cycles":56276992,"thread_start_us":1160382,"threads_started":5,"wal-append.queue_time_us":1111644,"wal-append.run_cpu_time_us":22885,"wal-append.run_wall_time_us":2949524}]]}
I20260709 13:03:58.593122  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.598513  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.602406  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.448122 13167 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.607172  8829 raft_consensus.cc:2410] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:03:58.601380  9355 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:58.610852  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:58.451155 12640 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:58.611594  8680 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:58.619259  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.619964  8826 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:58.611766  8823 raft_consensus.cc:2368] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
I20260709 13:03:58.610906  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.612304  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:58.624558  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.625046  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:58.625882  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:58.629415  8830 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:58.629806 13448 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:58.611678 13167 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:58.591202 13249 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:03:58.635129  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:58.636045  8826 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:58.618867 13419 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.637126 13419 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.596225 13484 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:58.639438 13419 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:58.449668 13359 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.164s
I20260709 13:03:58.647409  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:58.648192  8680 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:03:58.649085  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:03:58.589277 13340 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12.
I20260709 13:03:58.644898 13167 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.649688  8680 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:58.592557 13474 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.651928  8821 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 12: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:58.651590 13474 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:58.654126  8679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2532) took 3119 ms (client timeout 2933 ms). Trace:
I20260709 13:03:58.654225 13407 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:58.665963  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.621495 13399 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.654798 13076 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:58.654587  8679 rpcz_store.cc:269] 0709 13:03:55.534273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.534396 (+   123us) service_pool.cc:224] Handling call
0709 13:03:58.653914 (+3119518us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400257bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:58.654049 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3017799424}
I20260709 13:03:58.655834  9382 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:58.651616  9388 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 718) took 2966 ms (client timeout 1862 ms). Trace:
W20260709 13:03:58.657092  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.657238 13401 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:58.592351 13384 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:03:58.659328 13345 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.659843 13061 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.660300 13360 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.662922 13394 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:58.663126 13498 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.667136  8827 raft_consensus.cc:2410] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:03:58.668063  8676 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:03:58.688951  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.691063  8823 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
W20260709 13:03:58.668471  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 47607.4us
I20260709 13:03:58.693495  8645 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:58.620854  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:58.695392  8829 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:58.605183 13488 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.765656 13488 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.685786  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:03:58.686691  8722 leader_election.cc:336] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Timed out: RequestConsensusVote RPC to 127.7.89.132:41749 timed out after 1.674s (SENT)
I20260709 13:03:58.769661  8822 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:03:58.768250  8796 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:58.677259 13249 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:58.697481 13299 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:58.789709  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.677059 13382 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 13:03:58.676492 13407 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.791190 13382 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:58.791611 13382 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:03:58.792928 13407 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:58.699321 13076 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:58.699445  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:03:58.794271  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 94901.5us
I20260709 13:03:58.794541 13076 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:58.795048 13076 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:58.795204  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:58.795495 13076 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:58.710722  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2541) took 2244 ms. Trace:
I20260709 13:03:58.719244  8578 catalog_manager.cc:5697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:58.798646  9382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 905) took 2692 ms. Trace:
I20260709 13:03:58.798952 13480 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:58.799438 13480 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:58.800033 13480 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:58.799847  8578 catalog_manager.cc:5697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:58.800904  8677 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:58.722420  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:58.767257 13498 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:58.804526 13498 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:58.798241  8681 rpcz_store.cc:276] 0709 13:03:56.466549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.686523 (+219974us) service_pool.cc:224] Handling call
0709 13:03:56.696607 (+ 10084us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:56.972110 (+275503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.972147 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:56.972746 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 13:03:57.513080 (+540334us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:58.710082 (+1197002us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.710331 (+   249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.710426 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.710467 (+    41us) raft_consensus.cc:1685] finished
0709 13:03:58.710480 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.710635 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1470301,"prepare.queue_time_us":1198712,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":283017,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":246,"thread_start_us":533151,"threads_started":1,"wal-append.queue_time_us":792858}
I20260709 13:03:58.806038  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:58.736418  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:58.799237  9382 rpcz_store.cc:276] 0709 13:03:56.105906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.972034 (+866128us) service_pool.cc:224] Handling call
0709 13:03:58.655597 (+1683563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.674854 (+ 19257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.798354 (+123500us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b540025ce50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:58.798543 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3870532864}
I20260709 13:03:58.808562  8578 catalog_manager.cc:5697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 7 to 9, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 9 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:03:58.809585 13167 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:58.737846  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:58.810695  8830 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:58.744490  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:58.745383 13475 log.cc:927] Time spent T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.657s	user 0.005s	sys 0.000s
I20260709 13:03:58.746824  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1954) took 1027 ms. Trace:
I20260709 13:03:58.825383  8824 rpcz_store.cc:276] 0709 13:03:57.719254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.719410 (+   156us) service_pool.cc:224] Handling call
0709 13:03:57.719833 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.186018 (+466185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.186060 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:58.186122 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:58.186173 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.746527 (+560354us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b54002297d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:58.746580 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.746762 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12081,"raft.queue_time_us":12794,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":8644,"spinlock_wait_cycles":1043195136}
I20260709 13:03:58.750537 13191 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:58.755820  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:58.832099  8749 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:58.756529  8756 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:58.758467  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:58.758800  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:58.759738  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:58.760579  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:58.834576  8755 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:03:58.834986  8796 leader_election.cc:400] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9.
I20260709 13:03:58.836417  8796 leader_election.cc:403] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:58.835098  8747 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:03:58.761479 13474 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:58.688683 13061 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.690171 13360 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.836820 13315 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:58.690109 13345 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:58.675268  9388 rpcz_store.cc:269] 0709 13:03:55.684849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.716328 (+ 31479us) service_pool.cc:224] Handling call
0709 13:03:58.651363 (+2935035us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540025ce54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:58.651534 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2745615616}
I20260709 13:03:58.769625  8722 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:58.676433  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:03:58.867996  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 191619us
I20260709 13:03:58.773537  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:58.775691  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:03:58.869479  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 93872.5us
I20260709 13:03:58.870407  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:58.781888 13249 raft_consensus.cc:697] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:03:58.785398 13348 consensus_meta.cc:287] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.724s	user 0.010s	sys 0.000s
I20260709 13:03:58.787174  8679 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:58.787968 13486 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:58.767714 13394 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:58.912577  8676 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:58.683820 13399 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:58.712088  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2101) took 1126 ms. Trace:
I20260709 13:03:58.734157  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.806943  8826 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:58.746227  9385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:03:58.980469  9385 raft_consensus.cc:606] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:58.980949  9385 raft_consensus.cc:614] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:58.816555  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:58.981446  8749 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:03:58.829653  8828 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:58.824642  8578 catalog_manager.cc:5697] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 7 to 9, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 9 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:58.838167  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 65375.2us
W20260709 13:03:58.775676  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 174382us
I20260709 13:03:58.838586  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.838728  9353 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:58.763176  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:58.990113  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:58.990707  8749 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:58.991145  8676 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:58.861992  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:58.812299 13384 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:58.765545 13421 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.877235  8830 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:03:58.997637  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.998529  8830 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:58.909797 13411 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:58.910666  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.831084 13487 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:58.880445  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:58.916224 13394 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.914283  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:59.012686  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:59.014048  8822 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:58.853961 13299 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 8. Current term is 9.
I20260709 13:03:58.975939  8821 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:59.018005  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:58.830945 13191 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:58.977047  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:58.973894  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:58.920296  8678 rpcz_store.cc:276] 0709 13:03:57.585715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.276166 (+690451us) service_pool.cc:224] Handling call
0709 13:03:58.276534 (+   368us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:58.292223 (+ 15689us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.292259 (+    36us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:58.297771 (+  5512us) log.cc:844] Serialized 78 byte log entry
0709 13:03:58.363483 (+ 65712us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:58.710908 (+347425us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.711416 (+   508us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.711664 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.711774 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.711831 (+    57us) raft_consensus.cc:1685] finished
0709 13:03:58.711848 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.712012 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":362241,"prepare.queue_time_us":353176,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":19603,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":34436,"thread_start_us":45609,"threads_started":1,"wal-append.queue_time_us":81087}
I20260709 13:03:58.860343 13340 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:58.995519  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:59.085975  8824 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:03:58.997792  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:59.087553  8752 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:03:58.878562  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 191962us
I20260709 13:03:58.999430  9380 raft_consensus.cc:2368] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13.
I20260709 13:03:58.908756  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:03:59.088567  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 66411.2us
I20260709 13:03:59.089480  9355 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:59.090186  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:59.090476  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:59.090746  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:59.091507  8749 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 6: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:59.091459  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:58.908408 13407 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:59.091555  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:59.002252  8827 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
I20260709 13:03:58.911938 13474 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:59.097678  8753 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:59.005420  8578 catalog_manager.cc:5697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 5 to 7, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 7 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:59.098009  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:58.792552 13382 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:58.911543 13505 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:59.018803  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.020589  8796 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:58.877882 13469 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.102515  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:58.994889 13167 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.866878 13315 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:03:59.001362  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 163580us
I20260709 13:03:59.114658  8829 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:59.100116  8749 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:59.115958  8679 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:58.978729 13398 consensus_queue.cc:1059] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:03:59.102913  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:59.127017 13249 consensus_queue.cc:237] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.004158 13498 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.128111  8756 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
W20260709 13:03:59.103793  8793 leader_election.cc:336] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.933s (SENT)
I20260709 13:03:59.128364 13498 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.129294  8793 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:59.130220 13167 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.112702 13495 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:59.133095  8747 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 16: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:59.133709 13167 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:59.134169 13469 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:59.134871  8751 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:03:59.103185  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:59.040089  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:59.146932  8827 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:03:59.147084  8754 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:03:59.147558  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:59.148373  8827 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:59.149140  8821 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:59.150096  8827 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:59.111304  8823 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:59.023940  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:59.122663  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2115) took 1100 ms. Trace:
I20260709 13:03:59.123379  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:59.001931 13411 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:59.101902 13370 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:03:59.125324  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:03:59.017204 13472 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 13:03:59.084930 13338 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:59.010461 13484 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:59.125671 13191 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.125797  8578 catalog_manager.cc:5697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 4 to 8, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 8 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:03:59.147742  8754 raft_consensus.cc:606] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:59.101768  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:59.150527  8821 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:03:59.251549  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 147770us
I20260709 13:03:59.252727  8830 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
W20260709 13:03:59.253695  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 233225us
I20260709 13:03:59.255146  8678 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:59.200141  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.256533  8679 rpcz_store.cc:276] 0709 13:03:58.022284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.767045 (+744761us) service_pool.cc:224] Handling call
0709 13:03:58.787037 (+ 19992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.121994 (+334957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.122572 (+   578us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.253747  8578 catalog_manager.cc:5697] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 5 to 7, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 7 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
W20260709 13:03:59.107785 13428 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.000s
I20260709 13:03:59.151475  9355 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:59.102157 13348 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:59.168736 13299 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:59.263837 13411 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:59.275224  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 56935.5us
I20260709 13:03:59.261806  8826 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:59.021620 13437 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:59.218587 13468 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.627s	user 0.011s	sys 0.000s
I20260709 13:03:59.329905  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:59.330240 13468 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:59.318092  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 229752us
I20260709 13:03:59.019861 13421 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:59.324388  8825 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:59.019038 13394 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:59.167042 13474 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:59.324819  8829 raft_consensus.cc:2376] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13. Re-sending same reply.
I20260709 13:03:59.330778  8826 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:03:59.330186 13437 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.167940 13435 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:59.322252  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 170968us
W20260709 13:03:59.319562 13401 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.637s	user 0.013s	sys 0.000s
I20260709 13:03:59.099810 13340 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:03:59.343536  8722 leader_election.cc:336] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Timed out: RequestConsensusVote RPC to 127.7.89.132:41749 timed out after 1.853s (SENT)
I20260709 13:03:59.344177  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:59.316498 13497 raft_consensus.cc:3055] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:59.344538  8578 catalog_manager.cc:5697] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 12, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 12 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:03:59.317742 13484 raft_consensus.cc:697] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:59.348006  8673 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:03:59.266451 13061 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:59.333127 12656 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:59.312224 13347 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:03:59.348676  8722 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:59.356173  8754 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:03:59.364817  8754 raft_consensus.cc:606] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:59.335124 13338 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:59.357808  8826 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:59.369338  8826 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:59.371335  8722 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:59.320019 13348 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.352187 13416 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.379005 13360 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:59.380406 13340 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:59.380743 13486 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:59.354405 13370 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:59.381747 13167 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:59.379076 13416 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.383543 13498 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:59.385886 13360 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:59.351336 13488 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:59.334529 13441 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:03:59.387145  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:59.387308  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:59.388051 13474 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.389967 13447 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:59.390285  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:59.348923 13497 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:03:59.400229  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.318851 13428 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.383641 13399 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:59.370260  8826 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.169461 13315 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.400169  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:59.410061  8578 catalog_manager.cc:5697] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 4 to 5, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 5 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
W20260709 13:03:59.438944  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 160506us
I20260709 13:03:59.402868 13061 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:03:59.405336 13435 raft_consensus.cc:3055] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:59.420451 13437 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:03:59.468003  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.436895 13469 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:59.437001 13348 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:59.473867  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 827) took 1179 ms. Trace:
I20260709 13:03:59.474478  8677 rpcz_store.cc:276] 0709 13:03:58.294435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.801845 (+507410us) service_pool.cc:224] Handling call
0709 13:03:58.802192 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:58.818190 (+ 15998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.818255 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:58.859939 (+ 41684us) log.cc:844] Serialized 26 byte log entry
0709 13:03:59.308662 (+448723us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:59.308723 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.308830 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:59.473494 (+164664us) raft_consensus.cc:1685] finished
0709 13:03:59.473531 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.473774 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1825,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":110,"thread_start_us":313474,"threads_started":1,"wal-append.queue_time_us":560132}
I20260709 13:03:59.475539 13338 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.467108 13484 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.495855  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:59.497125  8822 raft_consensus.cc:2368] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
I20260709 13:03:59.498147  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:59.490885  8578 catalog_manager.cc:5697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 10, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 10 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:03:59.499167  8822 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:03:59.493765  8678 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:03:59.505455  8678 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:59.506352  8826 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:59.509687  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:59.510389  8822 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:59.504588 13076 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.698s	user 0.013s	sys 0.000s
I20260709 13:03:59.470649 13435 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:59.606393  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:03:59.617570  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:03:59.506470 13433 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.012s	sys 0.012s
W20260709 13:03:59.650239  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.607710 13360 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.639714 13167 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:59.643610 13433 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.013s	sys 0.012s
I20260709 13:03:59.607826 13437 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:59.651338  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 315768us
I20260709 13:03:59.640316 13370 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:59.663560  8822 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:03:59.661299  9353 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:03:59.661458  8578 catalog_manager.cc:5697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:03:59.652246 13348 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.657207  8826 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:59.647922 13521 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:59.665282  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:03:59.661074 13340 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.616276 13076 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.652295 13447 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:03:59.657727 13167 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:03:59.701555 13167 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:59.643831 13497 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.703521 13497 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:59.704398 13076 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:03:59.705783 12640 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:03:59.706359 12640 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.706888 12640 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.707506 13428 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:59.709492  8679 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:59.695698  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:59.674542 13437 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.708475  8578 catalog_manager.cc:5697] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:59.720943  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 75988.2us
I20260709 13:03:59.724514  8821 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 12: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:59.726917  8578 catalog_manager.cc:5697] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 5 to 9, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 9 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:03:59.660766 13360 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.711608  8678 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:59.733404  8578 catalog_manager.cc:5697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 3 to 8, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 8 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:59.738117  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:59.741823  8756 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:59.744351  8756 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:59.708370 13249 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:59.724336 13480 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.924s	user 0.011s	sys 0.004s
I20260709 13:03:59.749557  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:59.751613  8678 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:59.751924  8821 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 12: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:59.752924  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:59.753255  8678 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:59.756101  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:03:59.681233 13348 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.748721  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2089) took 1159 ms. Trace:
I20260709 13:03:59.767076  8828 rpcz_store.cc:276] 0709 13:03:58.590082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.595688 (+  5606us) service_pool.cc:224] Handling call
0709 13:03:59.748532 (+1152844us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b540016bf54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:59.748655 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322234240}
I20260709 13:03:59.684577 13521 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.754739 13347 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:03:59.708645 13340 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.778393 12640 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:59.676090 13435 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.759611 13360 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:59.724112 13523 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:59.779727 13340 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:59.780220 13523 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:03:59.781490 13348 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:59.781324 13523 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.782045 13411 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.768719  9355 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:03:59.746505  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:59.784539  8826 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:03:59.769827  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:59.774753  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:59.768603  8578 catalog_manager.cc:5697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 4 to 8, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 8 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:03:59.730243 13327 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:03:59.793107  8754 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:59.789981  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:59.790747  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:03:59.795305  8821 raft_consensus.cc:2410] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:03:59.790318  8578 catalog_manager.cc:5697] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 2 to 5, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 5 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:03:59.791853  8749 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:03:59.767165 13480 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.796559  8754 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:59.796804 13411 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:59.801429  8676 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:59.808012  8750 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:03:59.815209 13338 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 13:03:59.792584 13340 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:03:59.818759 13327 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:59.819310 13340 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.822151 13340 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:59.824796 13437 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:59.798775 13480 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:59.789417 13476 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:59.828557 13521 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:59.829057 13061 raft_consensus.cc:3055] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:59.770596 13394 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:03:59.829473 13061 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:03:59.831615 13394 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:03:59.832623  8681 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:03:59.836066  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:03:59.836877  8826 raft_consensus.cc:2376] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11. Re-sending same reply.
W20260709 13:03:59.809571 13526 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:03:59.851733  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 64716.2us
I20260709 13:03:59.830470 13061 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:03:59.860414  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.868151  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:59.876662  9387 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:59.877017  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:59.877997  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:59.880638  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:59.859987 13411 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.882705  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:59.873786  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:03:59.886647  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:03:59.889936  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:59.893477  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 975251us
I20260709 13:03:59.894163  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:59.887768  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:03:59.900372  8647 connection.cc:256] RPC call timeout handler was delayed by -1.2909071300000505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2416s for a response
I20260709 13:03:59.882371 13441 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:03:59.903890 13441 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:59.904359 13441 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:03:59.905373 13441 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.886227 13061 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:59.886727 12640 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:59.911403 12640 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:03:59.911883 12640 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:59.912245  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 837) took 1202 ms. Trace:
I20260709 13:03:59.912698  8676 rpcz_store.cc:276] 0709 13:03:58.709693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.282244 (+572551us) service_pool.cc:224] Handling call
0709 13:03:59.315022 (+ 32778us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.801990 (+486968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.911915 (+109925us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b54001f9854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:03:59.912164 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1112942976}
I20260709 13:03:59.882761 13340 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:59.898787  9388 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:03:59.869536 13167 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.921969 13382 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:59.922760 13167 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:03:59.925804  8722 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:59.910003 13474 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:59.928359 13441 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:59.947999  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:03:59.882191 13411 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:03:59.885268 13437 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:03:59.975932  8830 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:59.940135  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:03:59.945648 13521 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:59.981243  8828 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:03:59.981752  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 1.16506e+06us
I20260709 13:03:59.969163 13360 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:59.958485 13167 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:59.994019  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 53943.8us
I20260709 13:03:59.996470  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2596) took 1236 ms. Trace:
I20260709 13:04:00.002463  8750 rpcz_store.cc:276] 0709 13:03:58.759761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.773168 (+ 13407us) service_pool.cc:224] Handling call
0709 13:03:59.995963 (+1222795us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b540031cc54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:03:59.996358 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2103623040}
I20260709 13:03:59.982990  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:03:59.969545 13487 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:00.005800  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:03:59.958323 13340 consensus_queue.cc:1059] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:04:00.013674  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:00.015338  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:00.016587 13476 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:00.021260 13542 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:00.021301 13476 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.022029 13542 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.024451 13476 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:03:59.983255 13289 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.021469 13481 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:59.992956 13492 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:00.018185  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:00.028978 13521 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.029241 13487 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.051494  8749 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:04:00.032675  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.997375 13360 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.002358 13401 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:00.046423  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:00.048101  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:00.050006  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:00.031774  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:00.032013 13481 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.062470  8750 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:00.024717 13542 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:00.064260  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:00.045058 13289 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.065783  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.070497  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:00.092684  8676 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:00.079530  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:00.044759 13394 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:00.083630  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:00.069741  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 834) took 1360 ms. Trace:
I20260709 13:04:00.095224  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:00.095345  8679 rpcz_store.cc:276] 0709 13:03:58.708980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.259140 (+550160us) service_pool.cc:224] Handling call
0709 13:03:59.316686 (+ 57546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.709991 (+393305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.069399 (+359408us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b540033aed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:00.069653 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":902744960}
I20260709 13:04:00.096346  8830 raft_consensus.cc:2410] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:04:00.097875  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:00.095325  8826 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:04:00.099918  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:00.054751 13476 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.059108 13167 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 13:04:00.102530 13533 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:04:00.102312 13476 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.080688 13411 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:00.104187 13394 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.104457  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.104904 13476 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:00.105512 13479 consensus_queue.cc:1059] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:00.105816  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:00.106418  8676 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:00.109395  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.111054  8750 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:00.058147 13360 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:00.066141 13428 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:04:00.055320 13521 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:00.106431 13535 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:04:00.066390 13469 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:04:00.116756  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2196) took 1026 ms. Trace:
I20260709 13:04:00.116892 13315 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:00.117473  8756 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:00.066913 13527 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:00.119509  8673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2117) took 1786 ms (client timeout 2061 ms). Trace:
W20260709 13:04:00.141623  8673 rpcz_store.cc:269] 0709 13:03:58.332956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.369091 (+ 36135us) service_pool.cc:224] Handling call
0709 13:04:00.119252 (+1750161us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b54002a99d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:00.119418 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2248785664}
I20260709 13:04:00.143546  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.144526  8673 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 5.
I20260709 13:04:00.144061  8794 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:00.120697 13532 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:04:00.047510 12640 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.060423 13407 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:00.068061 13447 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:04:00.104846 13394 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:00.068017 13481 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:00.132139  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2103) took 1030 ms. Trace:
I20260709 13:04:00.193202  8829 rpcz_store.cc:276] 0709 13:03:59.101951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.123036 (+ 21085us) service_pool.cc:224] Handling call
0709 13:04:00.131540 (+1008504us) spinlock_profiling.cc:243] Waited 70.4 ms on lock 0x7b54003dead4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:00.132017 (+   477us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161820032}
W20260709 13:04:00.136840  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 105175us
I20260709 13:04:00.195129  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:00.195356  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:00.137173  8747 rpcz_store.cc:276] 0709 13:03:59.090065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.090233 (+   168us) service_pool.cc:224] Handling call
0709 13:04:00.116486 (+1026253us) spinlock_profiling.cc:243] Waited 41.7 ms on lock 0x7b54002fdad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:00.116683 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96000768}
I20260709 13:04:00.067698 13305 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:00.082849 13401 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:00.145808  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.203078  8673 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:04:00.205103  8673 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:04:00.205535  8673 pending_rounds.cc:85] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 7
I20260709 13:04:00.159050  8676 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:00.208685  8676 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:00.171224 13433 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:00.146786 13491 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.176689 13527 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:00.182755  9380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 731) took 1686 ms (client timeout 2156 ms). Trace:
W20260709 13:04:00.221185  9380 rpcz_store.cc:269] 0709 13:03:58.496695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.497183 (+   488us) service_pool.cc:224] Handling call
0709 13:04:00.182380 (+1685197us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b540064e954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:00.182658 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1135429376}
I20260709 13:04:00.187062  9385 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:00.191185  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.125948 13345 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:00.226279  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 828) took 1931 ms. Trace:
I20260709 13:04:00.148241 13076 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:00.126001 13289 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:00.232059  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2107) took 2218 ms. Trace:
I20260709 13:04:00.205811  8673 pending_rounds.cc:107] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:04:00.233094  8673 raft_consensus.cc:2889] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:00.234844  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:00.174057  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:00.210559  8676 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:00.236343  8750 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:00.169068 13497 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.224758  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 793) took 1343 ms. Trace:
I20260709 13:04:00.232230  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:00.192994 13484 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.218257  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2102) took 2632 ms. Trace:
I20260709 13:04:00.169001 13488 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
W20260709 13:04:00.248454  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 248190us
I20260709 13:04:00.226859  8672 rpcz_store.cc:276] 0709 13:03:58.294699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.802748 (+508049us) service_pool.cc:224] Handling call
0709 13:03:58.803060 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:58.803423 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:58.803762 (+   339us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.048629 (+1244867us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:00.050956 (+  2327us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.051206 (+   250us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:00.180657 (+129451us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.225851 (+ 45194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.225954 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:00.226013 (+    59us) raft_consensus.cc:1685] finished
0709 13:04:00.226033 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.226198 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":934197,"prepare.queue_time_us":20504,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":959611,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":9362,"thread_start_us":489514,"threads_started":3,"wal-append.queue_time_us":540362}
I20260709 13:04:00.239415  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:00.245343  8674 rpcz_store.cc:276] 0709 13:03:57.586010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.494635 (+908625us) service_pool.cc:224] Handling call
0709 13:03:58.495153 (+   518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:58.599674 (+104521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.599708 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:58.600232 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 13:03:58.713376 (+113144us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b1800471160. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:03:58.995199 (+281823us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:59.092907 (+ 97708us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.252304 (+159397us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b1800471160. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32e20afd70 00007f32e20a3293 00007f32e2098cea 00007f32e209923a 00007f32e2503a2a 00007f32e25130bf 00007f32e2513038
0709 13:04:00.217809 (+965505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.217934 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:00.217987 (+    53us) raft_consensus.cc:1685] finished
0709 13:04:00.218004 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.218180 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":964642,"prepare.queue_time_us":101850,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":47471,"raft.queue_time_us":869387,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":18592,"spinlock_wait_cycles":54000384,"thread_start_us":488705,"threads_started":2,"wal-append.queue_time_us":461724}
I20260709 13:04:00.251151  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2104) took 1086 ms. Trace:
I20260709 13:04:00.251523  8827 rpcz_store.cc:276] 0709 13:03:59.164671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.239093 (+1074422us) service_pool.cc:224] Handling call
0709 13:04:00.251081 (+ 11988us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:00.240877  8751 rpcz_store.cc:276] 0709 13:03:58.881405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.976681 (+ 95276us) service_pool.cc:224] Handling call
0709 13:04:00.224556 (+1247875us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b54002fe4d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:00.224688 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52502784}
I20260709 13:04:00.241436  9380 raft_consensus.cc:2410] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:04:00.174474 13411 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:04:00.237169  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:00.252787  8647 connection.cc:256] RPC call timeout handler was delayed by -1.1176897999999937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2001s for a response
W20260709 13:04:00.256479  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.179801 13486 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:04:00.258175  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:00.261374  8647 connection.cc:256] RPC call timeout handler was delayed by -1.0921587000000272s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.202s for a response
I20260709 13:04:00.261984  8672 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:00.261384  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:00.262461  8647 connection.cc:256] RPC call timeout handler was delayed by -1.0066587000000027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21150000000000002s for a response
I20260709 13:04:00.229966 13061 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:00.239143 13076 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.272094  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1961) took 1358 ms. Trace:
I20260709 13:04:00.230432 13457 raft_consensus.cc:3055] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:00.216332 13472 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
W20260709 13:04:00.275864  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 47782.9us
I20260709 13:04:00.293632  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:00.244017 13544 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:00.278193 13289 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:00.278473  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:00.280426  8825 rpcz_store.cc:276] 0709 13:03:58.913537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.913722 (+   185us) service_pool.cc:224] Handling call
0709 13:03:59.324113 (+410391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.338436 (+ 14323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.271800 (+933364us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b540009ddd0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:00.272008 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":942928640}
I20260709 13:04:00.211717 13167 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:04:00.215560 13305 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.321532  8645 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:00.321553  8827 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:00.324486  8751 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:04:00.324888  8825 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:00.239768 13547 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:00.314937  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:00.238039  8680 rpcz_store.cc:276] 0709 13:03:58.013290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.651614 (+638324us) service_pool.cc:224] Handling call
0709 13:03:58.651939 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.652338 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:59.102660 (+450322us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.219605 (+1116945us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.231553 (+ 11948us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:00.231633 (+    80us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:00.231707 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.231802 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.231978 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":647631,"prepare.queue_time_us":61510,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":624656,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":6579,"thread_start_us":929925,"threads_started":3,"wal-append.queue_time_us":542178}
I20260709 13:04:00.332654  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:00.316962  8755 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:04:00.320282  8794 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:00.298946  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 103935us
I20260709 13:04:00.327906  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:00.333901  8825 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:00.224761 13491 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:00.330099  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51745.6us
I20260709 13:04:00.330605  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:00.335086  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:00.279418 13061 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:00.290719 13076 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:00.250159 13497 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:00.299144 13519 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:04:00.343817  8678 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:00.342375  8719 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:00.339556  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 820) took 3961 ms. Trace:
I20260709 13:04:00.343168  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.356176  8719 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:00.348387  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:00.298061 13541 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:00.288444 13457 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:00.351089  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:00.357754  8680 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:04:00.255801 13484 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.358985  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:00.340137  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:00.211969 13437 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:04:00.329206 13417 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:00.330549 13447 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:00.377820  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:00.353560 13061 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:00.378736  8827 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:04:00.378397  8647 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:00.368957  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 54758.7us
I20260709 13:04:00.370985  8828 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:00.381778  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:00.372282  8674 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:00.353612  8675 rpcz_store.cc:276] 0709 13:03:56.378283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.384589 (+  6306us) service_pool.cc:224] Handling call
0709 13:03:58.341654 (+1957065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.341925 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:03:58.710249 (+368324us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.917485 (+207236us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.892644 (+975159us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b18002f7ac0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:03:59.892879 (+   235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.892954 (+    75us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:59.893028 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.893133 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.339216 (+446083us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b54000dacd0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:00.339453 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1402711,"prepare.queue_time_us":372096,"prepare.run_cpu_time_us":168,"prepare.run_wall_time_us":14234,"raft.queue_time_us":1005895,"raft.run_cpu_time_us":401,"raft.run_wall_time_us":84531,"spinlock_wait_cycles":4472272896,"thread_start_us":199156,"threads_started":1,"wal-append.queue_time_us":307765,"wal-append.run_cpu_time_us":1805,"wal-append.run_wall_time_us":1158025}
I20260709 13:04:00.357090 13542 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:00.355172 13472 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:00.372150  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.386562  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:00.361024 13543 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:00.323732 13289 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.290244 13407 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.368618 13561 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:00.403471  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:00.376837 13488 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:00.406702  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 50227.8us
I20260709 13:04:00.407073  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:00.375893 13539 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 13:04:00.384191 13480 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:00.369086 13550 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:00.402628  8680 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:00.318921 13544 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:00.403498  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 61485.3us
I20260709 13:04:00.320132 13401 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.422488  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.423779  8680 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:00.424315  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:00.417356  8827 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:00.426159  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:00.402311 13417 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:00.388667 13399 raft_consensus.cc:2764] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:04:00.412106  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:00.409922 13543 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.394912 13542 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:00.401930 13411 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:00.388907 13472 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:00.429589  8722 leader_election.cc:400] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13.
I20260709 13:04:00.408928  8825 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:00.430732  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.445815  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.442890  8828 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:00.384665 13447 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.432736 13556 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:00.376590 13345 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:04:00.404966 13348 consensus_queue.cc:1059] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:04:00.407953 13469 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:00.422304  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:00.371692 13541 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.441804  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:00.435434 13417 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:00.440536 13542 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:00.428481 13167 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:00.371037 13545 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:04:00.455025  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:00.459095 13399 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:00.456002  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.438431 12640 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:00.461884 13469 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.350550 13527 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.418284  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.463302  8827 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:04:00.440711 13411 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.448593 13472 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:00.496210  8675 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:00.441308 13543 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:00.448907 13480 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:00.419216 13488 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.478282  8823 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:00.481609  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:00.487324  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:00.510242  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.485526  8828 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:00.434333 13305 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:00.492873 13551 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:04:00.468192 13417 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:00.515569  8828 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:04:00.494946 13544 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:00.494413 13249 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:00.531605 13447 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:00.474069 13536 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:04:00.421007 13561 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.533849  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 876) took 1518 ms. Trace:
I20260709 13:04:00.534260  8824 rpcz_store.cc:276] 0709 13:03:59.015546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.848002 (+832456us) service_pool.cc:224] Handling call
0709 13:03:59.848856 (+   854us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.849344 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.849386 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:59.852344 (+  2958us) log.cc:844] Serialized 26 byte log entry
0709 13:03:59.987897 (+135553us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:59.987960 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.988065 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:00.533613 (+545548us) raft_consensus.cc:1685] finished
0709 13:04:00.533637 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.533782 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":868,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":2321,"thread_start_us":127246,"threads_started":1,"wal-append.queue_time_us":212425}
I20260709 13:04:00.469024 13167 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.509213  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.539824  8675 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:00.540896  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:00.541347  8675 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:00.512404  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:00.434581 13457 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.512792  8680 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:00.521464 13558 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:00.522787 13289 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:00.489738 13399 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:00.489665 13521 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:00.527601 13557 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:00.528237 13541 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:00.494427 13568 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:04:00.497110 13469 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:00.537776  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:00.560019 13561 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:00.526332 13480 raft_consensus.cc:697] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:00.560510  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:00.590756 13249 consensus_queue.cc:1059] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
W20260709 13:04:00.562808  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 181008us
W20260709 13:04:00.563833  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 160455us
I20260709 13:04:00.507787 13411 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:00.590278  9388 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:04:00.590584 13382 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:00.495932 13567 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.557454 13167 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:00.604548  8680 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:00.559471 13399 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:00.538555  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 182451us
I20260709 13:04:00.614858  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:00.595795 13548 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:04:00.613830 13491 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:00.622203  8647 leader_election.cc:400] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
I20260709 13:04:00.656527  8647 leader_election.cc:403] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:04:00.625337  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 87678.2us
W20260709 13:04:00.619628  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 102953us
I20260709 13:04:00.595498 13437 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.508555 13564 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 13:04:00.594916 13497 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:00.648504  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.649283  8719 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:00.593734  8829 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:00.654235  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:00.686848  8828 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:00.660120  8793 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:00.689541  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:00.690394  9355 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:00.658190  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:00.625648 13484 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:00.670759 13566 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:00.671600 13560 raft_consensus.cc:3055] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:00.674566 13394 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.675666 13191 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:00.677635 13433 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:00.678794 13545 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10.
I20260709 13:04:00.679062 13249 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:00.646042 13561 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:00.700134  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:00.650455 13441 consensus_meta.cc:287] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.686s	user 0.019s	sys 0.000s
I20260709 13:04:00.674264 13559 raft_consensus.cc:3055] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:00.713912  8829 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:00.569701 13558 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.703195 13448 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:00.711786 13484 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:00.721087 13345 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:04:00.738492  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 115649us
I20260709 13:04:00.704653 13560 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:00.707182 13556 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:00.727840 13561 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.731922 13521 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:00.701804 13437 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.768659 13417 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.733652 13559 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:00.739677 13448 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:00.762046 13558 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:00.763417 13575 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:04:00.742535 13484 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:00.763984 13563 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.133s	user 0.001s	sys 0.001s
I20260709 13:04:00.767889 13570 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:04:00.651294 13567 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.778481 13448 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:00.758351 13561 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:00.707821 13566 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.780656 13484 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:00.789086 13567 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:00.789943 13559 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.741882 13472 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.790815 13488 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:00.734584 13480 consensus_queue.cc:237] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:00.798245 13481 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.583s	user 0.003s	sys 0.008s
I20260709 13:04:00.806176 13542 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.800730 13560 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.810355 13560 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:00.811549 13543 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:00.812927 13559 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:00.815358 13566 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:00.698173  8675 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
W20260709 13:04:00.746909  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.867604  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:00.869925  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:00.708370  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 59181.7us
I20260709 13:04:00.729139  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:00.886432  8751 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:04:00.870499  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.920480  8750 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:00.901557  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:00.896229 13370 raft_consensus.cc:993] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: : Instructing follower e814ed2013244fa6be42623ea5c0c7b9 to start an election
I20260709 13:04:00.954823 13545 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.955476 13370 raft_consensus.cc:1081] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Signalling peer e814ed2013244fa6be42623ea5c0c7b9 to start an election
I20260709 13:04:00.879733  9389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 736) took 1004 ms. Trace:
I20260709 13:04:00.954864 13521 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.960282  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:00.960995 13521 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.962397 13566 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.963601 13566 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:00.968397 13360 consensus_meta.cc:287] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.821s	user 0.012s	sys 0.006s
I20260709 13:04:00.969359 13360 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:00.757628 13345 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:00.928165 13441 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.895601  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:00.915324 13407 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:00.906468 13417 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:00.924814 13472 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:00.919458 13437 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:00.956197 13545 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:00.878691  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 207479us
I20260709 13:04:00.919282 13481 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.018173  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:01.019624 13345 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.023694  8753 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:01.024935  8672 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:04:01.026876  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:01.028461  8672 raft_consensus.cc:2410] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 13:04:01.030850  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 301613us
I20260709 13:04:01.032685  8793 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:01.034170  8676 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:01.020517  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:01.035503  8674 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:04:00.994513 13398 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.001s	user 0.010s	sys 0.011s
I20260709 13:04:01.003425 13407 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:01.042431  8755 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 14: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:00.975564 13441 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:01.041337 13398 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.057s	user 0.010s	sys 0.013s
I20260709 13:04:01.051394 13394 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.040087 13481 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:00.798168 13469 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:01.064153  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.042462 13407 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:01.056466 13566 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:01.074723  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:00.959257  9389 rpcz_store.cc:276] 0709 13:03:59.875450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.875586 (+   136us) service_pool.cc:224] Handling call
0709 13:03:59.885678 (+ 10092us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.964997 (+ 79319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.965057 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:59.981470 (+ 16413us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.179238 (+197768us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:00.179278 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.181888 (+  2610us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:00.878844 (+696956us) raft_consensus.cc:1685] finished
0709 13:04:00.878906 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.879382 (+   476us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16374,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"thread_start_us":238234,"threads_started":2,"wal-append.queue_time_us":250641}
I20260709 13:04:01.093849 13567 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:01.095047 13521 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:01.097446  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 65059.8us
I20260709 13:04:01.098819  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:01.101275  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:01.104321  9385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 732) took 1232 ms. Trace:
I20260709 13:04:01.104909  9385 rpcz_store.cc:276] 0709 13:03:59.871539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.871884 (+   345us) service_pool.cc:224] Handling call
0709 13:04:00.186753 (+314869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.222931 (+ 36178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.103742 (+880811us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b540064e950. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:01.104059 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":722900736}
I20260709 13:04:01.100772 13411 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.077874  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:01.106684 13560 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:01.121402  9383 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:04:01.106773 13541 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:04:01.132082 13521 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:01.132836 13521 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:01.132432 13448 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.133553 13521 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:01.134701 13488 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:01.135942  8830 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:01.146731 13555 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.147584 13555 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.120260 13560 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:01.138281 13167 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.161712 13437 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.162786 13437 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.165738 13437 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.159807 13167 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.170928  8647 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:01.172145 13398 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:01.174803 13360 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:01.179901  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 52011.9us
W20260709 13:04:01.183024  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 81787.2us
I20260709 13:04:01.184438  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:01.186743  8821 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
W20260709 13:04:01.210276  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 86693.8us
I20260709 13:04:01.190665 13560 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:01.182392  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:01.187960 13465 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:01.187891 13545 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.186554 13468 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.129347 13566 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 13:04:01.224920  8672 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:01.237547  8647 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:01.243119 13061 consensus_meta.cc:287] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.792s	user 0.005s	sys 0.020s
I20260709 13:04:01.248895  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:01.248629 13061 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.249971  8748 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
W20260709 13:04:01.252331 13076 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.891s	user 0.011s	sys 0.012s
W20260709 13:04:01.246117  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.255129 13468 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:01.256815  8672 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:01.257663  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2136) took 1014 ms. Trace:
I20260709 13:04:01.259407 13411 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:01.263085 13448 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:01.253036  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 70919.6us
I20260709 13:04:01.265524  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:01.265908  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 71595us
I20260709 13:04:01.267107  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:01.267729 13523 raft_consensus.cc:3055] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:01.268214 13523 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:01.269191 13566 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:01.269760 13566 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:01.270260 13566 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:01.239022 13419 raft_consensus.cc:3055] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:01.274799 13419 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:01.276618 13419 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.277733 13491 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:01.239377 13561 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.258020  8672 rpcz_store.cc:276] 0709 13:04:00.243152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.241054 (+997902us) service_pool.cc:224] Handling call
0709 13:04:01.256459 (+ 15405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.257360 (+   901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.257575 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:01.291108  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:01.294337  8822 raft_consensus.cc:2393] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:01.278297 13491 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:01.273874 13606 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:01.300115 13441 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:01.300056 13061 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:01.301038  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2133) took 1086 ms. Trace:
I20260709 13:04:01.301604  8674 rpcz_store.cc:276] 0709 13:04:00.214124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.193629 (+979505us) service_pool.cc:224] Handling call
0709 13:04:01.234592 (+ 40963us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.300516 (+ 65924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.300555 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:01.300646 (+    91us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:01.300683 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.300756 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.300922 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:01.305589  8752 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:01.306535 13484 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:01.308452 13399 consensus_meta.cc:287] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.645s	user 0.000s	sys 0.029s
I20260709 13:04:01.309476 13484 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.313848  8749 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:01.253046 13076 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:01.314790  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.315519 13491 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:01.315826  8673 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:01.316998  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2124) took 2326 ms. Trace:
I20260709 13:04:01.317507  8678 rpcz_store.cc:276] 0709 13:03:58.990801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.754033 (+763232us) service_pool.cc:224] Handling call
0709 13:03:59.754378 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.352792 (+598414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.316330 (+963538us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b540033a4d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:01.316679 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1354227584}
I20260709 13:04:01.318166  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2135) took 1089 ms. Trace:
W20260709 13:04:01.309109  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 85203.2us
I20260709 13:04:01.335570  8673 rpcz_store.cc:276] 0709 13:04:00.228455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.234442 (+  5987us) service_pool.cc:224] Handling call
0709 13:04:01.317862 (+1083420us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003a20d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:01.318077 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2482555648}
I20260709 13:04:01.336356  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:01.336901 13582 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.670s	user 0.003s	sys 0.000s
I20260709 13:04:01.337556  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:01.339433  8755 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:01.330487 13521 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:01.310212 13315 consensus_meta.cc:287] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.162s	user 0.016s	sys 0.009s
I20260709 13:04:01.342919 13398 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:01.343775 13521 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:01.344110 13556 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.344528 13345 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:01.344740 13556 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.346242 13345 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.347780  8672 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:01.348508  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2552) took 1015 ms. Trace:
I20260709 13:04:01.348835  8672 rpcz_store.cc:276] 0709 13:04:00.333339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.347112 (+1013773us) service_pool.cc:224] Handling call
0709 13:04:01.347594 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.348219 (+   625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.348424 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:01.269404 13523 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.353307  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2118) took 1288 ms. Trace:
I20260709 13:04:01.353932  8830 rpcz_store.cc:276] 0709 13:04:00.064480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.097356 (+ 32876us) service_pool.cc:224] Handling call
0709 13:04:01.352996 (+1255640us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b540016b054. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:01.353196 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466792832}
I20260709 13:04:01.309098 13407 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.340510  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:01.327073 13469 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.331066  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2137) took 1073 ms. Trace:
I20260709 13:04:01.333103 13543 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.331265 13076 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.341408  8793 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:01.309373 13399 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.353394 13561 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:01.319692  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 829) took 3024 ms. Trace:
I20260709 13:04:01.328045  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2606) took 1554 ms. Trace:
W20260709 13:04:01.364810  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 129474us
I20260709 13:04:01.366285  8676 rpcz_store.cc:276] 0709 13:04:00.257578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.261086 (+  3508us) service_pool.cc:224] Handling call
0709 13:04:01.330758 (+1069672us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b5400299d54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:01.330983 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1770132352}
W20260709 13:04:01.371351  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2109) took 1673 ms (client timeout 1592 ms). Trace:
I20260709 13:04:01.383729  8756 rpcz_store.cc:276] 0709 13:03:59.773565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.773738 (+   173us) service_pool.cc:224] Handling call
0709 13:04:00.117265 (+343527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.138527 (+ 21262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.327538 (+1189011us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b54002fdad0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:01.327806 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":789094784}
W20260709 13:04:01.388980  8822 rpcz_store.cc:269] 0709 13:03:59.621677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.670791 (+ 49114us) service_pool.cc:224] Handling call
0709 13:04:01.280094 (+1609303us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:04:01.295097 (+ 15003us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8195456}
I20260709 13:04:01.383134  8681 rpcz_store.cc:276] 0709 13:03:58.294946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.806284 (+511338us) service_pool.cc:224] Handling call
0709 13:03:58.806679 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.835762 (+1029083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.319132 (+1483370us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54003cf854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:01.319475 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2358386048}
I20260709 13:04:01.396816  8672 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: []
W20260709 13:04:01.399130  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 58730.6us
I20260709 13:04:01.395637  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:01.403715  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2554) took 1064 ms. Trace:
I20260709 13:04:01.404275  8672 rpcz_store.cc:276] 0709 13:04:00.333908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.349662 (+1015754us) service_pool.cc:224] Handling call
0709 13:04:01.350155 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.397510 (+ 47355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.397806 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:01.405653  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:01.294303 13468 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 13:04:01.418850  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:01.423157  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 43686.6us
I20260709 13:04:01.426510  8748 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:01.412976 13484 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:01.428869 13553 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.029s	user 0.008s	sys 0.000s
I20260709 13:04:01.427889 13469 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.432204  8676 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:01.431851 13469 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.433136  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2559) took 1096 ms. Trace:
I20260709 13:04:01.433588  8676 rpcz_store.cc:276] 0709 13:04:00.336706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.413152 (+1076446us) service_pool.cc:224] Handling call
0709 13:04:01.431877 (+ 18725us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.432770 (+   893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.433060 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:01.434245  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2617) took 1335 ms. Trace:
I20260709 13:04:01.426719  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:01.434736  8754 rpcz_store.cc:276] 0709 13:04:00.098504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.098712 (+   208us) service_pool.cc:224] Handling call
0709 13:04:00.099361 (+   649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:00.102517 (+  3156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.102553 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:00.103137 (+   584us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.254751 (+151614us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:00.254818 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.254917 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.274061 (+1019144us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:01.433689 (+159628us) raft_consensus.cc:1685] finished
0709 13:04:01.433715 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.434035 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3690,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":8500992,"thread_start_us":151098,"threads_started":1,"wal-append.queue_time_us":215318}
I20260709 13:04:01.437301 13586 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.438076 13399 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.438953 13399 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:01.438097 13586 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.439340 13399 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.439401  8823 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:01.439806 13399 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:01.440492  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:01.430789 13571 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:01.448036 13437 consensus_queue.cc:1059] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:01.448541 13167 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:01.450016 13437 raft_consensus.cc:3055] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:01.430160  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:01.443004 13338 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:01.417938 13465 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:04:01.456557  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.463780 13491 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.465271  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:01.459120 13191 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:01.468161 13191 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.468659 13191 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:01.468806 13407 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:01.460641  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2142) took 1147 ms. Trace:
I20260709 13:04:01.450547 13437 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:01.471741 13370 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:01.470798  8673 rpcz_store.cc:276] 0709 13:04:00.313439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.340044 (+1026605us) service_pool.cc:224] Handling call
0709 13:04:01.340486 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.346147 (+  5661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.346184 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:01.346774 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 13:04:01.402998 (+ 56224us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:01.403050 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.403158 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.460307 (+ 57149us) raft_consensus.cc:1685] finished
0709 13:04:01.460333 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.460546 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2847,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":115,"thread_start_us":44051,"threads_started":1,"wal-append.queue_time_us":111106}
I20260709 13:04:01.472273 13370 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.472765 13370 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:01.466282  8679 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:01.474901  9352 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:01.476884  8673 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:01.478895 13484 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.461663  8678 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:01.485314 13416 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.486245  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2557) took 1151 ms. Trace:
I20260709 13:04:01.486665  8678 rpcz_store.cc:276] 0709 13:04:00.336055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.401839 (+1065784us) service_pool.cc:224] Handling call
0709 13:04:01.426223 (+ 24384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.485932 (+ 59709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.486174 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:01.487931  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:01.488921  8678 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:01.489486 13327 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:01.490024 13327 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.490092  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:01.494733  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:01.466944 13465 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:01.463565  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 122331us
W20260709 13:04:01.491030  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 83018.2us
I20260709 13:04:01.491073 12945 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.492033 13613 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:01.472698 13437 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.426487 13345 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:01.535611  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:01.536129  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:01.536706  8756 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:01.493120 13555 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:01.496439 13076 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.538040  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:01.538144 13076 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:01.538976  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 873536us
W20260709 13:04:01.492076 13607 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 13:04:01.541090  9383 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 738) took 1664 ms (client timeout 2151 ms). Trace:
I20260709 13:04:01.498067 13407 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:01.511878  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 71541.2us
I20260709 13:04:01.515339  8793 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:01.490545 13327 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:01.525350  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2141) took 1213 ms. Trace:
I20260709 13:04:01.493155 13547 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:01.531595  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:01.532785  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.532397 12945 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.533708  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:01.541826  9383 rpcz_store.cc:269] 0709 13:03:59.876201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.878747 (+  2546us) service_pool.cc:224] Handling call
0709 13:04:01.540694 (+1661947us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54006275d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:01.540880 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2796972928}
I20260709 13:04:01.497216 13484 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.471338 13491 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.551095 13560 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.529034 13613 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.554265 13315 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:01.555123  8793 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:01.547513  8674 rpcz_store.cc:276] 0709 13:04:00.300276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.302939 (+1002663us) service_pool.cc:224] Handling call
0709 13:04:01.303599 (+   660us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.304206 (+   607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.304238 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:01.304914 (+   676us) log.cc:844] Serialized 26 byte log entry
0709 13:04:01.417140 (+112226us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:01.417220 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.417306 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.513124 (+ 95818us) raft_consensus.cc:1685] finished
0709 13:04:01.513146 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.513321 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1668,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":196,"thread_start_us":111635,"threads_started":1,"wal-append.queue_time_us":194559}
I20260709 13:04:01.568290  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:01.573167  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:01.579027  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 100672us
I20260709 13:04:01.589618  8748 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:01.590440  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:01.591027 12945 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.591552  8674 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:01.591781 12945 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:01.592794  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:01.593657  8674 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:01.589625 13571 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.005s
W20260709 13:04:01.596009  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.597705 13571 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.598589 13527 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.598452 13571 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.599327 13527 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.601924 13527 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.602026 13571 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:01.603212 13606 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:01.607527  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:01.607700 13061 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:01.609759 13061 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:01.610888 13472 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:01.611445 13472 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:01.605756  8674 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:01.612736 13472 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.612872 13191 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.613427  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:01.613988 13472 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:01.614266  8674 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:01.614960 13061 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:01.616145 13191 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.616825 13061 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.619284 13370 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.627632 13560 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:01.629114 13437 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:01.574294 13613 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:01.635238  8674 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:04:01.638203  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:01.639664 13560 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:01.640216 13560 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.640550 13560 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:01.642942  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:01.645188  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 90245.8us
I20260709 13:04:01.669839  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2608) took 1893 ms. Trace:
I20260709 13:04:01.668569  8828 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:01.651705  8756 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:01.678330  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:01.681635  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:01.693272  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:01.677918  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:01.676112  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:01.647639  8674 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:01.733110  8753 rpcz_store.cc:276] 0709 13:03:59.776135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.776292 (+   157us) service_pool.cc:224] Handling call
0709 13:04:00.225174 (+448882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.024405 (+799231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.667796 (+643391us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b54002fe4d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:01.669534 (+  1738us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b54002fe4d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:01.669748 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2866449536}
I20260709 13:04:01.672811  9386 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:58628 (request call id 0) took 1794 ms. Trace:
I20260709 13:04:01.698190  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:01.622332 13370 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.652127  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:01.654090  8681 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:01.704813  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 1987) took 1255 ms. Trace:
I20260709 13:04:01.575107 13315 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:01.693718  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:01.747437  8823 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:01.745644  8749 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:01.741009  9386 rpcz_store.cc:276] 0709 13:03:59.878401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.915536 (+ 37135us) service_pool.cc:224] Handling call
0709 13:04:01.672131 (+1756595us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54003a6454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24ff735 00007f32e24f79fa 00007f32e74bf4c9 00007f32e0a3bc54 00007f32e0a3bbe2 00007f32e0a3bb11 00007f32e0a3ba8c 00007f32e0a3ad42 00007f32e083448c 00007f32e08338c6
0709 13:04:01.672551 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":596000512}
I20260709 13:04:01.677312  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:01.676744  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:01.748526  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 290351us
I20260709 13:04:01.678330  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:01.757215  8755 raft_consensus.cc:2368] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:04:01.767692  9388 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:04:01.653241 13487 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.328s	user 0.005s	sys 0.015s
I20260709 13:04:01.647166 13076 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.679909 13556 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:01.774876  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:01.775725  8674 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:01.782001  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:01.750093  8647 connection.cc:256] RPC call timeout handler was delayed by -1.5900512100000697s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
I20260709 13:04:01.782953  8756 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:01.783906  8674 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:01.778650  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:01.654342 13484 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:01.687963 13586 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.783936  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:01.777129 13487 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.542s	user 0.009s	sys 0.023s
I20260709 13:04:01.654433 13542 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:01.793221  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.654675 13555 consensus_queue.cc:1059] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.122s
I20260709 13:04:01.647296 13399 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.746349 13566 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.803021  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:01.748154  8825 rpcz_store.cc:276] 0709 13:04:00.449557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.449719 (+   162us) service_pool.cc:224] Handling call
0709 13:04:00.450238 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:00.533958 (+ 83720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.534000 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:00.534493 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.713855 (+179362us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:00.713906 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.703977 (+990071us) spinlock_profiling.cc:243] Waited 83.4 ms on lock 0x7b54002297d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:01.704078 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.704168 (+    90us) raft_consensus.cc:1685] finished
0709 13:04:01.704189 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.704558 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191826816,"thread_start_us":178970,"threads_started":1,"wal-append.queue_time_us":225334}
I20260709 13:04:01.808123  8748 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:01.655511 13327 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.807432  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:01.809458  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:01.812548  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:01.814638  8752 raft_consensus.cc:2410] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:04:01.779417 13567 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:01.811597 13351 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.816345 13457 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:01.818017 13399 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:01.822556  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:01.814335  8823 raft_consensus.cc:2376] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10. Re-sending same reply.
I20260709 13:04:01.825026  8748 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:01.806936 13076 raft_consensus.cc:2764] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:04:01.832142  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:01.836148  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 138162us
I20260709 13:04:01.838451  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:01.839653  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:01.843502  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.829928  9353 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:01.846496  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:01.845759  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:01.848881  8680 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:04:01.832070 13351 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.860067 13327 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:01.860497  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.871436  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:01.815547 13560 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.873301  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:01.877463  8756 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:01.826695 13484 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.883726  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:01.885211  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 71272.4us
I20260709 13:04:01.847959  8755 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:01.886685  8825 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:01.892619 13479 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:01.894284  8722 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:01.895390  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:01.895912  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2561) took 1520 ms. Trace:
I20260709 13:04:01.900720  9384 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:01.897372  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:01.886135  8677 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:01.866153 13613 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:01.912518  8826 raft_consensus.cc:2368] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:04:01.909054  8748 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:01.927182  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:01.930629  9383 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:04:01.932055  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:01.903712 13491 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:01.942788  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:01.943363  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:01.943745  8827 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:01.905032 13479 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:01.945514 13479 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.946930 13479 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:01.948715 13625 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:01.948527 13619 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:01.949623 13619 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:01.950035 13619 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:01.950129  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 853) took 1031 ms. Trace:
W20260709 13:04:01.939029  8647 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.132:41749 timed out after 1.666s (SENT) [suppressed 32 similar messages]
I20260709 13:04:01.950639  8673 rpcz_store.cc:276] 0709 13:04:00.919304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.477814 (+558510us) service_pool.cc:224] Handling call
0709 13:04:01.478165 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.525993 (+ 47828us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.526044 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:01.562732 (+ 36688us) log.cc:844] Serialized 26 byte log entry
0709 13:04:01.825619 (+262887us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:01.825686 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.825787 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.949822 (+124035us) raft_consensus.cc:1685] finished
0709 13:04:01.949846 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.950045 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17976,"raft.queue_time_us":43539,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":146,"thread_start_us":202070,"threads_started":1,"wal-append.queue_time_us":319855}
I20260709 13:04:01.954125 13613 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:01.911273 13370 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:04:01.954973 13370 consensus_queue.cc:1243] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:01.955597 13370 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:01.955958 13370 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:01.908171 13567 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:01.959578  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:01.971189  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.971954  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:01.955607  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 61481.4us
W20260709 13:04:01.969821  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:01.959101  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 129322us
I20260709 13:04:01.965231  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:01.975339  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:01.975752  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:01.976804  8748 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:01.978493  9383 raft_consensus.cc:2393] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:01.962282  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:01.930480 13338 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:01.982914  8750 raft_consensus.cc:2368] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7.
W20260709 13:04:01.987547  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.958479 13564 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:01.991729  9382 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:01.993335  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:01.994156  8748 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:01.962468 13469 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:01.961344 13560 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:02.001096  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:02.002341  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:01.986900 13061 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:01.957018 13370 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:01.982149 13315 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:02.010198  8753 raft_consensus.cc:2368] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11.
I20260709 13:04:02.010543  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:02.007236  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:01.983855 13338 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:04:02.008683  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.996555 13564 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:02.010470  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:02.016176  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 107373us
I20260709 13:04:01.960283 13567 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.006783 13061 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.029914  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:02.031203  9382 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:02.028177  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:02.035813  8825 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:02.013046 13338 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:02.004272 13555 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
W20260709 13:04:02.039467  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 100487us
W20260709 13:04:02.042874 13401 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.308s	user 0.010s	sys 0.003s
I20260709 13:04:02.042958  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:02.043565 13401 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.367s	user 0.010s	sys 0.004s
I20260709 13:04:02.044021  9389 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:02.044812  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:02.023473 13566 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:02.053124  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:02.028182 13061 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:02.059866  8645 leader_election.cc:400] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:04:02.059770  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:02.060932  8753 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:02.061615  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:02.061820  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:02.062911 13370 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:02.067168 13627 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:04:02.066916 13507 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
W20260709 13:04:02.068287 13617 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:04:02.066742 13492 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:04:02.057920  9353 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:02.040946 13619 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:02.072019 13619 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:02.071947  8566 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7aff04f53d4845a6a4fa491fd2a06145 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:04:02.073238  8566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 45752us
I20260709 13:04:02.067055 13484 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:02.075414  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:02.077263  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50366.2us
I20260709 13:04:02.081017  8748 raft_consensus.cc:1275] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:04:02.082499 13492 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:02.082592  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:02.083096 13492 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:02.083475 13567 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:02.083477  8823 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:02.084224 13492 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:02.085734 13492 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.086865 13315 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:02.087267 13401 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:02.087675 13315 consensus_queue.cc:1243] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:02.088342  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:02.089389  8823 raft_consensus.cc:2410] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:04:02.090010  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.090845  8829 raft_consensus.cc:2368] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13.
I20260709 13:04:02.091655  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:02.092337  8829 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
W20260709 13:04:02.091310  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 67309.4us
I20260709 13:04:02.093176  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:02.094597  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:02.088238 13315 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:04:02.088225  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:02.096429  8827 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:04:02.097388  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.097489  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.098685  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.094686  9383 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:02.090332  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.094323  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.100514  8829 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.078779  8796 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:02.103257  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 102746us
W20260709 13:04:02.106808  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 45262.4us
I20260709 13:04:02.112926  8830 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:02.118451  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:02.105427 13620 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:02.127794 13567 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:02.128511 13567 raft_consensus.cc:697] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:02.129534 13567 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:02.131669 13624 log.cc:927] Time spent T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:04:02.133221  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 852) took 1252 ms. Trace:
I20260709 13:04:02.133669  8676 rpcz_store.cc:276] 0709 13:04:00.881004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.477252 (+596248us) service_pool.cc:224] Handling call
0709 13:04:01.477620 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.543217 (+ 65597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.543259 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:01.575230 (+ 31971us) log.cc:844] Serialized 26 byte log entry
0709 13:04:01.899697 (+324467us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:01.899761 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.901917 (+  2156us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:02.132924 (+231007us) raft_consensus.cc:1685] finished
0709 13:04:02.132948 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.133151 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34258,"raft.queue_time_us":43452,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":53,"thread_start_us":284440,"threads_started":1,"wal-append.queue_time_us":372544}
I20260709 13:04:02.141381  8756 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:02.143424  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.143524  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:02.151808  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:02.163955  8748 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:02.165079  8722 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:02.167814  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:02.159054 13315 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:02.172405 13613 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.172291 13315 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:02.172291  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 119191us
I20260709 13:04:02.174430  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:02.175283  9383 raft_consensus.cc:2368] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17.
I20260709 13:04:02.177780  8830 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:02.176122  8719 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:02.183461  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.186103  8794 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:02.186575  8645 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:02.188557  8722 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:02.172442 13487 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:02.192152  8719 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:02.193454  8722 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:02.198109 13382 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:02.199049 13421 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:02.202869 13521 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:02.203560 13521 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:02.204263 13561 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:02.204659 13521 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.205267 13561 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:04:02.203577  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.205576  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:02.186244 13465 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:02.213598  8826 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:02.221390  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:02.223582  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:02.226459  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:02.206560 13345 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:02.195915  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:02.232016  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.232697 13497 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:02.239454  8826 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:02.211483 13469 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.196353 13487 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:04:02.240507  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 58485.6us
I20260709 13:04:02.240729  8830 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:02.213171 13561 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:02.242161  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:02.254333  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 87346us
I20260709 13:04:02.232074 13457 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:02.264202  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:02.268049  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:02.268841  8748 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.278062  9383 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:04:02.282497  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.283352  8826 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.287163 13315 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:02.288810  9384 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:04:02.292052  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 1988) took 1837 ms (client timeout 1625 ms). Trace:
W20260709 13:04:02.292520  8821 rpcz_store.cc:269] 0709 13:04:00.454432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.454648 (+   216us) service_pool.cc:224] Handling call
0709 13:04:02.291781 (+1837133us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54003ab1d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:02.291958 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":586864384}
I20260709 13:04:02.293716  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.245404 13487 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:02.291164  8748 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:02.297320  8645 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:02.300385  8825 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:02.252020 13557 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:02.328886  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:02.332623  8827 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 6.
W20260709 13:04:02.334220  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 58612.6us
I20260709 13:04:02.233522 13465 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:02.250389 13469 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.352928  8821 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
W20260709 13:04:02.366277  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.307773 13566 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:02.378172  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:02.379252  8748 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 12: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:02.383138  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:02.327421 13557 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:02.388211  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.386950  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:02.396509  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 175175us
W20260709 13:04:02.398490  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.410794  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:02.415438  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.341574 13401 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:02.419968  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 164720us
I20260709 13:04:02.376463 13382 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:02.422993  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:02.424283  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.420523  9381 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:02.415937 13560 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:02.389410 13557 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:02.428792  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:02.389707 13566 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:02.438571 13638 log.cc:927] Time spent T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:04:02.447820  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:02.448446 13469 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:02.432260 13417 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:02.449985 13417 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:02.450356 13417 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:02.422401 13561 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.454061 13561 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:02.455273 13421 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:02.455596 13382 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:02.456617  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.457085 13382 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.457934  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.462129 13566 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.468693  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:02.450382 13465 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:02.455379 13338 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:02.474133  9389 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:02.470572  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:02.477260  8821 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:02.448565  8751 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
W20260709 13:04:02.474668  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46014.8us
I20260709 13:04:02.474248 13338 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:02.481726 13435 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.482309  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.483067  8827 raft_consensus.cc:2376] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10. Re-sending same reply.
I20260709 13:04:02.483250  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 801) took 1322 ms. Trace:
I20260709 13:04:02.483677  8747 rpcz_store.cc:276] 0709 13:04:01.160484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.166559 (+  6075us) service_pool.cc:224] Handling call
0709 13:04:01.167224 (+   665us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.168023 (+   799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.168053 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:01.168763 (+   710us) log.cc:844] Serialized 26 byte log entry
0709 13:04:01.227525 (+ 58762us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:01.227592 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.227684 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:02.482155 (+1254471us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b24001107f8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df2929c4 00007f32df290acb 00007f32df44bdc9 00007f32df44ba55 00007f32df44afb0 00007f32e20c1488 00007f32e20c1be8 00007f32e2099398 00007f32e209476a 00007f32e2093bb6
0709 13:04:02.482929 (+   774us) raft_consensus.cc:1685] finished
0709 13:04:02.482950 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.483185 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2362,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":191,"spinlock_wait_cycles":11520384,"thread_start_us":58117,"threads_started":1,"wal-append.queue_time_us":110309}
I20260709 13:04:02.485589  8829 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:02.478139 13345 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:02.487720 13345 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.488905  8647 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:02.481726  8751 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:02.490947 13401 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:02.474570  9380 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:04:02.484438  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:02.486567  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:02.500155  8829 raft_consensus.cc:2410] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:04:02.432395 13560 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:02.505103 13560 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.427263 13472 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:04:02.507831 13472 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:02.508314 13472 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:02.509403 13472 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.515504  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2144) took 1032 ms. Trace:
I20260709 13:04:02.516047  8826 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:02.518136  9385 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:02.516134  8680 rpcz_store.cc:276] 0709 13:04:01.483449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.708723 (+225274us) service_pool.cc:224] Handling call
0709 13:04:01.709229 (+   506us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.849343 (+140114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.512345 (+663002us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54002567d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:02.515402 (+  3057us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":251132160}
I20260709 13:04:02.521090  8827 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:02.538287  8748 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:04:02.535709 13497 raft_consensus.cc:697] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:02.545710  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.550352  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:02.552223  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.553169  8827 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:02.539068  8748 pending_rounds.cc:85] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 9
I20260709 13:04:02.561005  8821 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:04:02.560083  8645 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:02.568519 13497 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.556435 13613 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:01.900430  8672 rpcz_store.cc:276] 0709 13:04:00.359859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.413699 (+1053840us) service_pool.cc:224] Handling call
0709 13:04:01.493939 (+ 80240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.494318 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:01.754404 (+260086us) log.cc:844] Serialized 14 byte log entry
0709 13:04:01.879370 (+124966us) log.cc:844] Serialized 14 byte log entry
0709 13:04:01.879738 (+   368us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:01.879791 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:01.879850 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.879933 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.880090 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12090,"prepare.queue_time_us":400700,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":5407,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":8848,"thread_start_us":359111,"threads_started":2}
I20260709 13:04:02.541074 13611 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:02.569197 13441 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:04:02.575177  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 96258.7us
I20260709 13:04:02.566318 13061 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:02.557085 13472 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.543641 13557 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:02.579946  8679 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:02.581418  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.582921 13611 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:02.558495  8748 pending_rounds.cc:107] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:04:02.596517  8748 raft_consensus.cc:2889] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:02.585343  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:02.597591  8829 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:02.587051 13441 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:02.613380 13441 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.615127 13441 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:02.618541  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.620010  8748 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:02.618650 13611 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:02.621860 13611 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:02.624408  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 850) took 1744 ms. Trace:
I20260709 13:04:02.625078  8679 rpcz_store.cc:276] 0709 13:04:00.880218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.473923 (+593705us) service_pool.cc:224] Handling call
0709 13:04:02.579431 (+1105508us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:04:02.579713 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.624019 (+ 44306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.624317 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30383872}
I20260709 13:04:02.630759  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:02.633955  8753 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:02.622596 13611 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:02.626714 13465 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:02.652336  8829 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
W20260709 13:04:02.649184 13492 consensus_meta.cc:287] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.563s	user 0.007s	sys 0.010s
I20260709 13:04:02.672277  8823 raft_consensus.cc:2410] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:04:02.672407  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:02.673707  8827 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:04:02.674119  8821 raft_consensus.cc:2410] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:04:02.671926  8722 leader_election.cc:400] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13.
I20260709 13:04:02.677016  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:02.678969  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:02.680063  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:02.681278  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:02.681845  8829 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:04:02.690825  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:02.711421  8829 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:02.717365 13645 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:02.689272 13648 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:02.675835 13465 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:02.709470 13566 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:02.732678  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 133900us
I20260709 13:04:02.720314 13401 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:04:02.734741  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 254836us
I20260709 13:04:02.724805 13645 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.739562  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:02.741400  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:02.744282  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.759852  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:02.699488  8821 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:60438
I20260709 13:04:02.769606  8821 raft_consensus.cc:476] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Not starting forced leader election -- already a leader
I20260709 13:04:02.736157 13382 raft_consensus.cc:2764] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:02.771865  8829 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:04:02.774432 13465 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:02.775164  8824 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:02.775631 13644 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:02.780036 13652 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:02.781839  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:02.786927 13382 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:02.770054 13566 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:02.770308  8645 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:02.746397 13487 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:02.746821 13645 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:02.794660 13348 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:02.767848 13061 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:02.788388 13557 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:02.800201  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2203) took 1108 ms. Trace:
I20260709 13:04:02.788069 13401 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:04:02.821903  8749 rpcz_store.cc:276] 0709 13:04:01.692322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.694635 (+  2313us) service_pool.cc:224] Handling call
0709 13:04:01.734432 (+ 39797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.767144 (+ 32712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.799876 (+1032732us) spinlock_profiling.cc:243] Waited 39.4 ms on lock 0x7b54002ac950. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:02.800105 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90574208}
I20260709 13:04:02.810499 13428 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:02.810830 13487 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:02.798084 13419 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:02.812395  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:02.837333 13622 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.986s	user 0.000s	sys 0.002s
W20260709 13:04:02.841228  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.761256 13648 consensus_queue.cc:1059] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:04:02.810130 13566 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:02.814157 13611 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:02.848816  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:02.814558 13417 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:02.815052  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:02.823777  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 71798.3us
I20260709 13:04:02.855010  8826 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:02.819383 13348 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:02.824052 13428 raft_consensus.cc:697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:02.829036  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 894) took 2266 ms. Trace:
I20260709 13:04:02.862514  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:02.850093  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 90626.2us
I20260709 13:04:02.848816  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:02.856561  8722 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:02.863201  8824 rpcz_store.cc:276] 0709 13:04:00.562682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.568059 (+  5377us) service_pool.cc:224] Handling call
0709 13:04:00.591339 (+ 23280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.775534 (+2184195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.828703 (+ 53169us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b5400126454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:02.828945 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4950787456}
I20260709 13:04:02.867288  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:02.869346  8722 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:02.871848  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:02.869755 13487 raft_consensus.cc:2764] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:04:02.875237 13348 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.873725 13497 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:02.876214  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 902) took 1704 ms. Trace:
I20260709 13:04:02.876407 13348 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.876749  8828 rpcz_store.cc:276] 0709 13:04:01.172042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.172868 (+   826us) service_pool.cc:224] Handling call
0709 13:04:01.173384 (+   516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.740942 (+567558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.875949 (+1135007us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b54002292d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:02.876150 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1083278208}
I20260709 13:04:02.876307 13417 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:02.879128 13428 consensus_queue.cc:237] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.879930 13557 raft_consensus.cc:3055] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:02.880354 13557 raft_consensus.cc:740] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:02.881348 13557 consensus_queue.cc:260] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.870967 13647 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:02.872457 13643 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:02.887653 13647 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.889745 13647 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:02.891424 13523 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:02.904280  8828 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:02.905820  8828 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:02.910701 13629 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.498s	user 0.000s	sys 0.002s
I20260709 13:04:02.912137 13620 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.912695 13620 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:02.912865 13557 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:02.915053  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2156) took 1352 ms. Trace:
I20260709 13:04:02.916319  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:02.917097  8680 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:02.917325  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:02.888010 13497 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:02.918416  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:02.919139 13497 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:02.915494  8674 rpcz_store.cc:276] 0709 13:04:01.562877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.784735 (+221858us) service_pool.cc:224] Handling call
0709 13:04:01.801220 (+ 16485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.804606 (+  3386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.804654 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:01.818857 (+ 14203us) log.cc:844] Serialized 26 byte log entry
0709 13:04:02.004632 (+185775us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:02.004684 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.004763 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:02.909747 (+904984us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:02.914680 (+  4933us) raft_consensus.cc:1685] finished
0709 13:04:02.914704 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.914983 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2913,"raft.queue_time_us":3639,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":14198,"spinlock_wait_cycles":6660864,"thread_start_us":156516,"threads_started":1,"wal-append.queue_time_us":441448}
W20260709 13:04:02.925513  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.934082  8680 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:02.895028 13648 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:02.942214 13566 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:02.937395  8826 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:02.951331  8828 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:04:02.953732  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 81910.7us
I20260709 13:04:02.954630  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:02.957950  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:02.966056  9380 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:02.969250  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:02.972085  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:02.973109  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:02.973791  8675 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:02.973800  8673 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:02.964776  8645 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:02.984323  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:02.988256  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:02.989274  8679 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:02.991422  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:02.993834  8824 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
W20260709 13:04:02.996341  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 100152us
I20260709 13:04:02.997618  8645 leader_election.cc:400] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7.
I20260709 13:04:02.997977  8645 leader_election.cc:403] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:04:02.999976  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:03.001039  8672 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
W20260709 13:04:03.002418  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.994419  8824 pending_rounds.cc:85] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 8
I20260709 13:04:03.003073  8824 pending_rounds.cc:107] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:04:03.003435  8824 raft_consensus.cc:2889] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:03.007388  8681 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:02.983863 13645 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:03.015116  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:03.015282  8645 leader_election.cc:400] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11.
I20260709 13:04:03.026486  8645 leader_election.cc:403] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:04:03.015360 13249 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:04:03.015863  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:03.015910 13488 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:03.016495 13652 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 6. Current term is 7.
I20260709 13:04:03.018924  8674 raft_consensus.cc:2393] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:03.019326 13469 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:03.019663 13061 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:02.980597 13437 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:03.062397 13437 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.063153 13437 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:03.021602  8680 pending_rounds.cc:85] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 4
I20260709 13:04:02.982568 13611 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:04:03.067716  8680 pending_rounds.cc:107] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:04:03.068071  8680 raft_consensus.cc:2889] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:02.965963 13417 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:03.069120 13417 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:03.071138 13382 consensus_queue.cc:1048] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:03.071547 13417 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:03.023895  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.024058  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:03.022641  8722 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:03.022917  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:03.013029 13401 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:03.073500 13401 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.074031 13401 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:03.074652 13645 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:03.075078 13657 raft_consensus.cc:3055] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:03.075553 13657 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:03.051249  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:03.079227  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:03.057096  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:03.073590  8681 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:04:03.086021  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68471.2us
I20260709 13:04:03.086993  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:03.087105  8719 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:03.089264 13644 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:03.089887 13061 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:03.090379 13061 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:03.090482  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:03.089211 13645 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:03.091560 13645 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:03.091436 13061 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:03.092815 13061 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:03.092603 13645 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:03.089524 13643 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:03.089587 13421 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:04:03.094874 13566 consensus_queue.cc:1175] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" responder_term: 12 status { last_received { term: 12 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 12 index: 7 } }
W20260709 13:04:03.095405  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 72588.9us
I20260709 13:04:03.094877 13382 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:03.042297 13348 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.098156 13348 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:03.100847 13348 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:03.085913  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:03.107302  8678 raft_consensus.cc:2410] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:03.106074  8675 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:03.114871  8681 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:03.107076 13652 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:04:03.128878  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 838) took 4374 ms. Trace:
I20260709 13:04:03.129462 13652 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:03.129400  8677 rpcz_store.cc:276] 0709 13:03:58.754196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.476936 (+722740us) service_pool.cc:224] Handling call
0709 13:03:59.477413 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.910754 (+2433341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.128513 (+1217759us) spinlock_profiling.cc:243] Waited 2.4 s on lock 0x7b5400298954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:03.128797 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5527779712}
I20260709 13:04:03.131188  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:03.131933  8822 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:03.135401 13664 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:04:03.137739  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:03.140789  8826 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:03.089367 13469 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:03.149776  8750 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:03.150969  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:03.130239 13652 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:03.157495 13497 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.159193  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:03.111049 13487 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.163746  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:03.164783  8826 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:04:03.167570  8756 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:03.169593  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:03.187031  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:03.187711  8754 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:03.188171  8681 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 14: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:03.188098  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:03.188856  8747 raft_consensus.cc:2376] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11. Re-sending same reply.
I20260709 13:04:03.189262  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:03.197623 13665 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:04:03.201660 13348 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.113843 13472 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 13:04:03.202387 13348 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:03.208576  8753 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:03.223237  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:03.223701  8681 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:03.224313  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2220) took 1263 ms. Trace:
I20260709 13:04:03.156370 13644 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:04:03.224870  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2008) took 1033 ms. Trace:
I20260709 13:04:03.225303  8821 rpcz_store.cc:276] 0709 13:04:02.191260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.796133 (+604873us) service_pool.cc:224] Handling call
0709 13:04:02.796596 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.810933 (+ 14337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.810965 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:02.811533 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:04:02.969498 (+157965us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:02.969544 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.969616 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:03.224619 (+255003us) raft_consensus.cc:1685] finished
0709 13:04:03.224657 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.224806 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":157514,"threads_started":1,"wal-append.queue_time_us":347069}
W20260709 13:04:03.226930  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.227458  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:03.228503  8821 raft_consensus.cc:2410] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:04:03.224736  8755 rpcz_store.cc:276] 0709 13:04:01.961438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.222882 (+1261444us) service_pool.cc:224] Handling call
0709 13:04:03.224240 (+  1358us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:03.190733  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:03.231465  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:03.225711  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:03.252597  8677 raft_consensus.cc:2393] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:03.253391  8681 raft_consensus.cc:2410] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:04:03.299551  8755 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:03.300809  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:03.302563  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:03.319324  8749 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:04:03.323016  8679 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:04:03.324822  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:03.343602  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:03.345202  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:03.356897  8826 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:03.365490  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:03.375424  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:03.376674  8679 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:03.377730 13348 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:03.162037 13657 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:03.203610 13487 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:03.378522 13348 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:03.393054  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2009) took 1168 ms. Trace:
I20260709 13:04:03.393605  8829 rpcz_store.cc:276] 0709 13:04:02.225200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.809207 (+584007us) service_pool.cc:224] Handling call
0709 13:04:02.822493 (+ 13286us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:02.843099 (+ 20606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.843134 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:02.862711 (+ 19577us) log.cc:844] Serialized 52 byte log entry
0709 13:04:03.025050 (+162339us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:03.025094 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.025182 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:03.392777 (+367595us) raft_consensus.cc:1685] finished
0709 13:04:03.392802 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.392978 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5913,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":97,"thread_start_us":136150,"threads_started":1,"wal-append.queue_time_us":420706}
I20260709 13:04:03.355691 13465 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:03.412732  8822 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:03.340123 13656 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11.
W20260709 13:04:03.417760  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 321281us
W20260709 13:04:03.415947  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 907) took 2028 ms (client timeout 2426 ms). Trace:
W20260709 13:04:03.421200  8822 rpcz_store.cc:269] 0709 13:04:01.396082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.130837 (+1734755us) service_pool.cc:224] Handling call
0709 13:04:03.415879 (+285042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:03.254184 13497 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:03.413908  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 325285us
W20260709 13:04:03.416050  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 429381us
I20260709 13:04:03.433876  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:03.436007  8822 raft_consensus.cc:2376] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14. Re-sending same reply.
W20260709 13:04:03.438179  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.438308  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:03.439255  8829 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:03.417327 13523 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.525s	user 0.017s	sys 0.006s
I20260709 13:04:03.428821  8681 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:04:03.515990  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:03.517410  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.517311  8756 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:03.524330  9387 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:04:03.537350  8826 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:03.441787 13465 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.435801 13382 raft_consensus.cc:697] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:03.541977  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:03.448803 13657 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:03.448230 13645 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:03.548857  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:03.559633  8681 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:03.556659  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:03.371887 13652 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:03.535421 13523 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:03.564085  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:03.588398  8826 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:03.538630 13585 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:03.582526  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.590600  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:03.597640  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:03.612071  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:03.614974  9389 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:03.622116  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:03.623973 13370 raft_consensus.cc:3055] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:03.629412  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2013) took 1222 ms. Trace:
I20260709 13:04:03.637615  8824 rpcz_store.cc:276] 0709 13:04:02.407408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.879031 (+471623us) service_pool.cc:224] Handling call
0709 13:04:02.879379 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.010787 (+131408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.629098 (+618311us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b540041c154. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:03.629330 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240778112}
I20260709 13:04:03.630450  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 860) took 1730 ms. Trace:
I20260709 13:04:03.638837  8678 rpcz_store.cc:276] 0709 13:04:01.899671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.917036 (+1017365us) service_pool.cc:224] Handling call
0709 13:04:03.630188 (+713152us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400299354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:03.630350 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434444672}
I20260709 13:04:03.557655 13465 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:03.650170 13488 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:03.663774  9389 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:04:03.651678 13348 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:03.652573 13668 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 13:04:03.653450 13671 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:04:03.674777  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:03.675768  8676 raft_consensus.cc:2393] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:03.678426  8676 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
W20260709 13:04:03.655071  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:03.660583  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 315450us
I20260709 13:04:03.661430  8826 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:04:03.631273 13382 consensus_queue.cc:237] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:03.642570 13652 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:03.671394  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:03.604794 13645 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:03.637352 13370 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:03.665560 13620 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.743s	user 0.011s	sys 0.009s
I20260709 13:04:03.692509  8826 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:04:03.665421 13647 consensus_meta.cc:287] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.775s	user 0.011s	sys 0.008s
I20260709 13:04:03.701185  8829 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
W20260709 13:04:03.701712  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 259910us
I20260709 13:04:03.727937  8826 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:03.685966  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 250697us
I20260709 13:04:03.690321  8681 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:03.608024 13585 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:03.664288 13338 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:04:03.699642 13351 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:03.731009  9352 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:03.718304 13437 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.655s	user 0.013s	sys 0.002s
W20260709 13:04:03.775496  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 68976.1us
I20260709 13:04:03.680032 13488 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:03.772568  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:03.684705 13348 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:03.773639 13643 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.796288 13643 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:03.795567  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:03.787529  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:03.816534  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 85656us
I20260709 13:04:03.817401  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:03.828413  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.841490  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:03.713642 13620 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:03.847064  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:03.713440 13647 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:03.864054  8681 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:03.867956  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:03.879402  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:03.880125  8674 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:03.868834 13488 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:03.869221 13647 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:03.881695 13488 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.870290 13645 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:03.807483 13348 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:03.841490 13497 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:03.871469 13674 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:04:03.871773  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.871883 13643 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:03.872857 13682 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:03.873541 13692 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.884094  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2012) took 1537 ms. Trace:
W20260709 13:04:03.884199  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 335381us
I20260709 13:04:03.884488  8673 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:03.885151  9380 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:03.846149 13675 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:03.806047 13437 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:03.922855  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 392732us
I20260709 13:04:03.914459  8827 rpcz_store.cc:276] 0709 13:04:02.347037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.847282 (+500245us) service_pool.cc:224] Handling call
0709 13:04:02.847658 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.854716 (+  7058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.854752 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:02.863042 (+  8290us) log.cc:844] Serialized 26 byte log entry
0709 13:04:03.152397 (+289355us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:03.152467 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.152581 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:03.883809 (+731228us) raft_consensus.cc:1685] finished
0709 13:04:03.883838 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.884013 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":251870,"threads_started":1,"wal-append.queue_time_us":544079}
I20260709 13:04:03.928381  8674 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:03.929620  8828 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:04:03.930013  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:03.936877  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:03.882382 13488 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:03.974193  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:03.877151 13698 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:03.926330 13061 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.833s	user 0.023s	sys 0.001s
W20260709 13:04:03.998694  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.004333  8677 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 13:04:04.020260  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 157643us
I20260709 13:04:03.906538 13348 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:04:03.971315 13611 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:03.852419 13370 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.036006  8681 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:04:04.042527  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 61014.2us
W20260709 13:04:04.048422  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 74323.2us
I20260709 13:04:03.995061 13675 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:04.013720 13497 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:04.075645  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2157) took 1107 ms. Trace:
I20260709 13:04:04.076233  8680 rpcz_store.cc:276] 0709 13:04:02.968071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.982922 (+ 14851us) service_pool.cc:224] Handling call
0709 13:04:03.005598 (+ 22676us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:03.068779 (+ 63181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.068813 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:03.069328 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 13:04:03.449564 (+380236us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:03.449620 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.449722 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:04.075381 (+625659us) raft_consensus.cc:1685] finished
0709 13:04:04.075405 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.075568 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":379797,"threads_started":1,"wal-append.queue_time_us":615617}
W20260709 13:04:04.083798  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 400004us
I20260709 13:04:04.081817  8722 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:04.094059  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 72604.3us
I20260709 13:04:03.979826 13692 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.055899 13419 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.076002  9355 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:04.115625  8681 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:04.103225  8677 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:04.106549  8680 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:04.120396 13472 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:04.144285  8680 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:04:04.081451 13401 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.007s	user 0.005s	sys 0.018s
I20260709 13:04:04.146325 13401 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.149370 13401 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:04.150758  8676 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:04.118191  9355 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:04.154433 13679 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:04.156773  9355 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:04.080878 13465 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:04.061175 13249 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:04:04.142145  8681 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 13:04:03.879024  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:04.127502 13682 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:04.109529 13382 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
W20260709 13:04:04.294741  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.299139 13705 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:04:04.125633 13419 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.302001 12656 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.299502 13564 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.224898 13682 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.321121 13691 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:04.322463 13686 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
W20260709 13:04:04.325366 13696 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.198s	user 0.000s	sys 0.000s
W20260709 13:04:04.350306  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.463943  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.401029  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.429975  8679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.004s	sys 0.014s
W20260709 13:04:04.464991  8679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.005s	sys 0.015s
I20260709 13:04:04.080575 13698 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
W20260709 13:04:04.437041 13648 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.008s	sys 0.012s
I20260709 13:04:04.451246  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2025) took 1003 ms. Trace:
W20260709 13:04:04.452625  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 366187us
I20260709 13:04:04.476835  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2178) took 1500 ms. Trace:
I20260709 13:04:04.477313  8679 rpcz_store.cc:276] 0709 13:04:02.976317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.377727 (+401410us) service_pool.cc:224] Handling call
0709 13:04:03.378298 (+   571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.378600 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:03.816243 (+437643us) log.cc:844] Serialized 14 byte log entry
0709 13:04:04.476405 (+660162us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:04.476473 (+    68us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:04.476526 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.476620 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.476773 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10706,"prepare.queue_time_us":387909,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"thread_start_us":1086063,"threads_started":2}
I20260709 13:04:04.451365 13620 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:04.437530 13437 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:04.458460  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 297774us
I20260709 13:04:04.476527  8822 rpcz_store.cc:276] 0709 13:04:03.447693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.528113 (+ 80420us) service_pool.cc:224] Handling call
0709 13:04:03.556469 (+ 28356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.604305 (+ 47836us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:03.925075 (+320770us) log.cc:844] Serialized 14 byte log entry
0709 13:04:04.450694 (+525619us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:04.450781 (+    87us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:04.450861 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.450965 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.451139 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":47006,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":354139,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6612,"thread_start_us":696368,"threads_started":3,"wal-append.queue_time_us":350603}
I20260709 13:04:04.453715 13497 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:04.476274 13648 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.009s	sys 0.012s
I20260709 13:04:04.456593 13682 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:04.494266 13703 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:04:04.495760  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2654) took 1291 ms. Trace:
I20260709 13:04:04.479012 13491 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:04.434881 13675 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:04.478761 13714 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:04:04.477603 13699 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:04:04.502763  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 196638us
I20260709 13:04:04.486440 13564 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:04.481805 13401 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:04.496734 13647 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:04:04.567123  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.500772 13488 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:04.459403 12656 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.463949 13691 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.512547 13713 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:04:04.518101  8750 rpcz_store.cc:276] 0709 13:04:03.204400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.204742 (+   342us) service_pool.cc:224] Handling call
0709 13:04:03.306593 (+101851us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:03.340755 (+ 34162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.340821 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:03.367153 (+ 26332us) log.cc:844] Serialized 26 byte log entry
0709 13:04:03.708427 (+341274us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:03.708474 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.708545 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:04.495377 (+786832us) raft_consensus.cc:1685] finished
0709 13:04:04.495412 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.495659 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34374,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":9440,"thread_start_us":235204,"threads_started":1,"wal-append.queue_time_us":621569}
I20260709 13:04:04.470782 13710 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.064s
W20260709 13:04:04.557789  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 101469us
I20260709 13:04:04.560098  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2179) took 1583 ms. Trace:
I20260709 13:04:04.514320 13698 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:04:04.571882 13647 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.575598  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:04.577960 13736 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:04.579157  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.554598 13491 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.579335  8678 rpcz_store.cc:276] 0709 13:04:02.976675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.649121 (+672446us) service_pool.cc:224] Handling call
0709 13:04:03.653509 (+  4388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.653737 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:04.059994 (+406257us) log.cc:844] Serialized 14 byte log entry
0709 13:04:04.557197 (+497203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:04.557299 (+   102us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:04.557372 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.559795 (+  2423us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.560010 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":115008,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":70,"raft.queue_time_us":1642,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":4724,"thread_start_us":882140,"threads_started":2,"wal-append.queue_time_us":512927}
I20260709 13:04:04.574538 13699 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:04.589583 13652 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.889s	user 0.010s	sys 0.006s
I20260709 13:04:04.545229 13497 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.580045  9389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 753) took 1052 ms. Trace:
W20260709 13:04:04.580390 13723 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:04:04.582991 13732 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:04.613782 13732 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.597595 13648 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.595531 13401 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.617789  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:04.618548  8677 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:04:04.620252 13401 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:04.597818 13419 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:04.595306 13657 consensus_meta.cc:287] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.025s	user 0.014s	sys 0.000s
I20260709 13:04:04.598003  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:04.623983  8678 raft_consensus.cc:2410] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 13:04:04.624653  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.625157 13729 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:04.621075 13732 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:04.627130 13732 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.598502 13736 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.627833 13732 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.630182 13736 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:04.599273 13491 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:04.610397 13647 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:04.634337  8674 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:04.609887  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2656) took 1310 ms. Trace:
I20260709 13:04:04.635186  8756 rpcz_store.cc:276] 0709 13:04:03.298974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.300467 (+  1493us) service_pool.cc:224] Handling call
0709 13:04:04.609588 (+1309121us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540038a9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:04.609759 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336894848}
I20260709 13:04:04.603024  9389 rpcz_store.cc:276] 0709 13:04:03.527246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.527451 (+   205us) service_pool.cc:224] Handling call
0709 13:04:04.579828 (+1052377us) spinlock_profiling.cc:243] Waited 72.2 ms on lock 0x7b54006275d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:04.579975 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166067584}
I20260709 13:04:04.632114 13732 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:04.636546  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2180) took 1574 ms. Trace:
I20260709 13:04:04.605032  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2030) took 1078 ms. Trace:
I20260709 13:04:04.637157  8829 rpcz_store.cc:276] 0709 13:04:03.526345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.702062 (+175717us) service_pool.cc:224] Handling call
0709 13:04:03.702560 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:03.821253 (+118693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.821296 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:03.845709 (+ 24413us) log.cc:844] Serialized 26 byte log entry
0709 13:04:04.067580 (+221871us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:04.067638 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.067753 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:04.604777 (+537024us) raft_consensus.cc:1685] finished
0709 13:04:04.604804 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.604968 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15650,"raft.queue_time_us":493041,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":90559,"thread_start_us":259974,"threads_started":2,"wal-append.queue_time_us":325727}
I20260709 13:04:04.625975 13729 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.597473 13437 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:04.643453 13729 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:04.621219  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 871) took 1562 ms. Trace:
I20260709 13:04:04.643548 13437 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.644697  8673 rpcz_store.cc:276] 0709 13:04:03.058967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.721226 (+662259us) service_pool.cc:224] Handling call
0709 13:04:03.878420 (+157194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.918642 (+ 40222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.621004 (+702362us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54002a8fd0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:04.621157 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":360286080}
I20260709 13:04:04.636989  8674 rpcz_store.cc:276] 0709 13:04:03.061715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.975630 (+913915us) service_pool.cc:224] Handling call
0709 13:04:04.634141 (+658511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.634837 (+   696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.636290 (+  1453us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b54002a99d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:04.636479 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1513124224}
I20260709 13:04:04.597887 13699 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.609154  9382 raft_consensus.cc:2410] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 13:04:04.581221 13693 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:04:04.648542 13704 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.649154 13704 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.604817 12656 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:04.649575  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2173) took 1675 ms. Trace:
I20260709 13:04:04.650758  8672 rpcz_store.cc:276] 0709 13:04:02.974339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.232567 (+258228us) service_pool.cc:224] Handling call
0709 13:04:03.306741 (+ 74174us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:03.307049 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.307088 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:03.348724 (+ 41636us) log.cc:844] Serialized 26 byte log entry
0709 13:04:03.772866 (+424142us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:03.772909 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.772985 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:04.649278 (+876293us) raft_consensus.cc:1685] finished
0709 13:04:04.649312 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.649494 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":398252,"threads_started":1,"wal-append.queue_time_us":804405}
I20260709 13:04:04.617673 13648 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.655102 13648 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:04.606564 13691 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:04.657698 13710 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:04.658512 13710 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.676249  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:04.676440  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:04.677039  8756 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:04.677313  8748 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 14: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:04.678920  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:04.679536  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2131) took 2637 ms. Trace:
I20260709 13:04:04.679930  8825 rpcz_store.cc:276] 0709 13:04:02.042206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.042386 (+   180us) service_pool.cc:224] Handling call
0709 13:04:02.296748 (+254362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.328108 (+ 31360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.679343 (+2351235us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54003ab1d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:04.679479 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":525105408}
W20260709 13:04:04.683499 13472 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.516s	user 0.013s	sys 0.000s
I20260709 13:04:04.684026 13472 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:04.684326 13736 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.685032 13736 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.674343 13648 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.686197 13648 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.687590 13736 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:04.685345 13708 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.688563 13648 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:04.688382 13708 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.690603 13708 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:04.683303  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:04.691664 13726 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:04.692087  8750 raft_consensus.cc:2410] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:04:04.695814  8722 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:04.692092 13726 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:04:04.722451  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.724272  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:04.725201  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:04.725960  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:04.717720 13708 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.729369 13730 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:04.729734 13708 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.729987 13730 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.730304 13472 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.730945 13472 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.732080 13730 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:04.721115 13736 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:04.734812 13437 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:04.732101 13708 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:04.721207 13441 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.735976 13441 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.736508 13710 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:04.727385 13648 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:04:04.739005  8566 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7aff04f53d4845a6a4fa491fd2a06145 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:04.739110 13648 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:04.741120  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:04.745668  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:04.735178 13736 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.738276 13441 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:04.764389  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:04.765187  9385 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:04.728000 13698 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.763772  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:04.763599  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:04.760794  8645 leader_election.cc:336] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.666s (SENT)
I20260709 13:04:04.765826 13710 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:04.767859  9389 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:04.766744  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:04.771857  8796 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:04.785439  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.785643  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:04.777343 13441 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:04.786185 13351 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.003s	sys 0.019s
I20260709 13:04:04.823073  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:04.824056  8821 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:04:04.826136  8829 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:04.738292 13726 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:04.821396 13351 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.003s	sys 0.019s
I20260709 13:04:04.783341 13710 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.816762 13441 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.836755 13682 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.837560 13726 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:04.839154  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.768970 13698 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.839583  9380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 754) took 1304 ms. Trace:
I20260709 13:04:04.835264 13736 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:04.831070 13691 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.847595  9380 rpcz_store.cc:276] 0709 13:04:03.535351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.535509 (+   158us) service_pool.cc:224] Handling call
0709 13:04:04.839331 (+1303822us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54001ccd54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:04.839506 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":743260928}
I20260709 13:04:04.868139  9386 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:04.843554 13441 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:04.882822 13710 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:04.884120  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 824) took 1922 ms. Trace:
I20260709 13:04:04.884594  8751 rpcz_store.cc:276] 0709 13:04:02.961749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.961903 (+   154us) service_pool.cc:224] Handling call
0709 13:04:04.883405 (+1921502us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b5800083198. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:04:04.883576 (+   171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.883769 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.883795 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:04.883847 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:04.883879 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.883938 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.884051 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2624128}
I20260709 13:04:04.885353 13472 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:04.888614 13487 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:04.893414  9387 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 743) took 3217 ms (client timeout 2228 ms). Trace:
W20260709 13:04:04.893930  9387 rpcz_store.cc:269] 0709 13:04:01.676139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.676915 (+   776us) service_pool.cc:224] Handling call
0709 13:04:04.893186 (+3216271us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b540046a5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:04.893296 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4078733184}
I20260709 13:04:04.895931  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:04.898257 13634 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 2.472s	user 0.005s	sys 0.003s
I20260709 13:04:04.908154 13698 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:04.912675  8719 connection.cc:256] RPC call timeout handler was delayed by -1.0708429589999469s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20350000000000001s for a response
I20260709 13:04:04.921187 13704 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:04.922746  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 825) took 1122 ms. Trace:
I20260709 13:04:04.819406 13648 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:04.940941  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:04.964293  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:04.954203  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:04.965518  8675 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:04.955637  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:04.948197  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:04.967468  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:04.967659  8674 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:04.968344  8751 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:04.969682  8673 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:04:04.971447  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.975023  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:04.976380  8674 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:04.981247  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:04.982350  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:04.957643  8753 rpcz_store.cc:276] 0709 13:04:03.800834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.802141 (+  1307us) service_pool.cc:224] Handling call
0709 13:04:04.609991 (+807850us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.610479 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.610508 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:04.611115 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 13:04:04.788348 (+177233us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:04.788394 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.788477 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:04.856701 (+ 68224us) raft_consensus.cc:1685] finished
0709 13:04:04.856727 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.922401 (+ 65674us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b540038a9d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:04.922664 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":1856445184,"thread_start_us":176747,"threads_started":1,"wal-append.queue_time_us":242551}
I20260709 13:04:04.983940  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:04.983691  9352 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:04.983817  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:04.985044  8679 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
W20260709 13:04:04.985993  8793 connection.cc:256] RPC call timeout handler was delayed by -1.0800479880000466s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
I20260709 13:04:04.986202  8674 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:04.966928 13421 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:04.865605 13691 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:04.989070  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:04.994519  8678 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
W20260709 13:04:04.986451  8793 connection.cc:256] RPC call timeout handler was delayed by -1.0710479880000321s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
I20260709 13:04:04.989287  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:04.996011  8672 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:04:04.966568 13527 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:04.996937  8793 connection.cc:256] RPC call timeout handler was delayed by -1.0377479879999782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1572s for a response
I20260709 13:04:05.002322  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:04.975664 13726 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:05.006470  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.011022  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:05.016971  8829 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
W20260709 13:04:05.011745  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 2.22736e+06us
I20260709 13:04:05.019220  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:04.966456  8676 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:05.013083  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:04.988143 13698 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:04.983346 13472 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:04:05.023484  8647 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:05.025242  8829 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:05.031877  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.021688 13713 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.041981  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:04.994297 13677 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.039963  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:05.043124  8753 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:05.048048  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:05.048548  8647 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:05.048688 13713 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.049177 13491 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:05.044061 13678 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:04:05.049669  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:05.050880 13746 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:04:05.050823  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.067571  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:05.068444  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:05.068456  8830 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:05.071348  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:05.066431  8676 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:05.072023  8674 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:04.983044 13428 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.060s
W20260709 13:04:05.073436  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.035701 13487 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:05.081415  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2031) took 1554 ms. Trace:
I20260709 13:04:05.022533 13338 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:05.079384  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2207) took 3372 ms. Trace:
I20260709 13:04:05.026679 13647 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:04:05.094172  9352 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:05.082161  8826 rpcz_store.cc:276] 0709 13:04:03.526870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.729571 (+202701us) service_pool.cc:224] Handling call
0709 13:04:03.779115 (+ 49544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.779468 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:03.845969 (+ 66501us) log.cc:844] Serialized 14 byte log entry
0709 13:04:04.836175 (+990206us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b18000ad380. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:04:05.080704 (+244529us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.080824 (+   120us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:05.080912 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.081021 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.081321 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20868,"prepare.queue_time_us":87032,"prepare.run_cpu_time_us":150,"prepare.run_wall_time_us":6868,"spinlock_wait_cycles":24358144,"thread_start_us":243784,"threads_started":1}
I20260709 13:04:05.091701  8825 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:04:05.092657  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.093468  8647 leader_election.cc:400] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:04:05.096084  8647 leader_election.cc:403] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:04:05.096807  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:05.019570 13527 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.083479  8752 rpcz_store.cc:276] 0709 13:04:01.694607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.872277 (+177670us) service_pool.cc:224] Handling call
0709 13:04:01.882407 (+ 10130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.891130 (+  8723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.891183 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:01.891918 (+   735us) log.cc:844] Serialized 26 byte log entry
0709 13:04:02.001970 (+110052us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:02.002023 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.002103 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:04.897490 (+2895387us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:05.061233 (+163743us) raft_consensus.cc:1685] finished
0709 13:04:05.061259 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.067030 (+  5771us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8150,"raft.queue_time_us":9901,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"spinlock_wait_cycles":5052160,"thread_start_us":109570,"threads_started":1,"wal-append.queue_time_us":439754}
I20260709 13:04:05.101015  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:05.102005  8794 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:05.103596  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:05.105871  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:05.067909 13348 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:05.107765  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.101961  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:05.113201  8748 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:05.025621 13421 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.109580  8821 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:05.074676 13657 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:04:05.124907  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 74262.2us
I20260709 13:04:05.126224  8793 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.113224  8829 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:05.130074  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:05.132331  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:05.028985 13726 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.090394 13487 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:05.091285 13338 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.107501 13704 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:05.113541 13708 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:05.149166  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.157923  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.117046 13491 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:04:05.163769  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.072646 13677 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.165027  8748 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:05.166049  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.121903 13710 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:05.169656  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:05.154731 13487 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:05.170809  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:05.199771  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 97843.8us
I20260709 13:04:05.168390 13421 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:05.171893  8830 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
W20260709 13:04:05.202205  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 186580us
W20260709 13:04:05.158066 13741 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:04:05.155591 13704 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.179409  8677 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:04:05.199774  8826 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:05.171005  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 103299us
I20260709 13:04:05.208511  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.208983 13647 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:05.210047  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:05.211047 13491 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:05.211714 13491 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:05.212226 13491 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:05.155226 13338 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:05.217546  8750 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:05.194386  8821 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:04:05.178256 13710 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:05.220595 13710 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:05.160080 13708 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:05.210170 13421 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:05.222589  8750 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:05.223259 13708 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:05.222841 13421 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:05.202715  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.224926 13421 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.226001 13421 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:05.240860  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:05.247781  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:05.248879  8755 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:05.172056 13657 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.255940  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:05.255846  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.255713  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.199898 13488 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:05.269560  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.269704 13488 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.256708 13704 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:05.282488  8755 raft_consensus.cc:2410] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:05.283515 13488 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.268323 13657 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.288820 13682 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:05.291111 13657 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:05.291793 13657 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:05.293613  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.294500  8756 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:05.294999  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.295782  8748 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
W20260709 13:04:05.273006 13745 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 13:04:05.282321 13749 log.cc:927] Time spent T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
W20260709 13:04:05.334384  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.335371  9382 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 651) took 3216 ms (client timeout 1888 ms). Trace:
W20260709 13:04:05.335981  9382 rpcz_store.cc:269] 0709 13:04:02.119710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.119877 (+   167us) service_pool.cc:224] Handling call
0709 13:04:05.335038 (+3215161us) spinlock_profiling.cc:243] Waited 2.45 s on lock 0x7b54005ea1d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:05.335270 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5640803200}
I20260709 13:04:05.260787 13441 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.354807  8829 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:05.356572  8826 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:04:05.358114  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:05.362012  8826 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:05.351919 13691 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:05.352910 13758 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:05.375844  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 249721us
I20260709 13:04:05.377882  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.387964  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.386322  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 828) took 1584 ms. Trace:
I20260709 13:04:05.388952  8748 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
W20260709 13:04:05.382015 13748 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.097s	user 0.001s	sys 0.001s
W20260709 13:04:05.382643  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 132117us
I20260709 13:04:05.422818  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.389047  8754 rpcz_store.cc:276] 0709 13:04:03.801945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.820465 (+ 18520us) service_pool.cc:224] Handling call
0709 13:04:04.927863 (+1107398us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b5800083198. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:04:04.928055 (+   192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.928307 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:05.099203 (+170896us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.385894 (+286691us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.385965 (+    71us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:05.386022 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.386100 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.386251 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":237085,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":1914,"spinlock_wait_cycles":4049792,"thread_start_us":456652,"threads_started":2}
I20260709 13:04:05.380705 13488 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:05.426887  8750 raft_consensus.cc:2410] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:04:05.402508 13726 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:05.385749 13760 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:05.437759  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.441305  8826 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:05.443359  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.444197  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.388062 13527 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:05.436625  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 226634us
I20260709 13:04:05.445719  8756 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:05.448628  8748 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:05.448761  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:05.453866  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:05.425755 13657 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:05.436683 13488 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:05.454303 13752 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
W20260709 13:04:05.467181 13747 log.cc:927] Time spent T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 13:04:05.469707 13751 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:04:05.473117 13710 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.475499 13338 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.477156 13691 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:05.477929 13691 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:05.478451 13691 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.461727 13441 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:05.480290  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:05.464479 13677 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:05.481204  8827 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:05.481711 13708 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.481408 13338 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:05.456215 13726 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:05.491715  8827 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:05.492206  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:05.493101  8753 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:05.446234 13760 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:05.505862  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 44949.6us
I20260709 13:04:05.508352  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:05.511266 13726 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:05.511824 13726 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:05.512809 13726 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.515389  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:05.522913 13677 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:05.526326  8647 connection.cc:256] RPC call timeout handler was delayed by -3.2781688999999687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26080000000000003s for a response
W20260709 13:04:05.527276  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.488336 13657 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.533077 13760 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:05.536588  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 923) took 1025 ms. Trace:
W20260709 13:04:05.536798  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 158619us
W20260709 13:04:05.543040  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.544085  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:05.537456  8824 rpcz_store.cc:276] 0709 13:04:04.511359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.511495 (+   136us) service_pool.cc:224] Handling call
0709 13:04:04.557159 (+ 45664us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.575063 (+ 17904us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.575099 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:04.675055 (+ 99956us) log.cc:844] Serialized 26 byte log entry
0709 13:04:05.062761 (+387706us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:05.093372 (+ 30611us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.093639 (+   267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.093720 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.536286 (+442566us) raft_consensus.cc:1685] finished
0709 13:04:05.536321 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.536488 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17357,"prepare.queue_time_us":35276,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":22255,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":66996,"spinlock_wait_cycles":19961216,"thread_start_us":326748,"threads_started":1,"wal-append.queue_time_us":468525}
W20260709 13:04:05.579018  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.549674 13760 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:05.603255  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 61650.9us
I20260709 13:04:05.604699  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:05.552946 13691 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:05.562979 13710 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:05.562664 13708 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:05.622042  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.641372  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 93934.4us
W20260709 13:04:05.639851  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.554297 13765 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:05.555083 13657 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:05.542650 13704 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.637022  8793 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.642566  9355 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:05.550568 13763 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:04:05.639453 13772 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:05.644088 13756 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.087s	user 0.001s	sys 0.001s
I20260709 13:04:05.642680 13691 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:05.650821 13523 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:05.651278 13691 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.653437 13691 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:05.656065 13704 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:05.659936  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.663187  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.663133 13523 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:05.662854 13708 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:05.668428 13708 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:05.670202  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:05.662981 13772 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:05.646908 13487 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:05.671150  8647 connection.cc:256] RPC call timeout handler was delayed by -2.3776875940000082s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16870000000000002s for a response
W20260709 13:04:05.672024  8647 connection.cc:256] RPC call timeout handler was delayed by -2.1697875940000131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
W20260709 13:04:05.672487  8647 connection.cc:256] RPC call timeout handler was delayed by -1.969087593999916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2141s for a response
I20260709 13:04:05.671705 13691 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:05.668488 13657 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.675060  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.676555 13657 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.677004 13710 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:05.677924  8647 leader_election.cc:400] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17.
I20260709 13:04:05.677762 13726 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:05.678933  8647 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:05.678951 13657 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:05.680048 13726 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:05.681322  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:05.681746 13726 consensus_queue.cc:1059] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:05.682158 13710 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:05.683296  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:05.674093 13708 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:05.686053  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.686404 13710 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:05.686491 13726 consensus_queue.cc:1059] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:05.687091 13710 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:05.687700 13726 raft_consensus.cc:3055] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:05.688112 13710 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.693111 13487 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:05.695287  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:05.695961  8679 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:05.699388  8676 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:04:05.700697 13762 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 13:04:05.701660  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:05.688092 13726 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:04:05.705773  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.707226  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:05.706125 13726 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.707824  8680 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:05.708359 13726 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:05.709607 13644 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:05.709923  9382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:05.710458 13061 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:05.710762 13487 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.711082 13061 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.711329 13487 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.713266 13061 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:05.713557 13487 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:05.710566  9382 raft_consensus.cc:606] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:05.709740  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:05.674738 13772 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:05.716078 13692 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:05.686012 13491 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.717240 13675 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.717814 13675 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.667421 13523 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.719947 13675 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:05.718463 13491 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:05.719965 13585 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:05.658638 13765 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.720986 13061 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:05.724406 13061 raft_consensus.cc:697] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:05.725963 13061 consensus_queue.cc:237] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.683527 13691 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.716104 13772 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.744765  8722 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:05.743167 13715 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:04:05.746285 13765 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:05.747732  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.747653  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:05.748714  8673 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:05.749159  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:05.685832 13657 raft_consensus.cc:3055] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:05.752888  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:05.730108 13645 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:04:05.771606  8676 raft_consensus.cc:2410] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:04:05.754562  9382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:05.755249  8722 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:05.758708 13566 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:05.759994 13495 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:05.752579 13472 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:05.736357 13679 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
W20260709 13:04:05.767704 13700 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.697s	user 0.004s	sys 0.000s
I20260709 13:04:05.772964  9382 raft_consensus.cc:606] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:05.780640  9382 raft_consensus.cc:614] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:05.774273  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 88279.3us
I20260709 13:04:05.781487  8829 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:05.780541 13760 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:05.788414  8679 raft_consensus.cc:2410] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:05.789007 13672 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:05.787395  8680 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:05.773454 13657 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:05.780984 13495 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:05.796908 13495 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.799634 13495 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:05.786921 13692 consensus_queue.cc:1059] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:04:05.803174 13692 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:05.797109 13691 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:05.803534  9382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:05.791265 13465 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:04:05.804273  9382 raft_consensus.cc:606] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:05.790045 13708 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:04:05.812209  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:05.813114 13760 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.813635 13760 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.814550  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:05.815035  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.811172  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:05.822588  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:05.822512  8749 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:05.824187  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:05.825112  9385 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:04:05.817427  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:05.826223  8794 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:05.826489  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:05.826865  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:05.818980  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:05.827629  9385 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:05.819161  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:05.789660  8793 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.828613  8676 raft_consensus.cc:2410] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:05.780886 13566 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.780714 13645 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:05.836447 13566 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:05.837479  8647 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:05.819254  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:05.829470  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:05.820236  8673 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:04:05.839015  8672 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:05.839923  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:05.840020  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:05.840694  8673 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:05.841882  8673 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:04:05.842145  8677 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:05.831419 13770 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:05.815368  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:05.840696  8672 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:05.843228  8753 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:05.832093  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:05.832212 13708 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:05.832429  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:05.833184  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:05.844000  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 54439.6us
I20260709 13:04:05.844575  9389 raft_consensus.cc:2368] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14.
I20260709 13:04:05.833979  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:05.844995  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.834630  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:05.834941  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:05.835351  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:05.846860  8752 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:05.831337  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:05.847771  8681 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:05.849581 13417 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:05.849925  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:05.850157  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:05.850874  9380 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:05.851251  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:05.850076 13417 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.853183  9383 raft_consensus.cc:2410] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:04:05.855274 13771 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:05.861541 13771 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.861944 13771 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:05.863411  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:05.864652  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:05.868988  8675 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:05.855299 13611 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:04:05.870771 13712 log.cc:927] Time spent T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.532s	user 0.008s	sys 0.000s
I20260709 13:04:05.852206 13708 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:05.870528  8566 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7aff04f53d4845a6a4fa491fd2a06145 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:05.872280 13708 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:05.873677  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:05.855382 13441 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:05.803543 13692 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:05.875385  8681 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:05.876514  8829 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:04:05.877249 13733 log.cc:927] Time spent T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.886s	user 0.003s	sys 0.003s
I20260709 13:04:05.879947 13775 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:05.885353  9381 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:05.887874  8647 leader_election.cc:400] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14.
I20260709 13:04:05.904516  8647 leader_election.cc:403] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:04:05.889412  9353 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:05.857823  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:05.906049  9387 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:05.855297  8680 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:05.860355  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:05.869027  9386 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 758) took 1145 ms. Trace:
I20260709 13:04:05.892236  8752 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:05.896672  8749 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:05.908066  9386 rpcz_store.cc:276] 0709 13:04:04.723529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.724965 (+  1436us) service_pool.cc:224] Handling call
0709 13:04:05.868828 (+1143863us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54001a3bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:05.868967 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326183296}
I20260709 13:04:05.909092  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:05.903721 13657 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.910492 13657 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:05.815840 13760 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:05.916298  9353 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:05.855826 13417 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:05.923975  8647 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:05.898196  9382 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:05.855513  8793 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.934504  9381 raft_consensus.cc:606] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:05.935182  9381 raft_consensus.cc:614] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:05.936404  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 81639.9us
I20260709 13:04:05.902915  9380 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:05.939484  8647 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.940802  8796 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:05.949923  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.954083 13765 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:05.955047 13765 raft_consensus.cc:697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:05.956499 13765 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:05.920459 13740 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:05.982129  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:05.921959 13672 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:05.986325  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 131737us
I20260709 13:04:05.930418 13726 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14.
I20260709 13:04:05.925595 13692 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:05.995635 13672 raft_consensus.cc:697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:05.935706 13691 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:05.948808 13679 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:06.003805  8673 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:05.988231 13772 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:05.965760 13710 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:05.965636 13760 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:06.018446  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:06.005880 13772 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.025146  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:06.005003 13740 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:06.016264  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.009392 13710 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:05.965363 13770 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:04:06.015890 13465 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:06.031756  8828 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:04:06.034225  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:06.024029 13692 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:06.015597 13488 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:06.083226  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 115060us
I20260709 13:04:06.099784 13488 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.035928 13710 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:06.028609 13772 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:06.119119  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.035744 13770 consensus_queue.cc:1243] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:06.122596 13488 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:06.124965  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:06.037357 13465 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:06.093789  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.088567 13704 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:06.041164  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 59197.7us
I20260709 13:04:06.037636 13566 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.099115  8673 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
W20260709 13:04:06.129838 13779 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 13:04:06.144948 13790 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:04:06.131222 13788 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:06.152014  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.152736  8830 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
W20260709 13:04:06.132081 13777 log.cc:927] Time spent T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:04:06.154548  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.155426  8822 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:04:06.132853 13781 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:04:06.133239 13417 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.156637  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.157433  8822 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:06.158111 13417 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:06.134831  9388 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:04:06.160640  8678 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:06.143803  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:06.149305  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.150998  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:06.161623  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.162112  8826 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:06.154233  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.160498  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:06.147713  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:06.154138  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.163516  9380 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:06.158315 13566 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:06.090950 13740 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.158284  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.193604  8830 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:06.202939 13760 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:06.143528 13465 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.093124 13672 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.199493  8824 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:06.213589  9352 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:06.242208 13765 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.246388 13765 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.248397 13523 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:06.248769 13765 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:06.248914 13523 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.249357 13523 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:06.222275  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:06.251228  9382 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:06.242095 13417 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.252277 13417 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.253347 13698 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:06.253896 13698 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.230320 13691 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.243666 13726 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:06.254571 13417 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:06.258955 13691 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.259071 13726 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.261637 13691 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:06.256249 13775 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:04:06.257098 13772 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.269203 13771 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.269938 13772 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:06.270285  8825 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 12. Ops: []
I20260709 13:04:06.152594 13704 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:06.226356  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 82847.3us
I20260709 13:04:06.273109  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:06.279455  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:06.234596  8677 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:06.235562  8828 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:06.242841  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:06.232056  8719 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:06.284794  8678 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
W20260709 13:04:06.284922  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 52969.9us
I20260709 13:04:06.244263  9380 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:06.281560  8793 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:06.287791  8830 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:06.288211  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:06.288144  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.289846  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:06.261730 13726 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:06.231230 13794 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:06.304621  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.307906  8824 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:06.258673 13566 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:04:06.314450 13585 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:06.315155 13585 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.317543 13585 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:06.298554 13472 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:04:06.322382 13796 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:06.285813 13523 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.323220 13796 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:06.323688 13796 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:06.325522 13789 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:04:06.325798 13679 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.326361 13679 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:06.328907  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.300805 13647 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:06.302106 13720 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:06.258024 13710 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.309742 13727 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:06.343247 13465 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.311501 13736 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:06.238590 13760 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.354169  8828 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: [11.12-13.13]
I20260709 13:04:06.354305  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.355289  8824 raft_consensus.cc:2410] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:04:06.337329 13566 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:06.334600 13647 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:06.339411 13720 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.368510  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:06.371763  8821 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:06.349900 13727 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.373752  8674 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 13:04:06.375211  8674 pending_rounds.cc:85] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 12
I20260709 13:04:06.375587  8674 pending_rounds.cc:107] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 13:04:06.375923  8674 raft_consensus.cc:2889] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:06.372071 13698 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:06.377278  8830 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:04:06.377170  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:06.383944  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 135991us
I20260709 13:04:06.362963 13566 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.374475  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.370751 13647 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:06.388130  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.373497 13720 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:06.389425  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.389093  8821 raft_consensus.cc:2368] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13.
I20260709 13:04:06.372964 13784 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:06.388561 13708 consensus_meta.cc:287] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.516s	user 0.016s	sys 0.004s
I20260709 13:04:06.377622 13727 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:06.399652 13784 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.403012 13771 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:06.404780 13784 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:06.388113 13566 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:06.393039  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:06.398734  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 136297us
I20260709 13:04:06.394938  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.412338  8821 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:06.400137  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:06.415001  8827 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:06.415786 13585 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:06.417910 13585 raft_consensus.cc:993] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: : Instructing follower 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:04:06.419757 13465 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:06.421509  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:06.422003  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:06.422708  8679 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:06.422943  8678 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:06.378486 13794 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.409391  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.413491  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.427711  8821 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:06.393170  8824 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
W20260709 13:04:06.434070  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 147398us
I20260709 13:04:06.439465  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.440662  8719 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:06.418359 13585 raft_consensus.cc:1081] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Signalling peer 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:04:06.431491  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:06.442656  8678 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:04:06.427502  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.446103  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:06.447695  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.452517  8827 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
W20260709 13:04:06.463104  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 53525.1us
I20260709 13:04:06.472536  8679 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:06.464109  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:06.424232 13704 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:06.490818  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.495482  8722 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:06.496392  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 56497.3us
I20260709 13:04:06.497359  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:06.410967 13708 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.503628 13498 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:06.504220 13498 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:06.504393  8679 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:55676
I20260709 13:04:06.505113  8679 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:06.505584  8679 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.505153 13498 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.506685 13648 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:06.507593 13802 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:06.510587 13679 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:06.512673 13799 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:04:06.514279 13800 raft_consensus.cc:993] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: : Instructing follower e814ed2013244fa6be42623ea5c0c7b9 to start an election
I20260709 13:04:06.514909 13800 raft_consensus.cc:1081] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Signalling peer e814ed2013244fa6be42623ea5c0c7b9 to start an election
I20260709 13:04:06.516050  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:06.516316 13676 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:06.516917 13648 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:06.518594 13805 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:06.522928 13803 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:06.523274 13679 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:06.523133 13472 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:06.523933 13803 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:06.522481  8679 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.524978 13472 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:06.526635  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:06.527631  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:06.527798  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:06.531682  9352 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:06.536140  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.506455 13498 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:06.544783  8830 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:04:06.544119  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.548508  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:06.544862 13727 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.550674  8679 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:06.551935 13708 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:06.552191 13727 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:06.552963 13647 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.553771  8821 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:04:06.554703  8823 raft_consensus.cc:2368] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:04:06.433211 13794 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.542364 13704 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:06.562337  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:06.542932 13648 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:06.554682 13678 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:04:06.566687  8829 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.548336 13585 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:06.543162 13803 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:06.555857 13710 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:06.575170  8828 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:04:06.570842  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 48306.1us
I20260709 13:04:06.548962 13801 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:06.580909  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:06.581151  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:06.582388  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:06.582410  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:06.582842  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:06.583624  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:06.584061  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:06.584133  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:06.585237  8749 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:06.586171 13726 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:06.586930 13726 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:06.587270 13726 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:06.587653  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:06.587666 13726 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:06.588248  8756 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:06.582329  8748 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:06.568076 13648 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:06.566375 13704 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:06.625068 13704 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:06.625865 13802 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:06.627117 13802 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:06.589529  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:06.627490  8821 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:55168
I20260709 13:04:06.579134 13803 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.595643  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:06.629586 13657 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:06.630910 13657 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.630980  8824 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:06.631353 13657 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:06.627516 13802 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:06.632764 13802 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.570709 13585 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.634840 13585 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:06.598309  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.630234  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:06.636893 13691 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:06.637153  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:06.638118 13691 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:06.634357 13802 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:06.602276  8751 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:06.603379 13720 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.604256 13794 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:06.617341  8793 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:06.618340  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.621731  8825 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:06.623080 13801 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:06.624207 13370 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:06.588840 13713 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:06.652792  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:06.653644  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:06.629813  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:06.654325  8680 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:06.597544  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:06.641731  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:06.641736  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.656733  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:06.658862  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:06.660706  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:06.660614 13801 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.647522 13803 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:06.661926  8755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:04:06.654753 13710 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:06.662541  8755 raft_consensus.cc:606] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:06.662976  8755 raft_consensus.cc:614] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:06.660044  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:06.664119  8674 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:06.649206  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:06.665117  9384 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:06.599568 13676 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:06.642175  8719 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:06.628132  8821 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:06.666705  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:06.672497  8821 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:06.673130  8678 raft_consensus.cc:2393] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:06.673913  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:06.648943  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 123475us
I20260709 13:04:06.668953  8829 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:06.638419  8647 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:06.665673  9384 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:06.676690  8827 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:06.675486  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:06.683491  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:06.684378  8752 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:06.684876  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46515.4us
I20260709 13:04:06.686224  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:06.682493 13803 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:06.702780 13803 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.687274  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:06.711462  8674 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:06.690121 13776 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.901s	user 0.003s	sys 0.000s
I20260709 13:04:06.703694  9384 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.690758 13417 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:06.625486 13648 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.666167 13691 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.717429  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:06.649466 13796 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.679579 13647 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:06.708757  8821 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.757861  9384 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.763335  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:06.710696 13765 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:06.748876 13803 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:06.757867 13417 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.769590  8821 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:06.769755  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:06.768074  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:06.779153  8748 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:06.780087  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:06.775784  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:06.781888  8753 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:04:06.749337 13676 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:04:06.748549 13566 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:06.673924  8756 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:06.785472  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:06.818946  8751 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:06.831004  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:06.831887  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.779799 13417 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:06.817138  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:06.780073 13648 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:06.760084 13691 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.846962  8756 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:04:06.801614  8674 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:06.850893  9388 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:04:06.863511  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:06.864450  8674 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:06.864315  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:06.866294  9385 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:06.866726  9385 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:06.873301  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:06.784153 13813 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
W20260709 13:04:06.885259  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 178259us
W20260709 13:04:06.886626  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 204688us
I20260709 13:04:06.867517  9385 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.888326  9385 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:06.895589  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:06.896391  8751 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
W20260709 13:04:06.897656  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 222264us
I20260709 13:04:06.898404  8752 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:06.919974  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:06.927135  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 225612us
W20260709 13:04:06.930821  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 44036.6us
I20260709 13:04:06.933049  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:06.838416 13765 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:04:06.942837  9385 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:04:06.904728 13727 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:06.944906  8796 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:06.957253  9353 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:06.968272  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:06.969283  8674 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:06.978540  8794 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:06.985880  8794 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:06.986719  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:06.951417 13692 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:06.916098 13710 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:04:06.954807 13720 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:06.927217 13704 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:06.978161 13647 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:06.971382 13800 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.977324 13679 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:06.998776 13772 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:07.105733 13772 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:07.106789 13772 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:07.107467 13657 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.108800  8678 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:04:07.110535  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.112843  9384 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:07.035755 13704 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:07.036090  9381 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:04:07.036445  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.042747 13800 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:07.027851 13679 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:07.001298 13760 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:07.061046 13818 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:04:07.061376  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:07.064388  8674 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:06.988210 13758 consensus_queue.cc:1059] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:04:07.074947 13824 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.075090 13813 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:07.083348 13566 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:07.085358  8679 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:04:07.050244 13585 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:07.091089  8828 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:06.998471 13794 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:07.096473 13826 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:07.097318  8672 pending_rounds.cc:85] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 7
I20260709 13:04:07.097577  8747 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:04:07.136346  8676 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:04:07.098414 13822 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:04:06.983382 13698 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:04:07.136874  8676 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:07.098946 13832 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:04:07.103869 13691 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:07.104658 13796 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:07.141620  8676 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:07.153949  8676 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:07.155570  8679 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:04:07.135246 13710 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:07.190897  8679 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:04:07.191391  8679 pending_rounds.cc:85] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 8
I20260709 13:04:07.134778 13760 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:07.109923 13657 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:07.134934  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:07.120671 13679 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:07.136408  8672 pending_rounds.cc:107] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:04:07.213776  8674 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:07.213959  8672 raft_consensus.cc:2889] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:07.138021  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:07.191644  8679 pending_rounds.cc:107] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 11.9
W20260709 13:04:07.192605  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 236049us
I20260709 13:04:07.216094  8679 raft_consensus.cc:2889] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:07.129309  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 196363us
I20260709 13:04:07.151057 13794 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:07.189183 13765 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:07.192317 13710 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:07.231454  8674 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:07.209728  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:07.232568  8673 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:07.232875  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 334292us
I20260709 13:04:07.216938  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:07.192813 13691 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:04:07.208214 13585 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:07.220850  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:07.222183  8750 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:04:07.223573  9353 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:07.224906  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:07.130558 13824 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.239542  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:07.214986 13726 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.001s
W20260709 13:04:07.226935  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.192441 13826 raft_consensus.cc:3055] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:07.251776  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:07.252554  8750 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:04:07.235903  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:07.254638  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:07.242249  8748 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:07.259656  8645 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:07.259677  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.263770  8645 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:07.263926  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:07.265213  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:07.266072  8645 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:07.230896 13765 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:07.283360  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 395804us
I20260709 13:04:07.286473  8722 leader_election.cc:400] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13.
I20260709 13:04:07.202504 13417 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.258574 13832 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:07.256673 13775 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:07.256062 13826 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:07.256139 13796 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:07.277040 13691 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:07.290488  8722 leader_election.cc:403] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:07.244611 13837 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:07.259126 13713 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.299563  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:07.288121 13765 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:07.258154 13657 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:04:07.307709  9385 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:04:07.266496 13726 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.311139  8722 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:07.374130  8674 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:07.276110 13727 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:04:07.442734 13727 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.443507 13727 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:07.291989 13834 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:07.304600 13775 raft_consensus.cc:697] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:07.447995  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:07.448192 13775 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:07.308072 13832 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.309350 13794 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.302251 13710 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.303308 13835 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:07.455250 13835 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:07.302136 13691 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:07.456130 13691 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:07.309887 13796 raft_consensus.cc:697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:07.457298 13796 consensus_queue.cc:237] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.402474 13839 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:04:07.403453 13836 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:04:07.459975 13835 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.380348 13585 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:07.461442 13585 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:07.407373  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 173692us
I20260709 13:04:07.462466 13772 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:07.408260 13823 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:07.470394  8679 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:04:07.471680  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:07.472914  8675 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:07.430444 13800 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:07.448004 13834 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.478375 13834 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.491478 13834 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:07.426582 13824 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:07.493216 13691 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:07.496372 13828 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.284s	user 0.003s	sys 0.003s
I20260709 13:04:07.443428 13726 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:07.474124  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:07.412528 13469 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:07.499145 13469 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:07.499993  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:07.500141 13469 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.501528 13469 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:07.424762 13417 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:07.427452 13795 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:04:07.503437 13611 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.301733 13523 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:07.374305 13713 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.292609 13760 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.436731 13370 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13.
I20260709 13:04:07.438946 13826 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.440512 13837 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:07.510143 13713 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:07.394106 13765 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:07.504240 13611 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.512076  8679 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:07.495641 13691 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:07.522140  8676 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
W20260709 13:04:07.524070  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:07.524992  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 214011us
I20260709 13:04:07.525861  8677 raft_consensus.cc:3055] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:07.527721  8680 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:07.529050  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:07.508001 13498 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
W20260709 13:04:07.534538  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 269403us
I20260709 13:04:07.537551  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:07.517841  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:07.446486 13727 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:07.528779 13802 consensus_meta.cc:287] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.884s	user 0.014s	sys 0.000s
I20260709 13:04:07.549304 13585 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:07.540143 13692 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:04:07.550077 13611 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:07.556831 13765 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:07.529014 13370 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:07.630818 13370 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.566675 13710 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:07.631450 13370 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:07.568126 13803 consensus_meta.cc:287] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.781s	user 0.013s	sys 0.000s
I20260709 13:04:07.534492 13740 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:04:07.573426  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:07.576351  8679 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:07.576824  8750 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:07.565833 13679 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:07.614732  8676 pending_rounds.cc:85] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 9
I20260709 13:04:07.599334  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2150) took 1586 ms. Trace:
I20260709 13:04:07.615595  8681 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:07.615934  8751 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:07.635646  8748 raft_consensus.cc:2410] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:04:07.636808  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:07.648352  8749 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:07.640823  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:07.651006  8676 pending_rounds.cc:107] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:04:07.651412  8676 raft_consensus.cc:2889] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:07.651518  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:07.638372  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:07.617892  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:07.645051 13835 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:07.653591  8566 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7aff04f53d4845a6a4fa491fd2a06145 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:07.620106 13648 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.662739 13648 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.665248 13648 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:07.653891 13835 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.618255  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:07.597532 13853 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:07.592895  8578 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43478 (request call id 256) took 35673 ms (client timeout 15000 ms). Trace:
W20260709 13:04:07.627314  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 98340.1us
I20260709 13:04:07.630893 13849 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:07.680011 13849 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:07.680133 13370 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.681216 13849 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:07.683514 13370 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:07.686355 13702 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:07.686985 13370 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:07.689025  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:07.617949  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 346450us
I20260709 13:04:07.692235  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:07.692942  8753 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:04:07.640877  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:07.670204  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 52397.4us
W20260709 13:04:07.671501  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 53344.9us
I20260709 13:04:07.646726  8823 rpcz_store.cc:276] 0709 13:04:06.012966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.013964 (+   998us) service_pool.cc:224] Handling call
0709 13:04:07.599008 (+1585044us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b54003acad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:07.599247 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1198358912}
I20260709 13:04:07.618474  8677 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:07.676561 13657 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:07.677076 13648 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:07.678418 13835 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:07.691350  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:07.694063  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:07.579210  8673 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:07.706815  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:07.710390  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:07.721194  9388 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:07.716532  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:07.603579 13678 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 13:04:07.649276 13692 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.722437  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:07.740398  8673 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:04:07.741852  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:07.742165  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:07.715080  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:07.698864  8645 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:07.652902  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:07.743757  8673 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:07.743853  8676 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:07.323463  8752 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:07.745810  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:07.747511  8719 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:07.696084  8578 rpcz_store.cc:269] 0709 13:03:31.919831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.920066 (+   235us) service_pool.cc:224] Handling call
0709 13:03:35.902872 (+3982806us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b5400002490. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d50b2 00007f32e834ea89 00007f32e82e6da6 00007f32e82b8088 00007f32e83b637e 00007f32e68772a4 00007f32e6877232 00007f32e6877161 00007f32e68770dc
0709 13:04:07.592606 (+31689734us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:00.025719 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:00.029613 (+  3894us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:00.029677 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:04:00.029695 (+    18us) tablet.cc:662] Decoding operations
0709 13:04:00.085678 (+ 55983us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:00.085811 (+   133us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:00.085850 (+    39us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:04:00.117578 (+ 31728us) tablet.cc:701] Row locks acquired
0709 13:04:00.117649 (+    71us) write_op.cc:260] PREPARE: finished
0709 13:04:00.117840 (+   191us) write_op.cc:270] Start()
0709 13:04:00.118016 (+   176us) write_op.cc:276] Timestamp: P: 1783602240117768 usec, L: 0
0709 13:04:00.118039 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:04:00.118906 (+   867us) log.cc:844] Serialized 6305 byte log entry
0709 13:04:00.819450 (+700544us) op_driver.cc:464] REPLICATION: finished
0709 13:04:00.820986 (+  1536us) write_op.cc:301] APPLY: starting
0709 13:04:00.821090 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:01.552729 (+731639us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:01.552932 (+   203us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:01.851694 (+298762us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:01.852214 (+   520us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 13:04:01.852319 (+   105us) write_op.cc:312] APPLY: finished
0709 13:04:01.853023 (+   704us) log.cc:844] Serialized 148 byte log entry
0709 13:04:01.853428 (+   405us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:01.853739 (+   311us) write_op.cc:454] Released schema lock
0709 13:04:01.853836 (+    97us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:01.854544 (+   708us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:01.854574 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":29597,"reported_tablets":50,"spinlock_wait_cycles":43221248,"child_traces":[["op",{"apply.queue_time_us":1171,"apply.run_cpu_time_us":33143,"apply.run_wall_time_us":1033550,"cfile_cache_hit":66,"cfile_cache_hit_bytes":70943,"num_ops":16,"prepare.queue_time_us":115894,"prepare.run_cpu_time_us":17913,"prepare.run_wall_time_us":542621,"raft.queue_time_us":51740,"raft.run_cpu_time_us":1475,"raft.run_wall_time_us":1477,"replication_time_us":701215,"thread_start_us":546628,"threads_started":4,"wal-append.queue_time_us":535297,"wal-append.run_cpu_time_us":20079,"wal-append.run_wall_time_us":2198985}]]}
I20260709 13:04:07.726311 13832 raft_consensus.cc:3055] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:07.674574 13813 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:07.731937  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 58655.6us
I20260709 13:04:07.763777 13837 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:07.740629 13848 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:04:07.766841 13657 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:07.767997 13657 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:07.774721  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:07.776621  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:07.779891  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 64967.6us
I20260709 13:04:07.781000  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:07.781941  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 83160.6us
I20260709 13:04:07.783989  8645 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:07.634829 13803 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.666596 13802 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.678306 13679 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:07.736120  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:07.737239  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:07.738904  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:07.722900  8647 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:07.825176  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 106594us
I20260709 13:04:07.729326 13847 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:07.723668  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:07.834744  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 111130us
I20260709 13:04:07.835721  8747 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:07.717052 13834 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:07.646960 13850 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.138s
I20260709 13:04:07.761579 13732 raft_consensus.cc:3055] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:07.716532 13845 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:07.818272 13800 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.823638 13842 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:07.825268  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:07.816967  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:07.833822  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:07.834013  8793 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.528s (SENT) [suppressed 18 similar messages]
I20260709 13:04:07.751561 13832 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:07.841593  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2628) took 1176 ms. Trace:
I20260709 13:04:07.842420  8680 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:04:07.697845 13853 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.813746 13691 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:07.812939 13370 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:07.813555  9388 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:07.818581 13648 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:07.823483 13813 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:07.732504  8751 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:07.840653 13835 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:07.840337 13348 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:07.924079 13852 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:04:07.772742 13692 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:07.907251  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 90377.2us
I20260709 13:04:07.877300 13497 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 13:04:07.846942 13585 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:07.848994 13732 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:07.931939  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:07.878682 13845 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:07.973803 13845 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:07.848687 13523 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:04:07.839552 13802 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:07.975267 13845 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:07.957675  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.843238  8750 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:07.980151 13803 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:07.980352  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:07.961609 13348 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.926964 13765 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:04:07.933046 13819 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:07.964434  8645 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:07.984001  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:07.907011  8678 rpcz_store.cc:276] 0709 13:04:06.664920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.778416 (+113496us) service_pool.cc:224] Handling call
0709 13:04:07.841396 (+1062980us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b5400046d54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:07.841534 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":753721600}
I20260709 13:04:07.866518 13834 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:07.971773 13647 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:07.974323 13826 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:07.826148 13833 consensus_queue.cc:1059] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.097s
I20260709 13:04:07.840505 13726 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 13:04:07.939616 13648 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.946897 13813 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.996481  8680 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:07.947844 13823 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.997438  8678 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:07.908289 13469 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:04:08.000319  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:07.905828 13800 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:08.027799  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:07.921923 13838 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 13:04:07.995656 13765 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:07.978478 13861 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 13:04:07.965137 13835 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:08.025789  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:08.077241  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 243247us
I20260709 13:04:08.048544  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:08.081028  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:08.000841 13647 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:07.998646 13566 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:08.093313  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:08.111802  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 188333us
I20260709 13:04:08.113842  8645 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:08.082234  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:08.117648  8826 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:04:08.119448  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:08.091975  8827 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:04:08.097028  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:08.121132  8821 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:08.011428 13851 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:08.024614 13813 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:08.014046 13648 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:08.014642 13803 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:08.026171 13585 raft_consensus.cc:697] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:08.144796  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:08.147020  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:08.148315  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:08.183835  8830 raft_consensus.cc:2376] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11. Re-sending same reply.
I20260709 13:04:08.129477 13691 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:08.142467  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:08.189728  8827 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:08.191563  8827 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:08.155028  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:08.162195  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 224164us
W20260709 13:04:08.167829 13863 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.395s	user 0.001s	sys 0.000s
I20260709 13:04:08.130455 13855 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:08.128038  8680 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:08.036067 13823 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:08.069574 13839 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:08.080715 13726 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:08.083316 13765 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:08.186854 13851 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:08.185714  8826 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:08.154886 13803 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:08.188091  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:08.167455 13819 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:04:08.128600 13441 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:08.165632  8821 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:08.115402 13729 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
W20260709 13:04:08.218490  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 104783us
I20260709 13:04:08.180323 13566 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:08.224357  8822 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:04:08.183097  8823 raft_consensus.cc:2376] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15. Re-sending same reply.
I20260709 13:04:08.188320 13853 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:08.336799  8645 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:08.216782 13862 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.311s	user 0.000s	sys 0.001s
I20260709 13:04:08.337241  8830 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:08.364286  8751 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:04:08.387272  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 349986us
I20260709 13:04:08.395617 13876 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:08.395740  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:08.396574 13729 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:08.396782  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:08.396342 13876 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:08.397274 13729 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:08.398037  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:08.398691  8719 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:08.374315  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 237691us
I20260709 13:04:08.399405  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:08.400004 13876 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:08.399670 13838 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:08.391943 13726 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.400707 13838 raft_consensus.cc:697] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:08.401204 13726 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:08.401892  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:08.401763 13838 consensus_queue.cc:237] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:08.403038  8826 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:08.168071 13732 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:08.403906  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:08.204407 13814 consensus_queue.cc:1059] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.160s
I20260709 13:04:08.405324  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:08.400112 13729 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:08.338075 13691 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.364012 13837 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:08.420639  8722 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:08.427399  8830 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:08.403708 13726 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:08.436472  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:08.411865 13645 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:08.362962 13441 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:08.476414  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:08.403790 13876 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
W20260709 13:04:08.484872  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 64323.6us
W20260709 13:04:08.494098  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.473704 13497 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:08.428949 13833 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:08.417519 13710 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:08.334658 13835 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:08.479390 13732 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:08.466681 13800 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:08.527019  8825 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
W20260709 13:04:08.529292  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:08.530443  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.469946 13691 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:08.522394 13692 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:08.511776 13710 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:08.545032 13441 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:08.435310 13585 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:08.500280 13566 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:08.547297  8755 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:04:08.545274  8751 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:08.551494  9355 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.009s	sys 0.002s
I20260709 13:04:08.502353 13851 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:04:08.552100  9355 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.009s	sys 0.002s
W20260709 13:04:08.525341 13867 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.490s	user 0.001s	sys 0.001s
I20260709 13:04:08.559866  9355 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:08.545907 13710 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:08.551663  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:08.521682 13708 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:08.528878 13348 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:08.514453 13888 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:08.580704  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:08.582167  8822 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:08.548401 13645 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:08.552007 13834 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:08.554841 13823 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:08.590559 13871 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.226s	user 0.001s	sys 0.001s
W20260709 13:04:08.590811 13884 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
W20260709 13:04:08.592015 13879 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 13:04:08.593390 13885 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:04:08.593611 13370 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.649s	user 0.008s	sys 0.004s
I20260709 13:04:08.574962 13877 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:08.601734 13348 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:08.600461 13888 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:08.664980  8678 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:08.560525 13851 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:08.626343 13853 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:08.634788 13370 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:08.588307 13708 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:08.637298 13691 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:08.694871  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 45002.8us
I20260709 13:04:08.695181  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2701) took 1200 ms. Trace:
I20260709 13:04:08.696368  8756 rpcz_store.cc:276] 0709 13:04:07.495104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.530091 (+ 34987us) service_pool.cc:224] Handling call
0709 13:04:07.540592 (+ 10501us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:07.573209 (+ 32617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.573259 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:07.579296 (+  6037us) log.cc:844] Serialized 52 byte log entry
0709 13:04:07.813714 (+234418us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:07.813774 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.813853 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.694813 (+880960us) raft_consensus.cc:1685] finished
0709 13:04:08.694849 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.695091 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32112,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":7767,"thread_start_us":168196,"threads_started":1,"wal-append.queue_time_us":321066}
I20260709 13:04:08.646960 13710 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:08.698706  8674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.007s	sys 0.003s
W20260709 13:04:08.703186  8674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.008s	sys 0.004s
I20260709 13:04:08.717921  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2226) took 1074 ms. Trace:
I20260709 13:04:08.722504  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2227) took 1078 ms. Trace:
I20260709 13:04:08.723578  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:08.723095  8674 rpcz_store.cc:276] 0709 13:04:07.644174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.981203 (+337029us) service_pool.cc:224] Handling call
0709 13:04:07.981516 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:07.981741 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:08.023240 (+ 41499us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.722122 (+698882us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:08.722191 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:08.722245 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.722335 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.722448 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16604,"prepare.queue_time_us":44379,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"thread_start_us":684115,"threads_started":1}
I20260709 13:04:08.701601 13895 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.718691  8679 rpcz_store.cc:276] 0709 13:04:07.643765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.929172 (+285407us) service_pool.cc:224] Handling call
0709 13:04:07.929597 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:07.942454 (+ 12857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.942500 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:07.943191 (+   691us) log.cc:844] Serialized 26 byte log entry
0709 13:04:08.222719 (+279528us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:08.222769 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.222846 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.717208 (+494362us) raft_consensus.cc:1685] finished
0709 13:04:08.717244 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.717835 (+   591us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":856,"raft.queue_time_us":703,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":7893,"thread_start_us":265425,"threads_started":1,"wal-append.queue_time_us":520257}
I20260709 13:04:08.696147 13645 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:08.739280 13900 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:08.740043 13900 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:08.742578 13851 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:08.694271 13853 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:08.745297 13853 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:08.682408 13348 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:08.708988 13710 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:08.728885 13691 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.752264  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:08.725687 13839 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:08.755872  8674 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:08.731957 13878 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:08.742501 13900 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:08.825152  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:08.826090  8749 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
W20260709 13:04:08.831327 13826 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.819s	user 0.013s	sys 0.000s
I20260709 13:04:08.756417 13691 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:08.731062 13895 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:08.841490 13895 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:08.818650 13878 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.821549 13835 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.768306 13888 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:08.890111  8722 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:08.892268  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:08.846207  9388 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:04:08.853657  9384 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:08.854346  9381 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:04:08.863262  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2228) took 1218 ms. Trace:
I20260709 13:04:08.867347  8750 raft_consensus.cc:2410] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:04:08.904299  8793 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:08.903502  8681 rpcz_store.cc:276] 0709 13:04:07.644511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.985800 (+341289us) service_pool.cc:224] Handling call
0709 13:04:07.995966 (+ 10166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:08.041953 (+ 45987us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:08.042010 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:08.042077 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:08.042128 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.862795 (+820667us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b54001fa254. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:08.862885 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.863129 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104899456}
I20260709 13:04:08.889714  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2223) took 1420 ms. Trace:
I20260709 13:04:08.854748  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:08.852349  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:08.903087  8823 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:04:08.846558  9353 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:08.903504  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:08.936987  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:08.940253  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 93800.6us
I20260709 13:04:08.932420  8673 rpcz_store.cc:276] 0709 13:04:07.469460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.744669 (+275209us) service_pool.cc:224] Handling call
0709 13:04:07.744878 (+   209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:07.960050 (+215172us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:07.997303 (+ 37253us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.199307 (+202004us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:08.199432 (+   125us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:08.199569 (+   137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.889417 (+689848us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54003a20d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:08.889473 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.889627 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10496,"prepare.queue_time_us":37727,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":39477,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":8165,"spinlock_wait_cycles":494182144,"thread_start_us":193361,"threads_started":1,"wal-append.queue_time_us":391236}
I20260709 13:04:08.941488  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:08.942091  8673 raft_consensus.cc:2376] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13. Re-sending same reply.
I20260709 13:04:08.942464  9353 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:08.943560  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:08.944336  8673 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 14: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:08.944329  9385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 954) took 1265 ms. Trace:
I20260709 13:04:08.945246  9385 rpcz_store.cc:276] 0709 13:04:07.679312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.694479 (+ 15167us) service_pool.cc:224] Handling call
0709 13:04:07.694789 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:07.695002 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:07.915315 (+220313us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.740705 (+825390us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b1800420ca0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e20a14bc 00007f32df3ef48b 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287
0709 13:04:08.943613 (+202908us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.943911 (+   298us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:08.944028 (+   117us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:08.944079 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.944149 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.944266 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11399,"prepare.queue_time_us":275898,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":87721,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":54157,"spinlock_wait_cycles":243830528,"thread_start_us":521076,"threads_started":3}
I20260709 13:04:08.961254 13469 raft_consensus.cc:3055] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:08.962272 13469 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:08.963390 13469 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:08.965200 13469 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:08.965059  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:08.981564  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:08.983285  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:08.984155  8673 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:08.985172  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:08.834200 13710 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:08.858199 13647 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.710s	user 0.011s	sys 0.001s
W20260709 13:04:09.095849  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.929680 13472 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:08.880563 13846 raft_consensus.cc:3055] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:08.872089 13878 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:08.848171 13906 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:04:09.095963  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:08.920022 13732 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:08.846729 13834 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:08.922134 13845 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.926s	user 0.013s	sys 0.002s
I20260709 13:04:08.900437 13853 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:08.885612 13900 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:08.878552 13891 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:09.107070  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:08.938756 13765 consensus_meta.cc:287] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.602s	user 0.008s	sys 0.005s
I20260709 13:04:09.107362  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:08.887327 13441 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.878233 13826 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.846989 13897 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:08.859691 13348 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:09.117486 13880 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.372s	user 0.003s	sys 0.000s
I20260709 13:04:08.979720 13908 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:08.872961 13835 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.169301  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:09.160295  8677 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.223556  8830 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:09.223556  8677 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:09.170032  8826 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:04:09.170362  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:09.170719  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:08.876600 13708 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.167580 13833 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:04:09.173673 13648 consensus_meta.cc:287] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.991s	user 0.014s	sys 0.000s
I20260709 13:04:09.199239 13566 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:09.144541  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:09.167711 13645 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.129208 13702 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:09.217355  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:09.186206 13726 consensus_queue.cc:1059] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:04:09.167299 13902 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.225436  8821 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:09.185259 13853 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.171025  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:09.207557 13585 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.068s	sys 0.117s
I20260709 13:04:09.199079 13732 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:09.138577 13729 raft_consensus.cc:3055] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:09.180989 13846 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:09.212280 13441 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.229036  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:09.182524 13813 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.003s	user 0.013s	sys 0.000s
I20260709 13:04:09.272552 13813 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:09.125015 13890 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.484s	user 0.002s	sys 0.000s
I20260709 13:04:09.147881 13802 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.279917  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2078) took 1185 ms. Trace:
I20260709 13:04:09.237587 13566 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.263566 13729 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:09.231190  8679 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:04:09.205158 13472 raft_consensus.cc:697] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:09.283092 13729 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.283861  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:09.186766 13851 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:04:09.285007 13851 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:09.285393  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:09.285591 13851 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.286281  8679 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 14: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:09.232123  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:09.286559 13889 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.287180  8751 raft_consensus.cc:2410] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:04:09.170003 13691 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.185519 13900 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.288457 13894 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:09.289273 13894 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.232381  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:09.188310  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.291761 13732 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.184695 13834 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.238884 13645 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.227540 13713 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:09.222533 13845 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:09.298765 13713 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:09.299330 13713 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:09.299091 13845 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.301365 13845 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.205874 13765 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.255484  8821 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:09.247624  8827 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:09.303866  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.303974  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.226697  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:09.259150  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.194933 13647 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.257629 13902 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.280725 13835 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.289921 13900 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.285229 13846 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.287222 13889 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.287868 13851 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.237838  8674 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:09.288386 13691 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:09.304939  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:09.267885 13726 raft_consensus.cc:3055] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:09.215986 13826 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.307036  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 919) took 1110 ms. Trace:
I20260709 13:04:09.335265  8679 rpcz_store.cc:276] 0709 13:04:08.196871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.305937 (+1109066us) service_pool.cc:224] Handling call
0709 13:04:09.306278 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.306629 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.306664 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.306728 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:09.306767 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.306830 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.306957 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.337462  8679 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:09.109625 13710 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:09.308586  8830 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:09.309441  8753 raft_consensus.cc:2410] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:09.309794  8673 raft_consensus.cc:2410] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:04:09.280421  8829 rpcz_store.cc:276] 0709 13:04:08.094381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.095096 (+   715us) service_pool.cc:224] Handling call
0709 13:04:08.112674 (+ 17578us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.123809 (+ 11135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:08.123846 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:08.155376 (+ 31530us) log.cc:844] Serialized 26 byte log entry
0709 13:04:08.410982 (+255606us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:08.411030 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.411156 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.833679 (+422523us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b48000cd158. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:04:09.123137 (+289458us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:09.279651 (+156514us) raft_consensus.cc:1685] finished
0709 13:04:09.279681 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.279840 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10584,"raft.queue_time_us":11006,"raft.run_cpu_time_us":3429,"raft.run_wall_time_us":704936,"spinlock_wait_cycles":81119744,"thread_start_us":205050,"threads_started":1,"wal-append.queue_time_us":382145}
I20260709 13:04:09.312321  8827 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:09.313531  8681 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:04:09.220058 13897 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.283738 13472 consensus_queue.cc:237] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:09.265947 13585 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.068s	sys 0.118s
I20260709 13:04:09.234028 13648 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.298377 13645 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.325979  8674 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:04:09.326267  8821 raft_consensus.cc:2368] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14.
I20260709 13:04:09.368867  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:09.185804 13891 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.303442 13348 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.234289 13833 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.358340  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.357566  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.381232  8680 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:09.328387 13726 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:09.324517 13908 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.371608  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.387363  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.388154  8830 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:09.388725  8829 raft_consensus.cc:2410] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:09.373821  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:09.379132  8677 raft_consensus.cc:1275] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:04:09.380589  8674 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:09.389720  8677 pending_rounds.cc:85] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 5
W20260709 13:04:09.363209  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 420740us
I20260709 13:04:09.391942 13908 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:09.392827  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.394564 13908 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.395793  8830 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:04:09.396891 13875 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 1.208s	user 0.001s	sys 0.003s
I20260709 13:04:09.396801 13645 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.397749 13645 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.352373 13678 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.401008 13678 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.390115  8677 pending_rounds.cc:107] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:04:09.402021  8677 raft_consensus.cc:2889] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:09.402866  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2214) took 1936 ms. Trace:
I20260709 13:04:09.396956  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.403488 13851 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.403676 13678 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.401458 13842 raft_consensus.cc:3055] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:09.404158 13851 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.404611 13842 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:09.405596 13842 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.406701 13842 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:09.329105  8574 catalog_manager.cc:5697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 9 to 11, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 11 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:04:09.410748 13827 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:04:09.416891  8674 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:09.403183  8677 rpcz_store.cc:276] 0709 13:04:07.466924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.491040 (+ 24116us) service_pool.cc:224] Handling call
0709 13:04:07.525679 (+ 34639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.159844 (+1634165us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54004e0320. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e247706f 00007f32e24eff60 00007f32e24f17de 00007f32e24fbb49 00007f32e24fc416 00007f32e24f8d8f 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62
0709 13:04:09.402530 (+242686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.402798 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":413451136}
I20260709 13:04:09.419915  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:09.382207  8827 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:04:09.422935  8747 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:04:09.364003 13846 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:09.432761  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:09.436875  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.439880  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:09.453589  8827 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:09.458515  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.459088  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:09.423444 13834 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.339986 13691 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.459820  8794 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:09.465785  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:09.450573  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 194675us
I20260709 13:04:09.361088 13710 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.460457  8680 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
W20260709 13:04:09.461098  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.000s	sys 0.025s
W20260709 13:04:09.471063  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.000s	sys 0.025s
I20260709 13:04:09.386883 13348 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.473511  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:09.475039  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 538101us
I20260709 13:04:09.452292 13566 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.478588  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:09.469090  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:09.484995  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.410027  8574 catalog_manager.cc:5697] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 8 to 11, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 11 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:09.488248  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:09.489200 13708 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:09.489976 13708 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:09.490792 13708 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:09.491052  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:09.491726  8674 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:09.492751  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:09.493548 13566 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:09.494076 13566 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.494489 13566 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:09.489223  8574 catalog_manager.cc:5697] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 2 to 6, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 6 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:09.497035  8827 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:04:09.497092  8574 catalog_manager.cc:5697] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 5 to 8, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 8 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:04:09.502707 13900 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.503396 13900 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.505236  8827 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:09.505869 13900 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.502597  8574 catalog_manager.cc:5697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 6 to 8, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 8 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:09.506899 13566 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.509404  8574 catalog_manager.cc:5697] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:09.512818  8827 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:09.514930  8823 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:04:09.493283  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:09.516036  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.517130  8829 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:09.520929  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:09.518846  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:09.530059  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:09.471784 13692 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:09.519615  9389 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:04:09.467199 13702 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:04:09.534816  8830 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:09.538381  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.548583  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:09.537180  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:09.511372 13834 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:09.543085  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:09.550731 13853 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.553364 13441 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.553965 13691 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.554415 13833 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.518155 13926 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
W20260709 13:04:09.558177  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.482218 13908 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
W20260709 13:04:09.561587  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.534183 13765 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.533247 13921 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:09.577513  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 137673us
I20260709 13:04:09.584107  8574 catalog_manager.cc:5697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 8 to 11, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 11 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:09.542667 13692 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.586987 13692 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:09.600260 13713 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.605751 13889 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.609167 13760 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:09.609386  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 970) took 1054 ms. Trace:
I20260709 13:04:09.610244 13889 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:09.610958 13889 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.612152 13894 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.533264 13922 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:09.578423  9355 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:09.609882  8825 rpcz_store.cc:276] 0709 13:04:08.555001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.567994 (+ 12993us) service_pool.cc:224] Handling call
0709 13:04:08.568539 (+   545us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:08.579792 (+ 11253us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:08.695511 (+115719us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.897942 (+202431us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:08.931482 (+ 33540us) log.cc:844] Serialized 52 byte log entry
0709 13:04:08.931700 (+   218us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:08.932020 (+   320us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.420698 (+488678us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b180038a040. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:09.608361 (+187663us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b180038a040. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e5ef13c7 00007f32e25039a7 00007f32e25130bf 00007f32e2513038
0709 13:04:09.608656 (+   295us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.608930 (+   274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.609051 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:09.609105 (+    54us) raft_consensus.cc:1685] finished
0709 13:04:09.609120 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.609313 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31839,"prepare.queue_time_us":228297,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":62,"raft.queue_time_us":4926,"raft.run_cpu_time_us":696,"raft.run_wall_time_us":63813,"spinlock_wait_cycles":222124416,"thread_start_us":272045,"threads_started":2,"wal-append.queue_time_us":399622}
I20260709 13:04:09.549751 13927 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:09.597033 13897 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.615954 13878 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.646943 13891 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.631983 13922 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.648432 13878 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.641446 13927 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:09.649857 13922 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.651305 13710 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.650249 13927 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:09.653785  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 156188us
I20260709 13:04:09.642115 13902 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.653785 13889 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.665649 13848 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:09.632649 13894 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.666899 13894 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.674249  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:09.676152  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 158955us
W20260709 13:04:09.664665  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 86321.7us
I20260709 13:04:09.678426  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.678804  9355 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:09.679245  8827 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:09.679705 13713 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:09.680354 13702 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:09.681170  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.682793 13878 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:09.683897 13894 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.614583 13921 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:09.668373 13710 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.687090  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:09.687830  8829 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:09.687780 13710 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.668043 13926 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:09.687127 13921 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.690795  8674 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:09.691193 13926 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.693758 13926 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.672747 13648 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.704097 13921 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:09.708101  8829 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:09.709826 13921 consensus_queue.cc:1059] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:09.712137 13921 consensus_queue.cc:1243] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:09.712668 13922 raft_consensus.cc:3055] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:09.713081 13922 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:09.713256  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:09.713958 13922 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.715118 13922 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:09.713965  8821 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:09.719146 13647 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.721159 13645 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.722002  8674 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:09.730265  8821 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:09.710850  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:09.731191  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:09.734309  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:09.736032  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:09.740245  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.738111  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.823623  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:09.824877  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:09.827356  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:09.827607  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.828620  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.829514  8825 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 17: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:09.745868  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:09.735903  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.831234  8821 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:09.832173  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.782410  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:09.783088  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:09.820755  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.833762  9385 raft_consensus.cc:2393] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:04:09.843423  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.732805  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:09.840096  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.859627  8824 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:04:09.863965  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:09.742717 13891 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:09.869946  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.871977  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.745584 13647 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.743567 13645 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:09.870144  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 137422us
I20260709 13:04:09.819491 13699 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:09.858580 13566 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:09.895988  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.866744 13835 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.856729 13921 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:09.896864  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.868798  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:09.746493 13850 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.136s
I20260709 13:04:09.812556 13897 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:04:09.880501 13878 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:04:09.912716  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 48276.8us
I20260709 13:04:09.902539  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.911660  8821 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:09.932888  8647 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:09.886109 13889 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:09.887743 13710 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:09.870082 13834 consensus_queue.cc:1059] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:04:09.870023 13824 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:04:09.940291  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:09.955211  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.929857 13891 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:09.961097  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:09.904186 13923 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:04:09.966270  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:09.891438 13647 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.994073  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2087) took 1106 ms. Trace:
I20260709 13:04:09.891839 13645 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.990060  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:09.947232 13765 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:09.994765  8828 rpcz_store.cc:276] 0709 13:04:08.887425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.896269 (+  8844us) service_pool.cc:224] Handling call
0709 13:04:08.943970 (+ 47701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:08.944244 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:09.862648 (+918404us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.993644 (+130996us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.993711 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:09.993778 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.993860 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.993999 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":665955,"prepare.queue_time_us":1042895,"thread_start_us":381743,"threads_started":2}
I20260709 13:04:09.901270 13699 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.002947  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:09.953811 13878 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:10.005295  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 99658.5us
I20260709 13:04:09.946300 13566 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:04:09.971176 13850 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:09.931113 13921 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.924312 13835 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:09.992025 13834 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:10.015573  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:09.815372  8574 catalog_manager.cc:5697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 4 to 8, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 8 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:04:10.007462  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:10.019593  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:10.011694 13691 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:04:10.027369 13935 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 13:04:10.027539 13859 log.cc:927] Time spent T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 2.348s	user 0.005s	sys 0.006s
I20260709 13:04:10.029946  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2620) took 3368 ms. Trace:
I20260709 13:04:10.031595 13699 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:10.030414  8672 rpcz_store.cc:276] 0709 13:04:06.661895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.662037 (+   142us) service_pool.cc:224] Handling call
0709 13:04:07.096675 (+434638us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:04:07.096978 (+   303us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:07.215015 (+118037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.215068 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:07.217579 (+  2511us) log.cc:844] Serialized 52 byte log entry
0709 13:04:07.472220 (+254641us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b180022a5c0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:04:07.536684 (+ 64464us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:07.549003 (+ 12319us) log.cc:844] Serialized 14 byte log entry
0709 13:04:07.594361 (+ 45358us) log.cc:844] Serialized 14 byte log entry
0709 13:04:07.594603 (+   242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.594736 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:09.954775 (+2360039us) spinlock_profiling.cc:243] Waited 9.86 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:10.029636 (+ 74861us) raft_consensus.cc:1685] finished
0709 13:04:10.029663 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.029870 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68211,"prepare.queue_time_us":68182,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":103,"raft.queue_time_us":48203,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":10817,"spinlock_wait_cycles":38592384,"thread_start_us":64174,"threads_started":1,"wal-append.queue_time_us":461417}
I20260709 13:04:10.033380  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:10.034394  8672 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:10.036435 13645 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:10.037418 13878 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.024809 13850 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:10.041472 13850 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:10.041909 13850 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:10.033331  8574 catalog_manager.cc:5697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 5 to 7, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 7 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:10.052946 13923 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.053728 13923 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:10.057535  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.057746 13846 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.620s	user 0.007s	sys 0.006s
I20260709 13:04:10.058777  8672 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:10.060046  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.059841 13850 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.060850  8672 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:10.062028  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.053158  8574 catalog_manager.cc:5697] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 6 to 8, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 8 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:10.063066  8672 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:10.064073  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:10.063586  8574 catalog_manager.cc:5697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 5 to 8, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 8 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:10.064759  8672 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:10.015879 13853 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:10.078078 13702 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.078925 13849 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:10.015198 13833 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:04:10.095866  8672 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:10.095052  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:10.097792  8821 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
W20260709 13:04:10.104243 13898 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 1.470s	user 0.002s	sys 0.000s
W20260709 13:04:10.112362 13918 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.648s	user 0.000s	sys 0.002s
I20260709 13:04:10.115319  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:10.117475  8672 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:10.120435  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.122651  8672 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:04:10.115861  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2088) took 1228 ms. Trace:
I20260709 13:04:10.124071  8822 rpcz_store.cc:276] 0709 13:04:08.887810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.887958 (+   148us) service_pool.cc:224] Handling call
0709 13:04:08.888406 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.888881 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:08.888919 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:08.889615 (+   696us) log.cc:844] Serialized 26 byte log entry
0709 13:04:09.210550 (+320935us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:09.210598 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.223889 (+ 13291us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.111767 (+887878us) spinlock_profiling.cc:243] Waited 3.51 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:10.115507 (+  3740us) raft_consensus.cc:1685] finished
0709 13:04:10.115531 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.115793 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":793,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":49,"spinlock_wait_cycles":8064384,"thread_start_us":320568,"threads_started":1,"wal-append.queue_time_us":493789}
W20260709 13:04:10.128585  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.129190  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.131100 13887 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:10.131599 13887 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:10.128361  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.133704  8672 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:10.136111  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.137781  8672 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:10.069101 13923 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:10.139490  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 103756us
I20260709 13:04:10.091686 13853 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.141657  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.144042  8822 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:10.096949 13702 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.148054 13941 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:10.149407 13833 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:10.151451 13891 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:10.163380 13943 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:04:10.096567 13834 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.091672 13897 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:10.051304 13925 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.129s
I20260709 13:04:10.132498 13887 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.173069 13850 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:10.176988  8680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2630) took 1653 ms (client timeout 1688 ms). Trace:
I20260709 13:04:10.177325  8679 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:04:10.178200 13692 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.591s	user 0.012s	sys 0.000s
W20260709 13:04:10.178062  8681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2646) took 1293 ms (client timeout 1513 ms). Trace:
W20260709 13:04:10.201062  8681 rpcz_store.cc:269] 0709 13:04:08.884863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.979341 (+ 94478us) service_pool.cc:224] Handling call
0709 13:04:10.177847 (+1198506us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400353c54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:10.177995 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295510272}
I20260709 13:04:10.065739  8574 catalog_manager.cc:5697] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:10.148135 13647 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:10.193007  8752 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:10.197538  8755 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:10.177976 13765 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:10.100293 13849 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.211472  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:10.215106  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.214520  8756 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:10.180222 13941 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.183400  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:10.170603 13370 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:10.199326  8680 rpcz_store.cc:269] 0709 13:04:08.523587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.851984 (+328397us) service_pool.cc:224] Handling call
0709 13:04:10.176707 (+1324723us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54000da2d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:10.176905 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1027047680}
I20260709 13:04:10.223269  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.223503  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.224144  8680 raft_consensus.cc:2410] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:04:10.224959  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.225521  8680 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:10.190245  8753 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:10.203094  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.185448 13469 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.210361 13692 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.226405  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.227460  8680 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:10.222041 13566 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:10.225603  8752 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:10.220180  8719 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:10.253196  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:10.254387  8719 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:10.120474 13803 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.256246  8719 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:10.256903  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 237360us
I20260709 13:04:10.257750  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:10.258749  8719 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:10.193634 13833 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.195886 13834 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.264810  9381 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:10.265754  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:10.275559  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:10.276889  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:10.278040  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:10.273175  8719 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:10.287978  8825 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:10.288862  8719 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:10.293327  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:10.307344  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:10.311740  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.313782  8823 raft_consensus.cc:2376] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11. Re-sending same reply.
I20260709 13:04:10.311264  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.310745  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:10.321969  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.271912 13647 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:10.264027 13061 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:10.323169  8749 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:10.264959 13814 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:10.321142  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:10.327581  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:10.351751  8821 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:10.327889  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:10.330331  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.331964  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:10.293692 13922 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:10.287048 13803 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:10.264742 13891 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:10.324028  8681 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:10.335278  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.368316  9385 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:10.287598 13645 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:10.368892  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47859us
I20260709 13:04:10.272053 13889 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:10.369741 13889 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.369470 13645 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.370172 13889 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:10.287698 13566 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:10.291777 13846 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:10.372292 13566 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.373445 13846 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.373986 13846 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:10.227169 13370 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.295934 13824 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:10.375576 13824 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.376109 13824 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:10.376895 13370 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:10.358544  8756 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:10.379869  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.381589  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.294281 13921 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:10.313372 13837 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:10.383330 13837 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:10.383965 13837 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:10.295521 13691 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:10.362252  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.364135  8680 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:04:10.388401  8680 pending_rounds.cc:85] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 9
I20260709 13:04:10.388381 13645 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:10.323987 13842 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:10.389570 13842 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.390131 13842 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:10.291453 13827 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:10.368912 13954 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:10.359328  9389 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
W20260709 13:04:10.390892  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.391126 13954 raft_consensus.cc:697] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:04:10.391677  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.388841  8680 pending_rounds.cc:107] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:04:10.293447 13893 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:10.392441 13893 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.392850 13893 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:10.392216 13954 consensus_queue.cc:237] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.394826 13713 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:10.395447 13713 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.395956 13930 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:10.392236  8680 raft_consensus.cc:2889] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:10.397693 13930 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.398862  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 923) took 1004 ms. Trace:
I20260709 13:04:10.399257  8680 rpcz_store.cc:276] 0709 13:04:09.394543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.319279 (+924736us) service_pool.cc:224] Handling call
0709 13:04:10.359961 (+ 40682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.398517 (+ 38556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.398786 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:10.404908  8793 leader_election.cc:336] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.528s (SENT)
I20260709 13:04:10.323381 13925 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:10.413929  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.290088 13850 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 13:04:10.258194 13469 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.415462  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.419832  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.350135 13647 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:10.424635  8753 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:10.349171 13061 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:10.430883  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:10.431967  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 143216us
I20260709 13:04:10.433880  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.438128  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:10.438202  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.444985  9385 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:10.421370 13835 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:10.430678 13061 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:10.446319  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.483256  8752 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:10.447438  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:10.485327  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.438987  8821 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:10.452276  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.478826  9389 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:10.432463 13647 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:10.494802  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.497655  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.507833 13941 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:10.496345 13853 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:10.514981  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.515807  9388 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:10.490353 13702 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:10.516429  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:10.497206  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.519439  8748 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
W20260709 13:04:10.520183 13957 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:04:10.520515  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:10.522404 13950 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:04:10.527006 13061 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.528337  8755 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:10.529927  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.531365  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.531840  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.532569  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:10.533563  8566 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7aff04f53d4845a6a4fa491fd2a06145 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:10.535792  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.537987 13061 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:10.539593  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.539795 13469 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:10.540421  9384 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:10.542052  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:10.542752  8826 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:10.427711 13925 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.552959 13833 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:10.554979 13849 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:10.558857 13842 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.559762 13834 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:10.560076  8753 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:10.560648  9380 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:10.562736 13692 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:10.563948  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.567265  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.568403  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:10.510736 13814 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:10.571388  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:10.572077  8821 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:04:10.575311  8826 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:10.569796 13692 consensus_queue.cc:1059] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:10.561256 13853 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.572007 13727 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:04:10.422771  8574 catalog_manager.cc:5697] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 4 to 12, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 12 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:10.586030  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.570587 13814 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:10.589854  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.594060  8754 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:10.566438 13893 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.580940 13853 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.589135 13959 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:10.592234 13961 raft_consensus.cc:3055] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:10.595373 13876 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.615280 13876 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.593649 13370 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.616964 13370 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.620151 13959 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.626694 13710 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:10.630746 13710 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.634128 13710 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:10.635440 13889 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.636281 13930 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:10.639374 13925 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:10.639775 13889 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:10.640580 13889 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:10.641196  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2251) took 1156 ms. Trace:
I20260709 13:04:10.644591  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 917) took 2806 ms. Trace:
I20260709 13:04:10.650559  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2230) took 3003 ms. Trace:
I20260709 13:04:10.651326  8678 rpcz_store.cc:276] 0709 13:04:07.647160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.027296 (+380136us) service_pool.cc:224] Handling call
0709 13:04:08.027677 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:08.676151 (+648474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.650146 (+1973995us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b5400298954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:10.650453 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1464802176}
I20260709 13:04:10.653149  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.656703  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:10.653431  8751 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:10.660341 13441 raft_consensus.cc:2764] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:04:10.666216  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2637) took 2019 ms. Trace:
I20260709 13:04:10.666497  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.667523  8751 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:10.668995  8751 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 10. Current term is 12. Ops: [10.6-10.6]
I20260709 13:04:10.667114  8679 rpcz_store.cc:276] 0709 13:04:08.647091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.338571 (+691480us) service_pool.cc:224] Handling call
0709 13:04:10.176900 (+838329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.199435 (+ 22535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.665854 (+466419us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b54000da2d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:10.666109 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917635456}
I20260709 13:04:10.681097 13691 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:10.601826 13837 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.684736 13837 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:10.687537  8793 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:10.670387  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.641829  8674 rpcz_store.cc:276] 0709 13:04:09.484887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.536888 (+ 52001us) service_pool.cc:224] Handling call
0709 13:04:10.640923 (+1104035us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5400240ed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:10.641104 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324823936}
I20260709 13:04:10.709286  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.691838  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.693456  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.710203  8674 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:04:10.712764  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.714939  8674 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:10.645324  8676 rpcz_store.cc:276] 0709 13:04:07.837944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.347816 (+509872us) service_pool.cc:224] Handling call
0709 13:04:08.348308 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.546793 (+198485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:08.546833 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:08.547440 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 13:04:08.635644 (+ 88204us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:08.635700 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.643935 (+2008235us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b540033aed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:10.644021 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.644104 (+    83us) raft_consensus.cc:1685] finished
0709 13:04:10.644124 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.644490 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12341,"raft.queue_time_us":52916,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":21864,"spinlock_wait_cycles":455238016,"thread_start_us":87693,"threads_started":1,"wal-append.queue_time_us":159383,"wal-append.run_cpu_time_us":1967,"wal-append.run_wall_time_us":1022145}
I20260709 13:04:10.717353  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.719597  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:10.722709  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:10.724083  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:10.726946  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:10.728549  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:10.729928  8674 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:10.611720 13961 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:10.740825  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.748303 13853 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:10.742211 13727 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:10.682144 13691 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.742437 13837 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.734841 13710 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:04:10.641192 13889 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.723326 13930 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:04:10.755111  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.779945  8678 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:04:10.784066  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.741626 13926 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.833220  8674 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:10.756919 13710 consensus_queue.cc:1243] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:10.766534 13727 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:10.848421  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.859822  8751 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:10.841315  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:10.855960  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 265099us
I20260709 13:04:10.778512 13846 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.769963 13889 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:10.797405  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 52824.5us
I20260709 13:04:10.771584 13370 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:10.823247 13842 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:10.863750  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:10.863047  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:10.776145 13920 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:04:10.859043 13441 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:04:10.867748 13893 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:10.874696 13827 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:10.766884 13702 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.089s
W20260709 13:04:10.874696  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.899885  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.900588  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.883260 13824 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.507s	user 0.008s	sys 0.007s
I20260709 13:04:10.899283 13826 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:10.836166  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:10.882833  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.756179 13837 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.824436 13498 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:04:10.859859 13926 raft_consensus.cc:3055] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:10.890242 13853 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:10.907061  8674 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:10.907177  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:10.754580 13647 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:10.911188  8645 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:10.898798 13967 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.916793  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.917568  8751 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:10.918947  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.919050  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:10.923287 13967 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.926072 13967 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:10.923883 13866 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 2.810s	user 0.004s	sys 0.000s
I20260709 13:04:10.935664  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.913321 13853 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.906312 13926 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:04:10.948931  8750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2266) took 2552 ms (client timeout 2039 ms). Trace:
W20260709 13:04:10.973294  8750 rpcz_store.cc:269] 0709 13:04:08.396184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.476084 (+ 79900us) service_pool.cc:224] Handling call
0709 13:04:10.948557 (+2472473us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b5400138b54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:10.948785 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882932608}
I20260709 13:04:10.950906 13963 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:10.920544 13826 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:10.929536 13827 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:10.956432  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2704) took 2979 ms. Trace:
I20260709 13:04:10.958920  8751 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:10.969269  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:11.012698  8747 rpcz_store.cc:276] 0709 13:04:07.977004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.978769 (+  1765us) service_pool.cc:224] Handling call
0709 13:04:07.979270 (+   501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:09.431433 (+1452163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.956079 (+1524646us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b540031bd54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:10.956352 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3301871360}
I20260709 13:04:11.006916  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:11.013317  8753 raft_consensus.cc:2410] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:04:10.904948 13824 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:10.933220 13702 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.018189  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.971662 13853 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:11.049885  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:11.051407  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:11.022164  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:10.999222 13894 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.028618  8674 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
W20260709 13:04:11.025341  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.056236  8753 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 16: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:11.061138  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:11.060194  8750 raft_consensus.cc:2368] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18.
I20260709 13:04:10.883358 13961 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.014906 13827 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.067334  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:11.068538  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:11.069883  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:11.011955 13963 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.071063 13961 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:11.072202 13813 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.079427  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:11.081558  8747 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 10: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:11.074580  8674 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:04:11.082515  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:11.076808  8753 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:11.093125  8751 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:11.095947  8753 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:11.097116  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:11.099038  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2215) took 3631 ms. Trace:
I20260709 13:04:11.077903  8750 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:11.099472  8675 rpcz_store.cc:276] 0709 13:04:07.467243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.498676 (+ 31433us) service_pool.cc:224] Handling call
0709 13:04:07.515266 (+ 16590us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:07.589995 (+ 74729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.590046 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:07.595999 (+  5953us) log.cc:844] Serialized 52 byte log entry
0709 13:04:07.847693 (+251694us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:07.847764 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.847868 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.923043 (+3075175us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:11.092731 (+169688us) raft_consensus.cc:1685] finished
0709 13:04:11.092754 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.098944 (+  6190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74159,"raft.queue_time_us":75126,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"spinlock_wait_cycles":49224960,"thread_start_us":251133,"threads_started":1,"wal-append.queue_time_us":356686}
I20260709 13:04:11.100821  8750 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:11.050125 13959 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:11.075805 13827 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:11.101838  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:11.117396  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 186628us
W20260709 13:04:11.124588  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.064572 13846 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.088872 13837 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.053517 13702 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.098647 13657 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:11.128113  8747 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:11.128592  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.131577  8645 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:11.111263 13826 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:11.125602 13970 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:04:11.066319 13894 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:11.137737  8674 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:11.104147  8750 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.140743  8750 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:11.116743 13893 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:11.142233  8645 leader_election.cc:400] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18.
I20260709 13:04:11.123811 13834 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:11.144110  8645 leader_election.cc:403] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:11.147245  8645 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:11.148355  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:11.152436 13969 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:04:11.154305 13647 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:11.167286  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:11.076444 13926 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:11.146142 13657 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:11.180661  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:11.164064 13948 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18.
I20260709 13:04:11.172821 13837 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:11.161060 13893 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:11.190804 13926 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:11.209722  8822 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:11.214188  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 47054.4us
I20260709 13:04:11.195705  8747 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:04:11.182533 13886 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:04:11.199143 13834 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:11.231002  8750 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:04:11.237681 13826 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:11.241627  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:11.222026 13948 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:04:11.245465 13702 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.265228 13827 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.268455 13893 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.243798 13348 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:11.275828 13827 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:11.280102 13894 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.240322 13824 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.282835 13963 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:11.292326  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:11.307322 13853 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:11.326923  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 444165us
W20260709 13:04:11.327630  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 107192us
I20260709 13:04:11.335472 13702 consensus_queue.cc:1059] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:11.339727 13702 raft_consensus.cc:3055] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:11.340281 13702 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:11.341513 13702 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.342048 13853 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:11.342672 13702 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:11.343250 13973 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:11.339566 13849 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:11.347393 13973 raft_consensus.cc:3055] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:11.349501 13973 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:11.351225 13973 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.356691 13973 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:11.420339  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.430044  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.431609  8673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2640) took 2553 ms (client timeout 1938 ms). Trace:
I20260709 13:04:11.431655  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:11.433684  8681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2266) took 1275 ms (client timeout 1540 ms). Trace:
W20260709 13:04:11.436112  8754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2264) took 3040 ms (client timeout 1621 ms). Trace:
I20260709 13:04:11.436687  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:11.438030  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2708) took 1181 ms. Trace:
I20260709 13:04:11.430302  9389 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:04:11.438796  8752 rpcz_store.cc:276] 0709 13:04:10.256766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.257407 (+   641us) service_pool.cc:224] Handling call
0709 13:04:10.276501 (+ 19094us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.489487 (+212986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.437265 (+947778us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b540038b3d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:11.437727 (+   462us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474929536}
W20260709 13:04:11.434296  8681 rpcz_store.cc:269] 0709 13:04:10.158420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.202128 (+ 43708us) service_pool.cc:224] Handling call
0709 13:04:11.433123 (+1230995us) spinlock_profiling.cc:243] Waited 97.2 ms on lock 0x7b5400430dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:11.433435 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223559936}
W20260709 13:04:11.436726  8754 rpcz_store.cc:269] 0709 13:04:08.395236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.395397 (+   161us) service_pool.cc:224] Handling call
0709 13:04:11.435474 (+3040077us) spinlock_profiling.cc:243] Waited 2.2 s on lock 0x7b540031ae54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:11.435825 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5052959488}
W20260709 13:04:11.432557  8673 rpcz_store.cc:269] 0709 13:04:08.877570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.984912 (+107342us) service_pool.cc:224] Handling call
0709 13:04:11.415973 (+2431061us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400175554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:11.431223 (+ 15250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736416512}
W20260709 13:04:11.450737  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.456296  8749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 882) took 1256 ms (client timeout 1642 ms). Trace:
W20260709 13:04:11.456864  8749 rpcz_store.cc:269] 0709 13:04:10.199474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.199604 (+   130us) service_pool.cc:224] Handling call
0709 13:04:11.455706 (+1256102us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54002fe4d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:11.456017 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246361984}
I20260709 13:04:11.438117  8826 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:04:11.459729  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:11.460556  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2707) took 1204 ms. Trace:
I20260709 13:04:11.460945  8679 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:11.461179  8755 rpcz_store.cc:276] 0709 13:04:10.256452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.256887 (+   435us) service_pool.cc:224] Handling call
0709 13:04:10.257213 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.528835 (+271622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.459892 (+931057us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b54002ad354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:11.460319 (+   427us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623003520}
I20260709 13:04:11.462327  9355 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:11.452903  8673 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:11.465109  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 927) took 2069 ms. Trace:
I20260709 13:04:11.465620  8673 rpcz_store.cc:276] 0709 13:04:09.395872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.447553 (+2051681us) service_pool.cc:224] Handling call
0709 13:04:11.448463 (+   910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.464575 (+ 16112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.464982 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:11.460139  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:11.467279  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:11.469354  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:11.455435  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:11.472020  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2122) took 1029 ms. Trace:
I20260709 13:04:11.472554  8821 rpcz_store.cc:276] 0709 13:04:10.442918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.572771 (+129853us) service_pool.cc:224] Handling call
0709 13:04:11.470375 (+897604us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:04:11.470875 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.471421 (+   546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.471452 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:11.471516 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:11.471556 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.471616 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.471770 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3985280}
I20260709 13:04:11.472079  8681 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:11.499773  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:11.510548  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:11.513108  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:11.521063 13846 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.522135 13846 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:11.523332  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:11.524698  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:11.526356 13846 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.527904 13853 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:04:11.527034 13973 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:11.536936 13973 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.536957 13853 consensus_queue.cc:1243] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:11.537781 13973 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:11.551986  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 77187.4us
I20260709 13:04:11.553543  8647 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:11.554893 13853 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:11.559780  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:11.563621  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2271) took 1038 ms. Trace:
I20260709 13:04:11.563843  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:11.564265  8678 rpcz_store.cc:276] 0709 13:04:10.525110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.726670 (+201560us) service_pool.cc:224] Handling call
0709 13:04:11.563176 (+836506us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b5400174b54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:11.563383 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119230336}
I20260709 13:04:11.571697  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2235) took 2175 ms. Trace:
I20260709 13:04:11.572662  8822 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:04:11.552357  9380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 772) took 1068 ms. Trace:
I20260709 13:04:11.572268  8673 rpcz_store.cc:276] 0709 13:04:09.396151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.494740 (+2098589us) service_pool.cc:224] Handling call
0709 13:04:11.570880 (+ 76140us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.571247 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.571273 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:11.571351 (+    78us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:11.571395 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.571461 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.571609 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:11.579391  8678 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:04:11.570210  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:11.558354  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 924) took 2163 ms. Trace:
I20260709 13:04:11.608916  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:11.612376 13338 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:11.613040 13338 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:11.613508 13338 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:11.528335 13827 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.614697 13729 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:11.615244 13827 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:11.590885  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 49011.7us
I20260709 13:04:11.606925 13726 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:11.623466 13726 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:11.628356 13876 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.528923  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:11.577306  9380 rpcz_store.cc:276] 0709 13:04:10.483339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.483526 (+   187us) service_pool.cc:224] Handling call
0709 13:04:10.560472 (+ 76946us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.561049 (+   577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.551845 (+990796us) spinlock_profiling.cc:243] Waited 76.5 ms on lock 0x7b54006275d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:11.552121 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":175917312}
I20260709 13:04:11.587387  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2239) took 2190 ms. Trace:
I20260709 13:04:11.636726  8678 rpcz_store.cc:276] 0709 13:04:09.397249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.565512 (+2168263us) service_pool.cc:224] Handling call
0709 13:04:11.566148 (+   636us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.581894 (+ 15746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.587243 (+  5349us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:11.650746  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 88775.4us
W20260709 13:04:11.651844  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.656257  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.656497 13973 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.659601 13927 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:11.660750 13927 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:11.662072  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.662910  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.663823  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 925) took 2268 ms. Trace:
I20260709 13:04:11.664999 13927 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:11.666327  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.671800  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.672667  8829 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:11.664357  8675 rpcz_store.cc:276] 0709 13:04:09.395321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.140850 (+1745529us) service_pool.cc:224] Handling call
0709 13:04:11.144973 (+  4123us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.145384 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:11.415277 (+269893us) log.cc:844] Serialized 14 byte log entry
0709 13:04:11.639912 (+224635us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.660066 (+ 20154us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.660393 (+   327us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:11.660445 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.660579 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:11.663384 (+  2805us) raft_consensus.cc:1685] finished
0709 13:04:11.663408 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.663594 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":796,"prepare.queue_time_us":348493,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":57959,"raft.run_cpu_time_us":412,"raft.run_wall_time_us":18775,"thread_start_us":102039,"threads_started":1,"wal-append.queue_time_us":235394}
W20260709 13:04:11.673314  8829 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2208) took 1366 ms (client timeout 1621 ms). Trace:
W20260709 13:04:11.673784  8829 rpcz_store.cc:269] 0709 13:04:10.306550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.309783 (+  3233us) service_pool.cc:224] Handling call
0709 13:04:11.528559 (+1218776us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:04:11.673225 (+144666us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14845440}
I20260709 13:04:11.677429  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:11.680151  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:11.670940  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:11.671290 13973 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.681198  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2112) took 1898 ms. Trace:
W20260709 13:04:11.669934  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.657048  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.698872  8675 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:11.700372  8722 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:11.701747 13876 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:11.702049  8722 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:11.702371 13876 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.700543  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:11.703838  8676 raft_consensus.cc:2410] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:04:11.708546 13876 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:11.702589 13937 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 1.799s	user 0.016s	sys 0.004s
I20260709 13:04:11.711899 13832 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.713737 13832 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:11.714444  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 985) took 2016 ms. Trace:
I20260709 13:04:11.715953  8675 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:11.714941  8827 rpcz_store.cc:276] 0709 13:04:09.697921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.698636 (+   715us) service_pool.cc:224] Handling call
0709 13:04:09.699002 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:09.699384 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.699417 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:09.700427 (+  1010us) log.cc:844] Serialized 26 byte log entry
0709 13:04:09.868158 (+167731us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:09.868353 (+   195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.868459 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:11.701442 (+1832983us) spinlock_profiling.cc:243] Waited 9.81 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:11.713856 (+ 12414us) raft_consensus.cc:1685] finished
0709 13:04:11.713882 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.714214 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1680,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":22559104,"thread_start_us":167268,"threads_started":1,"wal-append.queue_time_us":202424}
I20260709 13:04:11.717468  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:11.721347 13977 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.722242 13977 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.723367  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:11.712031  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:11.725952  8679 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:11.726327  8830 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:11.726634  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2276) took 1087 ms. Trace:
I20260709 13:04:11.727047  8679 rpcz_store.cc:276] 0709 13:04:10.639056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.458995 (+819939us) service_pool.cc:224] Handling call
0709 13:04:11.726506 (+267511us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:11.728454  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:11.729324  8679 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:04:11.730638 13887 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:11.738005  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:11.689771  8825 rpcz_store.cc:276] 0709 13:04:09.782507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.869650 (+ 87143us) service_pool.cc:224] Handling call
0709 13:04:11.680703 (+1811053us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b5400218ed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:11.680976 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":954330880}
I20260709 13:04:11.740269  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:11.744720  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:11.691591 13827 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:11.588822  8680 rpcz_store.cc:276] 0709 13:04:09.394813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.400101 (+1005288us) service_pool.cc:224] Handling call
0709 13:04:10.400582 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:10.683558 (+282976us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:10.683612 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:10.684299 (+   687us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.555835 (+871536us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b1800284aa0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e20a14bc 00007f32df3ef48b 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1
0709 13:04:11.556219 (+   384us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:11.556314 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.557661 (+  1347us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b5400299354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:11.557729 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:11.557809 (+    80us) raft_consensus.cc:1685] finished
0709 13:04:11.557830 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.558096 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4083,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":25650,"spinlock_wait_cycles":664773760,"thread_start_us":49347,"threads_started":1,"wal-append.queue_time_us":170521}
I20260709 13:04:11.744776  8678 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:11.718479  8675 raft_consensus.cc:2376] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16. Re-sending same reply.
I20260709 13:04:11.751942  8681 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:11.749440  8828 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
W20260709 13:04:11.754228  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 44684.8us
W20260709 13:04:11.757889  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.756256  9389 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:04:11.763919  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:11.776834  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.791378  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.799772 13851 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.801059  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:11.801944  8755 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:04:11.791154 13902 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:11.809006  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:11.814224  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:11.817534  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:11.829815  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:11.826153 13726 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:11.828325  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:11.835637  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2240) took 2431 ms. Trace:
W20260709 13:04:11.845950  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 292654us
I20260709 13:04:11.820542 13338 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.844827  8673 rpcz_store.cc:276] 0709 13:04:09.418147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.583100 (+2164953us) service_pool.cc:224] Handling call
0709 13:04:11.671065 (+ 87965us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.671535 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:11.672055 (+   520us) log.cc:844] Serialized 14 byte log entry
0709 13:04:11.828599 (+156544us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:11.828651 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:11.828692 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.828750 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.828873 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":247,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":71,"raft.queue_time_us":17608,"thread_start_us":141265,"threads_started":1}
I20260709 13:04:11.855376  8680 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 8.
I20260709 13:04:11.838307 13977 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.834743 13726 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:11.869493  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:11.871331  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:11.879400  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:11.879978  8830 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:11.888342  8675 raft_consensus.cc:2410] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:04:11.873382 13850 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.893980  9355 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:11.896890  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 195053us
I20260709 13:04:11.898351  8722 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:11.901057  8722 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:11.897029 13732 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:11.883971 13729 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:11.905208  8678 raft_consensus.cc:2368] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13.
I20260709 13:04:11.905398 13729 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:11.905916  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:11.906478  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:11.907317  8678 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:11.907810 13729 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.904026 13732 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.887131  8756 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:11.910405 13729 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:11.911140 13827 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:11.919307  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:11.922640  8680 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:04:11.923223  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.917026 13973 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:11.926255 13803 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.926826  8753 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:04:11.927080 13803 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.930111  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:11.934104  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:11.935417  8824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2204) took 2154 ms (client timeout 2332 ms). Trace:
W20260709 13:04:11.935902  8824 rpcz_store.cc:269] 0709 13:04:09.781297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.862915 (+ 81618us) service_pool.cc:224] Handling call
0709 13:04:11.932094 (+2069179us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:04:11.935320 (+  3226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3814272}
I20260709 13:04:11.932655 13973 raft_consensus.cc:697] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:11.916857 13845 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:11.895143 13850 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:11.942540  8826 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:04:11.933456  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2247) took 2451 ms. Trace:
W20260709 13:04:11.952168  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.954023  8680 rpcz_store.cc:276] 0709 13:04:09.474239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.875193 (+2400954us) service_pool.cc:224] Handling call
0709 13:04:11.913955 (+ 38762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.923149 (+  9194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.925667 (+  2518us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:11.955998  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:11.956290 13850 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.949594  8824 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:11.885706 13897 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.162s
I20260709 13:04:11.967440  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:11.968211  8677 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:11.969177  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:11.969866  8751 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:11.971001 13900 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.971562 13900 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.976375 13876 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:11.977038 13713 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:11.979000  8754 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:11.981287 13732 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:11.950842 13338 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:11.993846  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2243) took 2574 ms. Trace:
I20260709 13:04:11.997498  8673 rpcz_store.cc:276] 0709 13:04:09.419059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.866159 (+2447100us) service_pool.cc:224] Handling call
0709 13:04:11.908718 (+ 42559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.982537 (+ 73819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.982569 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:11.993412 (+ 10843us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:11.993482 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.993575 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.993756 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:12.010076  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 104324us
I20260709 13:04:12.011026  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:12.020071  8755 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:12.025765  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:12.026862  8828 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:04:12.027830  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:12.028560  9389 raft_consensus.cc:2376] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11. Re-sending same reply.
I20260709 13:04:12.029657  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:12.045946  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2270) took 1563 ms. Trace:
I20260709 13:04:12.044018 13803 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:12.049964  8756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2715) took 1630 ms (client timeout 1939 ms). Trace:
I20260709 13:04:11.949604 13973 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:12.044021 13988 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:04:12.054576  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 926) took 2658 ms. Trace:
I20260709 13:04:12.055116  8674 rpcz_store.cc:276] 0709 13:04:09.395599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.149519 (+1753920us) service_pool.cc:224] Handling call
0709 13:04:11.566830 (+417311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.567479 (+   649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.567519 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:11.567582 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:11.567613 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.567696 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.054173 (+486477us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b5400174b50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:12.054349 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":958673664}
I20260709 13:04:12.056561  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:12.057714  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.050537  8756 rpcz_store.cc:269] 0709 13:04:10.419839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.419981 (+   142us) service_pool.cc:224] Handling call
0709 13:04:12.049689 (+1629708us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b5400139554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:12.049872 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3271725440}
I20260709 13:04:12.057456  8674 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:12.059343  9389 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:12.059499  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:12.060463  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:12.060966  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:12.062327  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2721) took 1563 ms. Trace:
I20260709 13:04:12.063123  8748 rpcz_store.cc:276] 0709 13:04:10.498814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.521393 (+ 22579us) service_pool.cc:224] Handling call
0709 13:04:11.492858 (+971465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.493637 (+   779us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:11.592108 (+ 98471us) log.cc:844] Serialized 14 byte log entry
0709 13:04:11.868615 (+276507us) log.cc:844] Serialized 14 byte log entry
0709 13:04:11.876984 (+  8369us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:11.877046 (+    62us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:11.877090 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.881439 (+  4349us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.047520 (+166081us) spinlock_profiling.cc:243] Waited 4.73 ms on lock 0x7b180033c640. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1
0709 13:04:12.048119 (+   599us) spinlock_profiling.cc:243] Waited 4.52 ms on lock 0x7b180033c640. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:12.061957 (+ 13838us) spinlock_profiling.cc:243] Waited 955 ms on lock 0x7b54002fe4d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:12.062114 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17622,"prepare.queue_time_us":380331,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":86,"raft.queue_time_us":25731,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":13361,"spinlock_wait_cycles":2221905280,"thread_start_us":231783,"threads_started":2,"wal-append.queue_time_us":321484}
I20260709 13:04:12.065646  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2291) took 1384 ms. Trace:
I20260709 13:04:12.062314  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:12.066251  8753 rpcz_store.cc:276] 0709 13:04:10.681599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.132787 (+451188us) service_pool.cc:224] Handling call
0709 13:04:11.133443 (+   656us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.927356 (+793913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.065229 (+137873us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b540034ab54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:12.065534 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1614854912}
I20260709 13:04:12.067821  8747 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:04:12.067968  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.068303  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:12.070312  8681 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:12.071087  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2242) took 2652 ms. Trace:
I20260709 13:04:12.071417  8681 rpcz_store.cc:276] 0709 13:04:09.418816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.775405 (+2356589us) service_pool.cc:224] Handling call
0709 13:04:11.856781 (+ 81376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.070146 (+213365us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54002939a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2470dc0 00007f32e24fb1c0 00007f32e24fc231 00007f32e24f8d8f 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c
0709 13:04:12.070818 (+   672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.071006 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253069568}
I20260709 13:04:12.072311  8750 pending_rounds.cc:85] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 9
I20260709 13:04:12.072639  8750 pending_rounds.cc:107] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:04:12.072670  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:12.073021  8750 raft_consensus.cc:2889] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:12.073670  8681 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:12.075662  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:12.076977  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:12.081811  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:12.059569  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:12.082135  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:12.083101  8756 raft_consensus.cc:2410] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:04:12.083158  8821 raft_consensus.cc:2410] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:04:12.083521 13726 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:12.085070  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:12.085814  8823 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:12.086272  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:12.087245  8822 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:12.088685 13713 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:12.089735 13900 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:12.091405  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:12.092252  8672 raft_consensus.cc:2368] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15.
I20260709 13:04:12.093508  8672 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:12.066175  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:12.094163  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2648) took 2601 ms. Trace:
W20260709 13:04:12.095202  8677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2241) took 2676 ms (client timeout 1634 ms). Trace:
I20260709 13:04:12.095672  8680 raft_consensus.cc:2376] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18. Re-sending same reply.
I20260709 13:04:12.090185  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:12.051682  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.097429  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:12.062798  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:12.087321  8753 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 15. Ops: []
I20260709 13:04:12.094555  8672 rpcz_store.cc:276] 0709 13:04:09.493216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.092875 (+2599659us) service_pool.cc:224] Handling call
0709 13:04:12.093298 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.093895 (+   597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.094090 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:12.110302  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:12.095701  8677 rpcz_store.cc:269] 0709 13:04:09.418494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.419619 (+  1125us) service_pool.cc:224] Handling call
0709 13:04:12.094983 (+2675364us) spinlock_profiling.cc:243] Waited 2.55 s on lock 0x7b54001f9d54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:12.095116 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5857895168}
I20260709 13:04:12.108916  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:12.114490  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2252) took 2627 ms. Trace:
I20260709 13:04:12.046511  8676 rpcz_store.cc:276] 0709 13:04:10.482644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.716939 (+234295us) service_pool.cc:224] Handling call
0709 13:04:12.045667 (+1328728us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b54002a8ad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:12.045878 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2263080320}
I20260709 13:04:12.123623  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:12.114099  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:12.129060  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:12.128880  8674 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:12.130223  8681 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:12.150959  8821 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:12.152206  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:12.090179 13713 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:12.096344 13691 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:12.162111  8679 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:12.151626  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:12.174140  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.177148  8677 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:12.183295  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2649) took 2688 ms. Trace:
I20260709 13:04:12.183940  8677 rpcz_store.cc:276] 0709 13:04:09.495220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.128388 (+2633168us) service_pool.cc:224] Handling call
0709 13:04:12.152754 (+ 24366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.179797 (+ 27043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.183199 (+  3402us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:12.153209 13827 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:12.169036  8647 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:12.189697  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 91489.9us
W20260709 13:04:12.181099  8719 leader_election.cc:336] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.634s (SENT)
I20260709 13:04:12.193998  8676 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:12.197206  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2259) took 2464 ms. Trace:
I20260709 13:04:12.197798  8676 rpcz_store.cc:276] 0709 13:04:09.732795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.186008 (+2453213us) service_pool.cc:224] Handling call
0709 13:04:12.193730 (+  7722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.196856 (+  3126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.197133 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:12.202250  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 46852.7us
I20260709 13:04:12.208290  8673 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:12.208571 13927 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:12.204485  8824 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:12.177193 13713 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:12.214148  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2265) took 2057 ms. Trace:
I20260709 13:04:12.214475 13848 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:04:12.214707  8673 rpcz_store.cc:276] 0709 13:04:10.156635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.209459 (+2052824us) service_pool.cc:224] Handling call
0709 13:04:12.209979 (+   520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.213644 (+  3665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:12.213692 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:12.213779 (+    87us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:12.213815 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.213893 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.214045 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:12.215588  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:12.231005  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.223814  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2260) took 2302 ms. Trace:
I20260709 13:04:12.235095  8679 rpcz_store.cc:276] 0709 13:04:09.921397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.178816 (+2257419us) service_pool.cc:224] Handling call
0709 13:04:12.203810 (+ 24994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.222999 (+ 19189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:12.223143 (+   144us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:12.223288 (+   145us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:12.223379 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.223505 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.223716 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:12.239964 13801 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:12.216899 13897 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:12.188771 13726 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:04:12.239533  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55128.8us
I20260709 13:04:12.278815  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2267) took 1845 ms. Trace:
I20260709 13:04:12.279527  8673 rpcz_store.cc:276] 0709 13:04:10.433366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.215922 (+1782556us) service_pool.cc:224] Handling call
0709 13:04:12.216392 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.273055 (+ 56663us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:12.273109 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:12.278327 (+  5218us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:12.278395 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.278500 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.278723 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:12.210456 13876 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:12.309368  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 157740us
I20260709 13:04:12.320468  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2262) took 2229 ms. Trace:
I20260709 13:04:12.324138  8677 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:12.114993  8678 rpcz_store.cc:276] 0709 13:04:09.487396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.907965 (+2420569us) service_pool.cc:224] Handling call
0709 13:04:11.908419 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.971465 (+ 63046us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:11.976073 (+  4608us) log.cc:844] Serialized 14 byte log entry
0709 13:04:12.114119 (+138046us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:12.114186 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:12.114235 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.114304 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.114429 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1199,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":9027,"thread_start_us":122842,"threads_started":1}
I20260709 13:04:12.323365  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:12.337821  8673 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:12.243628 13827 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:12.339789  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 160958us
W20260709 13:04:12.220170 13993 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:04:12.324981  8675 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:04:12.349665  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.358942  8676 rpcz_store.cc:276] 0709 13:04:10.090673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.218068 (+2127395us) service_pool.cc:224] Handling call
0709 13:04:12.239001 (+ 20933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.281097 (+ 42096us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:12.281151 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:12.281226 (+    75us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:12.320060 (+ 38834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.320168 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.320378 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4336,"raft.queue_time_us":35678,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":24205}
I20260709 13:04:12.217753 13900 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:12.338488 13713 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:12.281339 13801 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:12.362015  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2261) took 2404 ms. Trace:
I20260709 13:04:12.385500  8677 rpcz_store.cc:276] 0709 13:04:09.958142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.185030 (+2226888us) service_pool.cc:224] Handling call
0709 13:04:12.272754 (+ 87724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.359034 (+ 86280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.361896 (+  2862us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:12.375919  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2257) took 2706 ms. Trace:
I20260709 13:04:12.386968  8675 rpcz_store.cc:276] 0709 13:04:09.669581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.172299 (+2502718us) service_pool.cc:224] Handling call
0709 13:04:12.224842 (+ 52543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.375485 (+150643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.375825 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:12.348573 13838 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:12.379794  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 198828us
W20260709 13:04:12.380846  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 261483us
W20260709 13:04:12.382790  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.384833  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:12.394985  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2255) took 2832 ms. Trace:
I20260709 13:04:12.407007  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:12.404237  8680 rpcz_store.cc:276] 0709 13:04:09.562553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.181695 (+2619142us) service_pool.cc:224] Handling call
0709 13:04:12.282379 (+100684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.394380 (+112001us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:12.394420 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:12.394498 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:12.394559 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.394698 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.394902 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37462}
I20260709 13:04:12.396144  8679 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:04:12.434048  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.340029 13980 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.264s
I20260709 13:04:12.454960  8756 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:12.457182  8676 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:12.457201  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:12.384387 13839 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.471654  8673 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:12.474701  8678 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:12.406354 13900 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.477521 13963 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:12.478823 13927 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:12.479453 14003 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:04:12.471014 13839 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:12.393687 13801 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:12.481530  8677 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:12.481986 13698 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.482776 13698 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:12.485201 13801 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:12.485445 13698 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:12.474068 13992 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.499s	user 0.001s	sys 0.001s
I20260709 13:04:12.468214  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:12.458280 13713 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:12.478088 13963 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:04:12.518715  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 199772us
I20260709 13:04:12.505215  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 930) took 2930 ms. Trace:
W20260709 13:04:12.506470  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.495105 13726 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:12.433064 13838 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:12.522738 13845 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:12.523252  8674 rpcz_store.cc:276] 0709 13:04:09.574895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.186570 (+2611675us) service_pool.cc:224] Handling call
0709 13:04:12.213838 (+ 27268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.214204 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:12.214604 (+   400us) log.cc:844] Serialized 14 byte log entry
0709 13:04:12.504723 (+290119us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:12.504801 (+    78us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:12.504870 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.504970 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.505135 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":744,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":4176,"thread_start_us":289557,"threads_started":1}
W20260709 13:04:12.556269  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 151702us
I20260709 13:04:12.500221 13827 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:12.562294  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 105220us
I20260709 13:04:12.542757 13801 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.543771 13726 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:12.562247 13991 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.353s	user 0.000s	sys 0.002s
I20260709 13:04:12.564569  8794 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:12.535882 13900 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:12.570596  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:12.570681  9352 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:12.574627  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.578276  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.579078  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.566478 13726 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:12.587522 13801 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:12.588033  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:12.591658  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:12.595901  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 46914.8us
I20260709 13:04:12.597548 13845 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:12.598109 13845 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:12.598497 13845 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:12.602792 13851 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:12.607193 13338 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:12.607841 13338 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:12.608261 13338 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:12.611150  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 61384.3us
I20260709 13:04:12.613574  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:12.613618  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 891) took 1057 ms. Trace:
I20260709 13:04:12.575039 14009 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:12.621572  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 45688.2us
I20260709 13:04:12.623009  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:12.617359  8749 rpcz_store.cc:276] 0709 13:04:11.556180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.575564 (+ 19384us) service_pool.cc:224] Handling call
0709 13:04:11.650873 (+ 75309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.651649 (+   776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.651699 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.652477 (+   778us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.876722 (+224245us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:11.876772 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.876849 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.613363 (+736514us) raft_consensus.cc:1685] finished
0709 13:04:12.613393 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.613548 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1451,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":256,"thread_start_us":223704,"threads_started":1,"wal-append.queue_time_us":359340}
I20260709 13:04:12.626369  9380 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:12.626832  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.628074 13691 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:12.630894 13585 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:12.631139  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 44834.4us
W20260709 13:04:12.631520  8647 leader_election.cc:336] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.621s (SENT)
I20260709 13:04:12.633355  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:12.632459  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:12.634729  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2650) took 3085 ms. Trace:
I20260709 13:04:12.635260  8681 rpcz_store.cc:276] 0709 13:04:09.549201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.151395 (+2602194us) service_pool.cc:224] Handling call
0709 13:04:12.194580 (+ 43185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:12.210493 (+ 15913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:12.210537 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:12.275985 (+ 65448us) log.cc:844] Serialized 26 byte log entry
0709 13:04:12.527337 (+251352us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:12.557321 (+ 29984us) log.cc:844] Serialized 14 byte log entry
0709 13:04:12.557609 (+   288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.557698 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.634425 (+ 76727us) raft_consensus.cc:1685] finished
0709 13:04:12.634448 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.634653 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9057,"prepare.queue_time_us":29713,"prepare.run_cpu_time_us":153,"prepare.run_wall_time_us":4174,"raft.queue_time_us":12620,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":11665,"thread_start_us":166524,"threads_started":1,"wal-append.queue_time_us":306114}
I20260709 13:04:12.631677 13585 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:12.633535  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 45619.4us
W20260709 13:04:12.638051  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.567894 13963 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:12.644016  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.644197 13827 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:12.645193 14005 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:04:12.646245 13801 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:12.647415 14005 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:12.648698  8719 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:12.649904  8719 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:12.649946 13888 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:12.650873 13888 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:12.651746  8719 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:12.617753 13927 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:12.654997  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:12.636806 13472 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:12.668488 13889 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:12.669895 13370 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:12.671977 13900 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:12.641932 13585 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:12.682272  8821 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
W20260709 13:04:12.699671  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68214.9us
W20260709 13:04:12.705729  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 72911.7us
I20260709 13:04:12.707263 13691 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:04:12.728039  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 87559.1us
I20260709 13:04:12.727916 13691 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:12.731588 13971 consensus_queue.cc:1059] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:12.732918 13849 consensus_queue.cc:1059] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:12.734239 13847 raft_consensus.cc:3055] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:12.734699 13847 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:12.735754 13847 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:12.736907  8719 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:12.737221 13847 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:12.738126  8719 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:12.708474 13927 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:12.708523  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.667855 13801 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:12.716255 13845 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:12.716826 14015 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:12.652302 13888 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:12.747644  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.725751  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 112258us
I20260709 13:04:12.759244  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:12.768893  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:12.765333  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:12.773676  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:12.775527  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:12.776386  8828 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:12.779453  8680 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:12.780400 14005 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.780980 13726 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:12.784386 13845 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:12.785219 13845 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:12.785847 13845 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:12.781045 14005 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:12.793083  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:12.794147  8674 raft_consensus.cc:2410] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:04:12.797048  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:12.788511  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:12.800913  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 83134.1us
I20260709 13:04:12.801307  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:12.804613  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 45503.5us
I20260709 13:04:12.787711 13839 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:12.820256 13887 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:12.820956 13887 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:12.822271  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:12.823280  8681 raft_consensus.cc:2410] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:04:12.822899 13845 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:12.824456  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:12.825294  8681 raft_consensus.cc:2393] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 10: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:12.827225 14017 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:04:12.803661  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:12.808324  8823 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:12.808991  8647 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:12.830047  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:12.799554 13927 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:12.812026  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:12.812726 14015 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:12.818117  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:12.826247  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:12.832494  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 119759us
I20260709 13:04:12.836840  8828 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:04:12.832861  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:12.774886 13900 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:12.833842  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:12.802475 13876 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:12.838586  8647 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.660s (SENT) [suppressed 43 similar messages]
I20260709 13:04:12.843039  8821 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:04:12.844357  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:12.845206  8823 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:12.847479  8822 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:12.828733 14009 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:12.822014 13887 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:12.832072 14018 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.833289 13851 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.856410  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:12.858179  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:12.854496 14009 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:12.859458  8824 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:12.840337 13839 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:12.867009 13839 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:12.866923 14009 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:12.868785 14009 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:12.869997 13839 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:12.870832 14018 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:12.844084 13876 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:12.852833 14017 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:12.854545 13889 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:12.769140 13585 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:12.884863 13888 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:12.885228 13370 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:12.891407  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:12.906064  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.880512 13876 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:12.807425 14014 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:04:12.839713 14019 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:04:12.899808  8829 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:12.887609 14017 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:12.916720  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.926851  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:12.927830  8681 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:12.911756  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:12.888671 14018 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:12.949764  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 137837us
I20260709 13:04:12.963696 13927 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:12.963440 13338 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:12.964748  9380 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:12.965214 14005 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:12.858325 14015 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:12.967233  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:12.967936  8679 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:12.969007 14015 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:12.878897 13851 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:12.976105  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:12.977068  8674 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:12.974546  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:12.931967 14017 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:12.981599  8794 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:12.971719 13338 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:13.003078  8681 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:13.003391  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:13.004413  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:13.005179  8674 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 13. Current term is 14. Ops: [9.6-13.8]
I20260709 13:04:13.006191  8681 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:04:13.007251 13889 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.013531  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:13.013324 13370 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:12.966536 13999 log.cc:927] Time spent T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Append to log took a long time: real 0.179s	user 0.001s	sys 0.002s
I20260709 13:04:12.914402 13900 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:13.050439  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.032446  8824 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:13.035710  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:13.052863  9389 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
W20260709 13:04:13.029974  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 219578us
I20260709 13:04:13.045173 13585 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:13.046842 14016 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:13.046958  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 96058.6us
W20260709 13:04:13.059543  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.049548 13726 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:13.109999  8678 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:13.088913 14019 consensus_queue.cc:1243] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:13.117107  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:13.117955  8678 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:13.119678 14014 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:13.140889  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.150698 14025 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:13.152431 14030 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:04:13.158146  8680 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:13.160693  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.169781 13851 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:13.143112 14014 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.171124 14014 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:13.170853 13702 consensus_queue.cc:1059] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:13.171974 13849 raft_consensus.cc:3055] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:13.172231 13702 raft_consensus.cc:3055] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:13.172530 13849 raft_consensus.cc:740] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:13.172778 13702 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:13.175449 14029 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:04:13.175789 13900 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:13.178156 14028 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:04:13.183666 13963 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:13.152480 14000 log.cc:927] Time spent T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.671s	user 0.001s	sys 0.011s
I20260709 13:04:13.173852 13702 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:13.197538  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 108318us
I20260709 13:04:13.146296  8679 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:13.192574  8796 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:13.203847  8678 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:13.190016 13338 consensus_queue.cc:1059] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:04:13.210413 14037 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:13.213491 14037 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.213914 14037 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:13.221920  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.201756 13900 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.207283 13838 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:13.258998  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:13.259047  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 245582us
I20260709 13:04:13.253198 14016 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.257845 13851 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:13.265085 14009 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:13.278333  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.285701  8793 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:13.287309  8673 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:13.288239 14019 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:13.288846 14019 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.290927 14034 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.291639 14034 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:13.295003  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 909) took 1188 ms. Trace:
I20260709 13:04:13.279448  8672 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:04:13.280412  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:13.282027 14038 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:04:13.303854 13851 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:13.304534  8823 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:13.264016 13702 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:13.256595 13900 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:13.294020 14034 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:13.331249  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 245684us
I20260709 13:04:13.323637  9355 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:13.338685  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 86699.6us
I20260709 13:04:13.334592  8719 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:13.342422  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:13.331645  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:13.296696  8753 rpcz_store.cc:276] 0709 13:04:12.107066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.118540 (+ 11474us) service_pool.cc:224] Handling call
0709 13:04:12.123431 (+  4891us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.123714 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:12.154877 (+ 31163us) log.cc:844] Serialized 14 byte log entry
0709 13:04:12.493083 (+338206us) spinlock_profiling.cc:243] Waited 5.47 ms on lock 0x7b18003fc2e0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:13.048789 (+555706us) spinlock_profiling.cc:243] Waited 5.42 ms on lock 0x7b18003fc2e0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e5ef13c7 00007f32e25039a7 00007f32e25130bf 00007f32e2513038
0709 13:04:13.049222 (+   433us) log.cc:844] Serialized 14 byte log entry
0709 13:04:13.294410 (+245188us) spinlock_profiling.cc:243] Waited 9.12 ms on lock 0x7b18003fc2e0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:04:13.294580 (+   170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:13.294661 (+    81us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:13.294725 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.294799 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.294927 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":606895,"prepare.queue_time_us":1018028,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":102,"raft.queue_time_us":14877,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":9366,"spinlock_wait_cycles":46021248}
I20260709 13:04:13.294788 13713 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.348609  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:13.355284  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:13.306569 13851 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:13.320899 13897 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.359493  9381 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:13.361227  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:13.375813  8824 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:13.362329  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:13.365196  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 96089.8us
I20260709 13:04:13.371685  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 910) took 1263 ms. Trace:
I20260709 13:04:13.275492 14016 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.389149  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:13.396727  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:13.364607 13847 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:13.338294 14009 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:04:13.407202  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:13.407732  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.408558  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:13.377383  8751 rpcz_store.cc:276] 0709 13:04:12.107861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.155482 (+ 47621us) service_pool.cc:224] Handling call
0709 13:04:12.186963 (+ 31481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.190557 (+  3594us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:13.079966 (+889409us) log.cc:844] Serialized 14 byte log entry
0709 13:04:13.288220 (+208254us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b180012ae40. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e20a14bc 00007f32df3ef48b 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a
0709 13:04:13.371107 (+ 82887us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:13.371194 (+    87us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:13.371265 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.371392 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.371593 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":961812,"prepare.queue_time_us":880694,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"spinlock_wait_cycles":11915520,"thread_start_us":129884,"threads_started":2,"wal-append.queue_time_us":107040}
W20260709 13:04:13.417301  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 66028us
W20260709 13:04:13.417160  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 131568us
I20260709 13:04:13.411700  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:13.485673  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:13.364908 13713 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.409478 14027 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:13.493759  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:13.491971  8825 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:13.515563  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:13.494191 14018 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:13.522257 14018 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.525059 14018 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:13.526597 14034 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:13.527161 14034 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:13.528111 14034 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:13.529488 14034 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:13.531704  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 113310us
I20260709 13:04:13.532711  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:13.534214  8793 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:13.535003  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:13.536942  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.383913 13897 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.550052  8825 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:13.492090 13726 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:13.422690 14019 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:13.556362  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 142388us
W20260709 13:04:13.554128  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.557298  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:13.495416 14027 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:04:13.558919  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 161827us
I20260709 13:04:13.517138 13851 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.421599 13370 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.127s
I20260709 13:04:13.563499  8824 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:13.558862 13726 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:13.572671  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 179701us
I20260709 13:04:13.580005  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:13.582022 13726 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:13.583374  8749 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:13.585130  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:13.586772  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:13.588418 14040 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
W20260709 13:04:13.590557 14050 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:04:13.591225  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 57115.4us
I20260709 13:04:13.592182  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:13.592468  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2315) took 1225 ms. Trace:
I20260709 13:04:13.560834 14027 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:13.574821 13838 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:13.596491 14049 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:04:13.598253  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:13.599484  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:13.601082  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:13.573920 14057 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:13.601790  8828 raft_consensus.cc:2376] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12. Re-sending same reply.
W20260709 13:04:13.602380 14017 consensus_meta.cc:287] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.552s	user 0.014s	sys 0.000s
I20260709 13:04:13.602082 14057 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:13.602922 14017 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:13.603502 13927 raft_consensus.cc:3055] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:13.604036 13927 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:13.605007 14057 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:13.605335 13927 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.607575 13963 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:13.608690 13963 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:13.624187 13963 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:13.626048 13963 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:13.629701 14046 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:04:13.567075 14019 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:13.633023 14019 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:13.633556 14019 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.635658  8827 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:13.576938  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:13.636214  8827 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:13.612834  8754 raft_consensus.cc:2410] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:04:13.613133 14017 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:13.614382 13713 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:13.637411 14016 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:13.617449  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:13.637140  8827 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.638494  8756 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:13.638945  8827 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:13.640313  8756 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:13.641191  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:13.641876  8748 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:13.644265 13878 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.644850 13878 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:13.649232  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 80247.2us
I20260709 13:04:13.550747 14033 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:13.572134  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:13.659183  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:13.593068  8755 rpcz_store.cc:276] 0709 13:04:12.366899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.378088 (+ 11189us) service_pool.cc:224] Handling call
0709 13:04:12.395778 (+ 17690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:12.466168 (+ 70390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:12.466222 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:12.480033 (+ 13811us) log.cc:844] Serialized 26 byte log entry
0709 13:04:13.287189 (+807156us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b18003f7540. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:04:13.341857 (+ 54668us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:13.341918 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.342127 (+   209us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:13.592045 (+249918us) raft_consensus.cc:1685] finished
0709 13:04:13.592084 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.592386 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":47048,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":18914,"spinlock_wait_cycles":38650624,"thread_start_us":54235,"threads_started":1,"wal-append.queue_time_us":929749}
I20260709 13:04:13.635769 14019 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:13.626869 13472 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:13.691485  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 118004us
I20260709 13:04:13.696549  8824 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 7.
I20260709 13:04:13.695863 13927 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:13.699501  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:13.700258  8828 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:13.580569 13851 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:13.702698 14046 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:04:13.703927 13726 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.706197 13726 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:13.707290 13878 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:13.707099 13702 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:13.704259 13851 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:13.712432  8825 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:13.716300  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:13.713129  8830 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
W20260709 13:04:13.617471 13876 consensus_meta.cc:287] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.674s	user 0.008s	sys 0.004s
I20260709 13:04:13.719925 13897 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:13.721243 14027 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.723425  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:13.736742  8827 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:04:13.724414  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:13.725647 14031 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.594s	user 0.002s	sys 0.000s
I20260709 13:04:13.749537  9384 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:13.737355 13876 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.751264 14055 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:13.751900 14055 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.751390 13849 consensus_queue.cc:260] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:13.752281 14055 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:13.752978 13849 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:13.753281 13876 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:13.749570  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:13.754945  8825 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:04:13.755384 13897 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.755517  9353 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:13.755960 13897 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.756124  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:13.757148  8825 raft_consensus.cc:2393] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:13.712652 13472 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:13.729311  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:13.735023 13851 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:13.744882 14014 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.573s	user 0.012s	sys 0.000s
I20260709 13:04:13.747856  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:13.737203 13726 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:13.758399  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:13.766934 13726 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:13.767496  8756 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:13.769268 13801 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:13.769424 13699 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:13.772495 13370 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:13.773522 13851 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.773967 14016 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:13.777863 13851 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:13.777690 13963 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:13.771736 14014 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.780565  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:13.758323 13900 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:13.781891 13472 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:13.783469  8824 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:13.769547 13726 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:13.784768 14027 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:13.787364  8752 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:13.734565  8748 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:13.794866  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 931) took 4179 ms. Trace:
I20260709 13:04:13.738121  8827 pending_rounds.cc:85] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 7
I20260709 13:04:13.817677  8827 pending_rounds.cc:107] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:04:13.818063  8827 raft_consensus.cc:2889] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:13.819187  8825 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:13.761009  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:13.816860  8672 rpcz_store.cc:276] 0709 13:04:09.615628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.171806 (+2556178us) service_pool.cc:224] Handling call
0709 13:04:12.231625 (+ 59819us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:13.309209 (+1077584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.789181 (+479972us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400430dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:13.794757 (+  5576us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2408826752}
W20260709 13:04:13.801836  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 469724us
I20260709 13:04:13.835350  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:13.842420  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:13.845214  8828 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:13.847394  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:13.851872  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2309) took 2026 ms. Trace:
I20260709 13:04:13.856248  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2135) took 1088 ms. Trace:
I20260709 13:04:13.856739  8827 rpcz_store.cc:276] 0709 13:04:12.768026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.768187 (+   161us) service_pool.cc:224] Handling call
0709 13:04:12.792882 (+ 24695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:13.635546 (+842664us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b540035a820. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2470dc0 00007f32e24fb1c0 00007f32e24fc231 00007f32e24f8d8f 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c
0709 13:04:13.855895 (+220349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.856172 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626557696}
I20260709 13:04:13.789634 14016 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.818531 13370 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:13.854598 14037 consensus_meta.cc:287] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.632s	user 0.011s	sys 0.003s
I20260709 13:04:13.854573  8796 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:13.870266  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:13.873440  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 453227us
I20260709 13:04:13.854051 14019 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:13.879257  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:13.881448  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:13.879397 14037 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:13.880473  8830 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
W20260709 13:04:13.865628 14005 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:04:13.870256 14027 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:13.877945  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:13.886127  8830 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:13.879882  8827 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 11. Current term is 12. Ops: []
W20260709 13:04:13.877140  8829 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.010s	sys 0.029s
I20260709 13:04:13.893031  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:13.896413  8647 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:13.861771 13876 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:04:13.920044  8719 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:13.936264 13726 raft_consensus.cc:2764] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:13.936203 13900 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.942291 13900 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:13.945351 13900 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:13.946837  8822 raft_consensus.cc:2393] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:13.949090  8748 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
W20260709 13:04:13.898733  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 46115us
I20260709 13:04:13.920513  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:13.928099  8755 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:13.927910  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:13.846508 14033 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
W20260709 13:04:13.936137  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.855110  8750 rpcz_store.cc:276] 0709 13:04:11.825584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.071441 (+245857us) service_pool.cc:224] Handling call
0709 13:04:12.071835 (+   394us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:12.074503 (+  2668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:12.074532 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:12.075259 (+   727us) log.cc:844] Serialized 52 byte log entry
0709 13:04:12.302485 (+227226us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:13.112957 (+810472us) log.cc:844] Serialized 14 byte log entry
0709 13:04:13.825590 (+712633us) log.cc:844] Serialized 14 byte log entry
0709 13:04:13.851398 (+ 25808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.851511 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:13.851575 (+    64us) raft_consensus.cc:1685] finished
0709 13:04:13.851591 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.851788 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1105977,"prepare.queue_time_us":820534,"prepare.run_cpu_time_us":1382,"prepare.run_wall_time_us":18102,"raft.queue_time_us":268259,"raft.run_cpu_time_us":520,"raft.run_wall_time_us":21924,"thread_start_us":746014,"threads_started":3,"wal-append.queue_time_us":301253}
W20260709 13:04:13.966945  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 47110.6us
I20260709 13:04:13.975379  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:13.976359  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:13.980786  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:13.982525  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:13.982474  8647 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:13.984473  9389 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:13.894717 14027 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:14.002151  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:13.989547  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:14.003494  8830 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:13.983793 14043 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:13.988507 14027 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:13.900720 13370 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:04:13.992329  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.996179 13900 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:14.005009 13851 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:13.992015  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:14.027385  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.029086  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 436991us
I20260709 13:04:14.007781 13726 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:14.044540  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 62187.4us
I20260709 13:04:14.021729 13370 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.050414  8793 leader_election.cc:400] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13.
W20260709 13:04:14.052527  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.054778  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:14.055552  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:14.055961  8722 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:14.057623 14062 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:14.058066  8722 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:14.059156 13889 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:14.048132 13845 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:14.063220 13845 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.063702 14068 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:14.064155 14068 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:14.064531 14068 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:14.065697 13845 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:14.068418 14016 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:14.034207 13726 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:14.070958 13726 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:14.043635 14024 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:14.086582  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:14.087530  9353 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:14.087929  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:14.058346 14062 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:14.099960  8794 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:14.107476  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 102721us
I20260709 13:04:14.101418 14033 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:04:14.116022  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 60575.8us
I20260709 13:04:14.077950 13370 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:14.078647 14014 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:14.117041  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:14.128942  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:14.130690  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:14.110672  9353 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:14.137064  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 107036us
I20260709 13:04:14.139092 13699 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:14.139452 13900 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:14.139974 13900 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.140355 13900 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:14.141006  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:14.142765 14016 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:14.143638 14016 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:14.144153 14016 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:14.145126 14016 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.148566 14018 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:14.149240 14018 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:14.149755 14018 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:14.151260 14037 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:14.153394  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:14.154215  9388 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:14.131769 14034 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:04:14.164398 14034 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:14.164808 14034 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:14.165738 14034 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.115826 14064 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:14.128777 13977 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:14.167304 13977 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.167852 13977 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:14.117302 14009 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:14.184995 14009 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.185564 14009 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:14.187873 14069 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:04:14.190832  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54340.9us
I20260709 13:04:14.192762  9388 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:14.101342 14055 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:14.200970  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 139463us
W20260709 13:04:14.172814  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 46761.6us
I20260709 13:04:14.214954  8796 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:14.178011 13897 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:14.195859 13699 raft_consensus.cc:697] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:14.219826  9355 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:14.252116 14043 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:14.267818  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:14.292248  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 60750.2us
W20260709 13:04:14.293135  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.293576  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 152664us
I20260709 13:04:14.228168 14060 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:14.226783 14034 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:14.294540  8647 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:14.300755  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 81085.2us
I20260709 13:04:14.306767 14027 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.306962 13845 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:14.265131 14033 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:14.308791 14033 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:14.256284 13847 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.311228 13847 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.258422 13897 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:14.319069 13897 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.321969 13897 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:14.324120 13847 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:14.335680  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:14.324479  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 186275us
I20260709 13:04:14.328397 14077 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:14.343336  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 85775.8us
I20260709 13:04:14.296072 14037 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.357215  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:14.359300  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:14.351979  8756 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:14.298071 14060 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:04:14.340600 14072 log.cc:927] Time spent T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:04:14.271209 14043 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:04:14.309948 14033 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.337677 13698 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:04:14.377939  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 73963.6us
I20260709 13:04:14.365717 13585 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:04:14.381068 13824 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:14.381834 13824 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:14.382916 13824 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:14.386176  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.365989 13472 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:04:14.389776 13472 consensus_queue.cc:1243] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:14.390404 13472 raft_consensus.cc:3055] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:14.332526 13897 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.397593  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:14.403015  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.272713 14062 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.372032 13927 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:04:14.409528  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.416446  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:14.417495  8678 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:14.354656 14061 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:04:14.397781 13585 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:14.390800 13472 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:14.431164 14058 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:14.439105  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.434465  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.401139 13963 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:14.433240 14079 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:14.444747  8672 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:14.343253 14077 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:14.421120 13849 consensus_meta.cc:287] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.668s	user 0.013s	sys 0.003s
I20260709 13:04:14.440165  8749 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:14.448061  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.448179  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.457726  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.458038  8752 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
W20260709 13:04:14.454844  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 115404us
I20260709 13:04:14.381901 13699 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.462352  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.406131 13897 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.495267 13847 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.496886 13897 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:14.406435 13338 consensus_queue.cc:1059] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:04:14.497633 13897 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:14.497241 14009 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.498664 13338 raft_consensus.cc:3055] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:14.435389 13585 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:14.470047 13472 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.480974 14081 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:14.499181 13338 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:14.481967 14073 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:04:14.444988 14054 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.450377 14079 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.485028 14060 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.488364 14077 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:14.496348 13847 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.404093 14037 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.468640 13370 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.502460  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.440877 14027 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:14.504941  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.515286  8793 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:14.528653  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.499493 13963 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:14.542735  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.546526  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 169688us
W20260709 13:04:14.530004 13801 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.760s	user 0.016s	sys 0.000s
I20260709 13:04:14.548417  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:14.567387  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 141741us
W20260709 13:04:14.563184  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 268764us
I20260709 13:04:14.569830  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:14.532619 13702 raft_consensus.cc:3055] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:14.527172 14079 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:14.578248  9355 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:14.580901  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:14.545159 14077 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.589973  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1024) took 1294 ms. Trace:
I20260709 13:04:14.589769 14077 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.590942  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.592298 14064 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:14.593298 14037 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:14.594826 14085 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:14.595463 14085 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.596290  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.556818 13472 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:14.607177  8645 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:14.581494 13702 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:14.588431 13585 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:14.589136  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 129663us
I20260709 13:04:14.599069  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.615994  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:14.616205  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:14.572088 13801 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:14.610956  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:14.620659  8679 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:14.611186  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.613610  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:14.614068  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.614178  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 176400us
I20260709 13:04:14.617326  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:14.599931  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.623888  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:14.514540 14081 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.583905 14009 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:14.619167  8793 leader_election.cc:400] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15.
I20260709 13:04:14.583185 14060 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:14.527586 14054 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.564637 13963 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.590405  8829 rpcz_store.cc:276] 0709 13:04:13.295659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.296565 (+   906us) service_pool.cc:224] Handling call
0709 13:04:13.335155 (+ 38590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:13.335687 (+   532us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:13.928671 (+592984us) log.cc:844] Serialized 14 byte log entry
0709 13:04:14.099778 (+171107us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:14.109181 (+  9403us) log.cc:844] Serialized 26 byte log entry
0709 13:04:14.109350 (+   169us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:14.109653 (+   303us) log.cc:844] Serialized 14 byte log entry
0709 13:04:14.109918 (+   265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.110059 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.589715 (+479656us) raft_consensus.cc:1685] finished
0709 13:04:14.589740 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.589904 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":200,"prepare.queue_time_us":792350,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":91,"raft.queue_time_us":9563,"raft.run_cpu_time_us":695,"raft.run_wall_time_us":46463,"thread_start_us":726198,"threads_started":2,"wal-append.queue_time_us":297775}
I20260709 13:04:14.627254  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.570488 14024 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:14.623039  8675 raft_consensus.cc:2376] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15. Re-sending same reply.
I20260709 13:04:14.628643  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.623605  8750 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:04:14.618932  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.632262  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.633045  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.635175  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.636021  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:14.647619  8748 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:14.611721 14014 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.637470 13977 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.616041 13848 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:14.649766  8752 raft_consensus.cc:2410] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:04:14.650094  8754 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 12: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:14.615370 14085 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:14.657438 13977 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:14.619526 14055 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:14.673460 13726 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.602s	user 0.008s	sys 0.003s
W20260709 13:04:14.682034  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 66420.6us
W20260709 13:04:14.686045 14068 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.621s	user 0.013s	sys 0.000s
I20260709 13:04:14.687853  8675 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:14.687064 14068 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.674304 13726 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.598141 13338 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.709112 13338 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:14.711114  8679 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:14.716693 13847 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:14.720772  8823 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:14.727414 13702 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.728209 14009 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.729117 13702 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:14.728850 14009 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.731659 14009 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:14.646660 13585 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:14.744058 14090 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:14.744695 14090 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:14.745060 14090 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:14.667820 14014 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.752731  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:14.753028 14014 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:14.753480  8751 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:14.762853 14081 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:14.763221  8566 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:04:14.763713 14081 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:14.764371 14081 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.764798 14081 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.768234 13963 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:14.771162  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.772833  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.773334 14077 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:14.716737 14054 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:14.785856  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:14.788105 13851 consensus_meta.cc:287] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.756s	user 0.011s	sys 0.002s
I20260709 13:04:14.788460  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.789240  8755 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:14.725853  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:14.791656 14009 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:14.791734  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 57901.7us
W20260709 13:04:14.793072  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 72204.5us
I20260709 13:04:14.793937  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:14.793947  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:14.795334  9380 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
W20260709 13:04:14.796452 14018 consensus_meta.cc:287] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.646s	user 0.007s	sys 0.007s
I20260709 13:04:14.796902 14018 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.799250  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:14.799724  8796 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:14.799968  8676 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:14.801870  8676 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:14.803478  8676 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:14.788671 13851 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:14.804358  8674 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:14.805164  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:14.806198  8676 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:14.806854 13927 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:14.725440  9381 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:14.808305 13927 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:14.808701 13927 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.809036 13927 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:14.811197  9380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:14.812047  9380 raft_consensus.cc:606] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:14.812400  9380 raft_consensus.cc:614] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:14.819823 13726 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:14.820966  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:14.793130 14009 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:14.832623  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 262865us
I20260709 13:04:14.854507  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:14.855986  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 248964us
I20260709 13:04:14.858281  8645 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:14.858811  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.859845  9380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:14.872660  9380 raft_consensus.cc:606] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:14.895030  9355 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:14.899650  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:14.912740  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:14.874886 14083 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.057s	user 0.001s	sys 0.001s
I20260709 13:04:14.879388 13726 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:14.870842  8676 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
W20260709 13:04:14.925098  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.877892 14034 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:14.931134  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:14.940929  8676 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:14.933296  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:14.911226 14085 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:14.929997 13900 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.787s	user 0.014s	sys 0.002s
W20260709 13:04:14.957512  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.942441 14054 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:14.959969 14054 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.963459  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:14.966240  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:14.967424  8674 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:14.968279  9380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:14.969038  9380 raft_consensus.cc:606] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:14.971197  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.929015 13585 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:14.930696 13726 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:14.976172  8829 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:14.974700  8678 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:14.977837  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:14.978461  8678 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:14.979728  8751 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:14.981527  8794 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:14.982141  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 161357us
I20260709 13:04:14.983022  8793 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:14.985589  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 131125us
I20260709 13:04:14.947031 14018 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:14.951539 14081 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:14.993204  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:14.967171 13900 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:14.962442 14054 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:14.996644  8681 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
W20260709 13:04:15.005930  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 361809us
I20260709 13:04:14.975334 13585 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.981954 13726 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:15.017051  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.992399 14034 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:15.027451  8675 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:04:15.029100  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.035038  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:15.007153 13963 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:15.038246 13963 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:15.039490 13963 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:15.040784 13927 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:14.979104 14090 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:15.049733 14090 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:15.052377 14094 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:04:15.053390  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 70453.4us
I20260709 13:04:15.056054  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:15.057137  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:15.067708  8681 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:15.057793 14014 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:15.062309  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:15.075685  8748 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:15.062894  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:15.068315  8676 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:15.012993 13585 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:15.082046  8752 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:15.086123  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:15.086998  8747 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:04:15.093703 13927 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:15.102480 14054 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:15.103273 14054 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:15.104256 13370 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:15.073422 14033 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.079012 14034 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:15.089259 14037 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:15.021697 14088 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:04:15.111281 14068 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:15.126230 14064 consensus_meta.cc:287] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.533s	user 0.020s	sys 0.000s
I20260709 13:04:15.145640  8678 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:15.154237  8678 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:15.158090  8681 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:15.117336 14033 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:15.164927 14097 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:04:15.164481 14034 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:15.171298 14098 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:15.171794 14054 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:15.175748  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:15.196224  8824 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:15.200138 14033 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:15.206832  9381 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:15.208770  9385 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:15.212332 14070 raft_consensus.cc:993] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: : Instructing follower e814ed2013244fa6be42623ea5c0c7b9 to start an election
I20260709 13:04:15.210759 13726 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:15.214159  8676 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
W20260709 13:04:15.215387  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 159430us
I20260709 13:04:15.216336  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:15.217562  8678 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:15.144897 14064 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:15.183738 14009 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:15.187754 13900 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:15.213142 14070 raft_consensus.cc:1081] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Signalling peer e814ed2013244fa6be42623ea5c0c7b9 to start an election
I20260709 13:04:15.221128  9353 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:15.176177 14079 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.559s	user 0.008s	sys 0.003s
I20260709 13:04:15.224062  9353 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:15.227185  8747 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:15.225538  9355 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:15.231840 13370 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:15.239553  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:15.240149 13585 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:15.223201 14106 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:15.241842  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:15.230113 14053 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:15.246308  9385 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:15.248970 14009 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:15.245466 14053 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:15.250104 14053 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:15.253316 13977 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:15.263343 13977 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:15.263880 13977 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:15.231271 14054 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:15.230014 13889 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:04:15.269985  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:15.274060 13889 raft_consensus.cc:3055] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:15.274575 13889 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:15.274799  8829 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:55168
I20260709 13:04:15.275657  8829 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:15.276084  8829 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:15.275789 13889 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:15.277510 13889 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:15.230033 14079 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:15.281088 14079 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:15.281985  8673 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:15.267423 14054 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.283383 14054 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:15.286314  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:15.288967  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:15.291316 14060 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:15.291960  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:15.292207 14060 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:15.293341  9380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:15.295063  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:15.296412  8719 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:15.304008  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:15.290505  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:15.319862  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:15.316154 14060 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:15.313149 14054 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:15.322927  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:15.321637 14034 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:15.332624 14034 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.333469 14034 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:15.334916 14054 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:15.336251 13585 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:15.337708  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:15.339193  9380 raft_consensus.cc:606] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:15.339403 13726 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:15.339722  9380 raft_consensus.cc:614] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:15.339890  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:15.342095 13927 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:15.341316  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:15.343879  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:15.318280 13370 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:15.347532 13472 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:15.347575  8752 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:15.347922 14033 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:15.349604 13472 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:15.346166 13963 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:15.350642  8793 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:15.351691 13963 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:15.356411  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 423174us
I20260709 13:04:15.356623 13370 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:15.356472 14016 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:15.358791 13851 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:15.361747  8752 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:15.362779  8645 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:15.321393 14084 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:04:15.367206  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 380492us
I20260709 13:04:15.346236 14064 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.368244  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:15.368307 14064 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:15.369313  8647 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:15.370787 13927 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:15.371054 14018 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:15.371677 14018 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:15.371500 13927 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:15.372823 14018 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:15.374761 13927 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:15.343988 14060 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:15.376106 14060 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:15.379168  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:15.379982  8678 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:04:15.380941  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:15.381474  8678 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:15.383044  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:15.383798  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:15.327170 14055 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:15.386096  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:15.389953  8747 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:04:15.387346  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:15.387768  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:15.388760  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.360806 13977 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:15.394555  8674 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
W20260709 13:04:15.412842  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 50137.2us
I20260709 13:04:15.386312  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:15.417346  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:15.402364  8722 leader_election.cc:336] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.511s (SENT)
W20260709 13:04:15.419680  8722 leader_election.cc:336] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.513s (SENT)
I20260709 13:04:15.420245  8722 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:15.423724  8676 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:15.411324  8673 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:04:15.326849 14024 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:04:15.423890  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:15.428822  8677 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:15.430585  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.430979  8675 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:04:15.435153  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:15.370852 14081 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:15.374840 14095 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:15.441156  8752 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:15.443085  9380 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:15.446837  8676 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:15.448685  8794 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:15.450377 13370 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:15.425196 14084 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:15.435197 14096 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:15.435472 14033 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.401189 13585 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
W20260709 13:04:15.467741  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 97247.4us
I20260709 13:04:15.468647  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:15.411581 14062 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:04:15.465977  8678 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:15.443783 14095 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:15.564054 14095 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:15.439440 14064 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:15.451457 14068 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:15.567267 14068 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.567764 14068 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:15.489635  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:15.571178  9381 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 12: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:15.490840  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:15.484700  9380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:15.572572  8828 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:15.572710  9380 raft_consensus.cc:606] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:15.573173  9380 raft_consensus.cc:614] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:15.567853  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:15.575730  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:15.484661  9352 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:15.497275  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:15.576751  8830 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:15.458060 14085 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:15.577303  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:15.463989 13963 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:15.498884 13878 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:04:15.578199 13963 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:15.578732 13963 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:15.504243  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:15.577848 14070 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:15.506040 14084 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:15.579875  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:15.581087  8673 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 14: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:15.505048  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:15.507170  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:15.582960  8821 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:15.583199  8822 raft_consensus.cc:2393] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:15.508108  8827 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:15.513911  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:15.585918  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:15.511185  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:15.515087  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:15.587343  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:15.587682  9385 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:15.521822  8829 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:15.588238  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:15.588688  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:15.589856  8679 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:15.590041  8675 raft_consensus.cc:3055] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:15.531466  9388 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:15.473865 14033 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:15.533790  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.467233 14014 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:04:15.539091  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:15.498482 14082 consensus_queue.cc:1059] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:04:15.515461 13973 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:04:15.545203 14106 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:15.528123 14081 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.553037 14034 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:15.554832  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:15.622800 13878 raft_consensus.cc:3055] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:15.560096  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:15.575820  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:15.577121  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 92616.3us
I20260709 13:04:15.579416  8825 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:15.536067 13977 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:15.593148  8829 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:15.582260  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:15.590391  8675 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:15.592743  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:15.498715 14016 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:15.613296  9389 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:15.579910 14070 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.469259  8747 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:15.615864  8677 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:04:15.571429  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:15.657573  8678 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:15.621178  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:15.658834  8676 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:15.629288  8821 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:15.634224  9383 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:04:15.651368  9381 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:15.663074  8574 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43462 (request call id 268) took 43630 ms (client timeout 15000 ms). Trace:
I20260709 13:04:15.659201  9353 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:15.613914  8681 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:15.651609  8672 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:15.649240 13370 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.634565  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:15.668054  8674 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:15.671268  8647 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:15.634402 13900 raft_consensus.cc:3055] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:15.585732 14068 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:15.634243 13878 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:15.656184 14016 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.697755  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:15.679862  8675 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:15.697754 13370 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:15.725530  8675 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:15.706076  9355 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:15.706724  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 361465us
I20260709 13:04:15.706648  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:15.736526  9352 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:15.698329  9385 raft_consensus.cc:2410] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:04:15.704922  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:15.659984 14070 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:15.700564 13851 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:15.699827 14085 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.727362  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:15.663589  8574 rpcz_store.cc:269] 0709 13:03:32.032602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.064342 (+ 31740us) service_pool.cc:224] Handling call
0709 13:04:15.662778 (+43598436us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:11.535521 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:11.536570 (+  1049us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:11.536625 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:04:11.536660 (+    35us) tablet.cc:662] Decoding operations
0709 13:04:11.538452 (+  1792us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:11.575656 (+ 37204us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:11.586860 (+ 11204us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:04:12.016243 (+429383us) tablet.cc:701] Row locks acquired
0709 13:04:12.045387 (+ 29144us) write_op.cc:260] PREPARE: finished
0709 13:04:12.045611 (+   224us) write_op.cc:270] Start()
0709 13:04:12.045785 (+   174us) write_op.cc:276] Timestamp: P: 1783602252045533 usec, L: 0
0709 13:04:12.045830 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 13:04:12.047157 (+  1327us) log.cc:844] Serialized 5134 byte log entry
0709 13:04:12.787261 (+740104us) spinlock_profiling.cc:243] Waited 5.56 ms on lock 0x7b18003f76c0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:13.266781 (+479520us) op_driver.cc:464] REPLICATION: finished
0709 13:04:13.494508 (+227727us) write_op.cc:301] APPLY: starting
0709 13:04:13.515021 (+ 20513us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:13.617869 (+102848us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:13.618072 (+   203us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:13.727239 (+109167us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:13.727687 (+   448us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 13:04:13.727758 (+    71us) write_op.cc:312] APPLY: finished
0709 13:04:13.728238 (+   480us) log.cc:844] Serialized 121 byte log entry
0709 13:04:13.746796 (+ 18558us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:13.747005 (+   209us) write_op.cc:454] Released schema lock
0709 13:04:13.747085 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:13.748162 (+  1077us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:13.748203 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1146388,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":127862,"apply.run_cpu_time_us":24858,"apply.run_wall_time_us":331478,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58415,"num_ops":13,"prepare.queue_time_us":330439,"prepare.run_cpu_time_us":22250,"prepare.run_wall_time_us":814787,"raft.queue_time_us":76091,"raft.run_cpu_time_us":24591,"raft.run_wall_time_us":85025,"replication_time_us":1218123,"spinlock_wait_cycles":12781952,"thread_start_us":586663,"threads_started":4,"wal-append.queue_time_us":334249,"wal-append.run_cpu_time_us":10441,"wal-append.run_wall_time_us":1956171}]]}
W20260709 13:04:15.764442  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 185333us
I20260709 13:04:15.703770 13900 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:15.729264 14016 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:15.740990  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:15.741834 14062 consensus_queue.cc:1059] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:04:15.741961  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:15.742398  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:15.743013 13977 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:15.748594 14102 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:15.745514  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:15.745726  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:15.725193  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 54094.3us
I20260709 13:04:15.755401  8830 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:15.756660  8680 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:15.759855  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:15.710436 13702 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:15.766485  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:15.789140  8827 raft_consensus.cc:2410] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:15.766639  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:15.652935 14081 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:15.769608  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:15.708154 14053 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:15.765151  8722 leader_election.cc:336] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.565s (SENT)
I20260709 13:04:15.810806  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:15.728816 14077 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:15.777321  8821 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:15.744683  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:15.786520  8647 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:15.791741  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:15.821151  8830 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:15.821290  8828 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:15.821628  8675 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:04:15.793658  8829 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:15.701390 14018 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:15.794586  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 49159.7us
I20260709 13:04:15.803640  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:15.823364  8823 raft_consensus.cc:2410] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:04:15.793253  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:15.823916  8822 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:15.822773  8675 pending_rounds.cc:85] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 9
I20260709 13:04:15.824424  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:15.824952  8675 pending_rounds.cc:107] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:04:15.825328  8675 raft_consensus.cc:2889] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:15.823211  8719 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:15.830710  8719 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:15.832890  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:15.834793  9384 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:15.769285 14085 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:15.790275 13977 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:15.847450  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 82337.7us
W20260709 13:04:15.855790  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 96141.2us
I20260709 13:04:15.820247 14024 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:15.889196  8827 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:15.853964  8830 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:15.822495 13702 raft_consensus.cc:697] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:15.907267  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 56342.8us
I20260709 13:04:15.832036 14096 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:15.913095  8821 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:15.829705 14082 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:15.838315 13889 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:15.915624  8830 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:15.849433 14055 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:15.855208 14095 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:15.855070 14062 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:15.855262 13698 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.934211  8829 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:04:15.940049  8829 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:04:15.944860  8830 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:04:15.995464  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.918347 14096 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:15.846746 14099 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.106s
I20260709 13:04:16.016870  8830 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:16.009743  8829 pending_rounds.cc:85] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 7
I20260709 13:04:16.020042  8829 pending_rounds.cc:107] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:04:16.020359  8829 raft_consensus.cc:2889] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:16.013795  8828 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:04:16.014561  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 116959us
W20260709 13:04:16.036404  8722 leader_election.cc:336] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.810s (SENT)
I20260709 13:04:16.037322  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:16.039808  8824 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:15.942869 14033 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:15.838794 14018 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:16.047209  8830 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:16.004184 14034 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:15.833119 13878 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:16.058737  8722 leader_election.cc:336] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.876s (SENT)
W20260709 13:04:16.052196  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.858446 13900 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:16.051047  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:16.060969  8828 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:16.061987 14068 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:16.062731 13900 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:16.064915 14081 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:16.065968 13977 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:16.066951 14088 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.067608 14088 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:16.001456 13698 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:16.019677 14096 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:16.075085  8722 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:16.003078 13889 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:04:16.080008  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:16.080399  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:16.092516  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:16.099241  8829 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:16.100592  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:16.101819  8829 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:16.094828  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:16.023214 14062 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:04:16.103593  8824 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:16.116171  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:16.183578  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:16.185250  8828 raft_consensus.cc:2410] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:16.196473  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:16.198915  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:16.200111  8828 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:16.210675  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:16.213100  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:16.219468  8824 raft_consensus.cc:2393] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:16.218701  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:16.220737  8830 raft_consensus.cc:2376] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18. Re-sending same reply.
W20260709 13:04:16.225638  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.085736 13889 consensus_queue.cc:1243] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:16.247422  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:16.091976 14054 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.096907 13370 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:16.187647 13878 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:16.254788  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 196067us
W20260709 13:04:16.257404  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.258109  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:16.218699 14116 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.479s	user 0.002s	sys 0.000s
I20260709 13:04:16.260488  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:16.260771  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:16.261368  8824 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:16.262679  8722 leader_election.cc:336] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.575s (SENT)
W20260709 13:04:16.264369  8722 leader_election.cc:336] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.894s (SENT)
I20260709 13:04:16.075480 13963 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:16.269524 13370 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:16.071928 13702 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:16.189015 14126 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:16.281399  8722 leader_election.cc:336] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.167s (SENT)
I20260709 13:04:16.249303 14055 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:16.310835  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:16.312690  8828 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:16.255436 13698 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:16.315439  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:16.323969  8829 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:16.332887  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:16.339466 14126 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:16.341711  8825 raft_consensus.cc:2393] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:15.917254  8572 catalog_manager.cc:5697] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 11 to 14, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 14 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:16.264622 14053 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:16.348042  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 127855us
I20260709 13:04:16.356187  8828 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:16.357571  8824 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:16.365332  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:16.366283  8828 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:16.260075 14054 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:16.373034  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 106454us
I20260709 13:04:16.344153 14085 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:16.380930  8824 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:16.395879  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:16.396564  8824 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:16.382735  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:16.382788  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:16.383991  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:16.399302  8674 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:16.400897  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:16.401895  8673 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
W20260709 13:04:16.389961  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.380903  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:16.405683  9355 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:16.409637  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:16.410897  8680 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 14: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:16.420717  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:16.421171  8829 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:16.421958  8678 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:04:16.425345  8822 consensus_meta.cc:287] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.595s	user 0.007s	sys 0.006s
I20260709 13:04:16.426103  8822 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:16.426469  8751 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:04:16.429745 14088 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:16.430349  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:16.431773 14088 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:16.432356 14088 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:16.433343 14088 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:16.434978 13889 consensus_queue.cc:1059] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:16.431699  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:16.437178 13889 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:16.438730 13370 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:16.439956 13370 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:16.440394 13370 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:16.441524 13370 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:16.444094  9353 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.003s	sys 0.015s
I20260709 13:04:16.428591  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:16.446357  9353 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.004s	sys 0.015s
W20260709 13:04:16.447543  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 535830us
I20260709 13:04:16.446580  8752 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:16.340319 14126 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:16.381007 14106 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.448038 14121 raft_consensus.cc:993] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: : Instructing follower e814ed2013244fa6be42623ea5c0c7b9 to start an election
I20260709 13:04:16.459321  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:16.460000  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:16.445268 14024 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.003s	sys 0.015s
W20260709 13:04:16.450438  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:16.480370 14024 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.003s	sys 0.016s
I20260709 13:04:16.449728 14095 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:16.489202  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:16.476431  8719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.007s	sys 0.013s
I20260709 13:04:16.491647  8754 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
W20260709 13:04:16.491899  8719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.007s	sys 0.013s
W20260709 13:04:16.492280  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 577485us
I20260709 13:04:16.492673  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:16.479902  8750 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:16.503125 14140 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.503916 14140 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:16.506206 14140 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:16.507330 14129 consensus_queue.cc:1059] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:16.508234 14129 consensus_queue.cc:1243] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:16.508762 14129 raft_consensus.cc:3055] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:16.509114 14129 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:16.503358 14056 raft_consensus.cc:1081] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Signalling peer e814ed2013244fa6be42623ea5c0c7b9 to start an election
I20260709 13:04:16.503135  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:16.489776 13370 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:16.477984 14099 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:16.516162  8719 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:16.748092  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 232055us
I20260709 13:04:16.748953  8719 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:16.490137 14024 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:16.580869  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 59777.6us
I20260709 13:04:16.751466 14024 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:16.752612  8748 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:16.469866 14106 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:16.597739  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.507357 14018 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:16.449409 14055 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:16.617285  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 211703us
W20260709 13:04:16.628947  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 183236us
W20260709 13:04:16.632627  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 140020us
W20260709 13:04:16.590133 14122 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.371s	user 0.002s	sys 0.000s
I20260709 13:04:16.501443 14084 consensus_queue.cc:1059] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
W20260709 13:04:16.640205 14138 log.cc:927] Time spent T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:16.641251 14126 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:16.642261 13963 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:16.645143 13900 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.000s
I20260709 13:04:16.675873  8750 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
W20260709 13:04:16.673341 14016 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.888s	user 0.009s	sys 0.008s
I20260709 13:04:16.510141 14129 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:16.694357  9388 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:16.636379  8572 catalog_manager.cc:5697] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:16.699613 14054 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:16.700443 14150 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:16.732302 14151 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:16.740551 14070 consensus_meta.cc:287] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.974s	user 0.014s	sys 0.000s
I20260709 13:04:16.491293 14014 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:04:16.749712 14124 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:16.800401 14124 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:16.801070  9380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 994) took 1037 ms. Trace:
I20260709 13:04:16.773932 14016 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:16.801497 14124 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:16.801659  9380 rpcz_store.cc:276] 0709 13:04:15.763382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.763609 (+   227us) service_pool.cc:224] Handling call
0709 13:04:15.764131 (+   522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:15.764502 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:15.764530 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:15.770070 (+  5540us) log.cc:844] Serialized 26 byte log entry
0709 13:04:15.935066 (+164996us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:15.935123 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.935567 (+   444us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:16.800754 (+865187us) raft_consensus.cc:1685] finished
0709 13:04:16.800781 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.800956 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2604,"raft.queue_time_us":3466,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":4004,"thread_start_us":123018,"threads_started":1,"wal-append.queue_time_us":261089}
I20260709 13:04:16.803750 14016 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:16.806733  8572 catalog_manager.cc:5697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 9 to 12, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 12 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:16.757853 13847 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:16.758701  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:16.756726  8719 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:16.816515  9384 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
W20260709 13:04:16.816821  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 60197.2us
W20260709 13:04:16.758323  8722 leader_election.cc:336] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.129s (SENT)
I20260709 13:04:16.762409  9355 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:16.819325  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 56995.3us
I20260709 13:04:16.821429  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:16.754951 14096 consensus_meta.cc:287] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.649s	user 0.012s	sys 0.003s
I20260709 13:04:16.776000  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2156) took 1073 ms. Trace:
I20260709 13:04:16.822122  8830 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:04:16.755682 14152 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:16.777413 14070 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:16.781924  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:16.757308  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:16.810496  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2164) took 1044 ms. Trace:
I20260709 13:04:16.823794  8823 rpcz_store.cc:276] 0709 13:04:15.702744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.913637 (+210893us) service_pool.cc:224] Handling call
0709 13:04:15.931391 (+ 17754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:15.944240 (+ 12849us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:16.449098 (+504858us) log.cc:844] Serialized 14 byte log entry
0709 13:04:16.775301 (+326203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:16.775381 (+    80us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:16.775507 (+   126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.775582 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.775935 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":398310,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":92478,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":691108,"threads_started":3,"wal-append.queue_time_us":244865}
I20260709 13:04:16.837865  8821 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:55168
I20260709 13:04:16.840896  8821 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:16.841367  8821 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:16.762919 13370 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:04:16.776232 14084 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:16.851960 14159 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:16.855540  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:16.840035  8826 rpcz_store.cc:276] 0709 13:04:15.765911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.938545 (+172634us) service_pool.cc:224] Handling call
0709 13:04:16.018830 (+ 80285us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:16.050259 (+ 31429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.050293 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:16.192519 (+142226us) log.cc:844] Serialized 52 byte log entry
0709 13:04:16.372381 (+179862us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:16.372424 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.372498 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:16.810177 (+437679us) raft_consensus.cc:1685] finished
0709 13:04:16.810202 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.810424 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25700,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":17977,"thread_start_us":109329,"threads_started":1,"wal-append.queue_time_us":221087}
I20260709 13:04:16.856516  8823 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:16.768291 14099 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:16.850831 14096 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:16.865625  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 107326us
I20260709 13:04:16.838711 13585 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:16.843362 14126 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:16.882524  9355 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:16.886914  9355 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:16.880420  8821 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:16.889938 14102 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:16.890595 14102 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.891074 14102 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:16.896252  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:16.902786  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:16.903829  8828 raft_consensus.cc:2410] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:16.904500  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:16.907835  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.909473 14131 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:16.910859 14095 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:16.911304 14095 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:16.912328 14095 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:16.919929  8830 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:04:16.916082  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:16.885633 13585 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.926414 13585 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:16.926566 14154 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:16.815690 14151 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:16.860296 14084 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:16.899973 14018 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:16.930291 14084 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:16.931250 14018 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:16.931933 14084 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:16.898767 13963 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:04:16.933482  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:16.934171  8827 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:16.827515 13847 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:16.865397 14016 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:16.935758  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 178505us
I20260709 13:04:16.940028  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:16.942176  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:16.886034 13900 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.809237  8572 catalog_manager.cc:5697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:16.919317 14053 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:16.947750  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 104465us
I20260709 13:04:16.949028  8793 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:16.952775  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:16.954582  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:16.960559  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.923636 14126 raft_consensus.cc:697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:16.968403  9353 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:16.943226 14014 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:16.950099 13963 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:16.967742 14053 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.949579 14151 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:16.965734 14154 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:16.983639  8827 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:04:17.002890  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:17.004011  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:16.975805 14018 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:16.997051  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 122305us
I20260709 13:04:16.951782 13900 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:16.970696 14016 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.001127 14053 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:17.031844  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 77366.6us
W20260709 13:04:17.035118  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.044178  8793 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:17.016999 14137 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:04:17.044976  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.022912 13963 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:17.025313 14018 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:17.048965  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:17.011673 14161 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:17.051090  8674 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260709 13:04:17.052299  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 72361.6us
I20260709 13:04:17.025235 14106 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:17.165359  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:17.065450 14111 raft_consensus.cc:3055] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:17.063756  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:17.105026 14070 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:17.167695  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:17.173213 14111 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:17.167091  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:17.066849 14068 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:17.176952 14068 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:17.177064  8754 raft_consensus.cc:2410] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:04:17.064285 14106 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:17.053839 14014 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:17.178128 14068 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:17.051976 14154 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:17.179520 14068 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:17.178910 14111 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.179688 14154 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:17.130973  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.180801 14154 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:17.049264 13963 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.181880 13963 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:17.150245 14099 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:17.050139 14152 raft_consensus.cc:697] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:17.077669  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:17.116376 13900 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:17.079252  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:17.054834 14137 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:17.184814  8674 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:17.150018  8572 catalog_manager.cc:5697] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:17.185896 14137 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:17.134234  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:17.067202 14156 log.cc:927] Time spent T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:04:17.187183  8572 catalog_manager.cc:5697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:04:17.123148 14157 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.124s	user 0.000s	sys 0.002s
I20260709 13:04:17.079605 14062 raft_consensus.cc:3055] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:17.189517 14062 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:17.189054  8572 catalog_manager.cc:5697] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 13, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 13 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:17.050022 14018 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:17.077642  8672 pending_rounds.cc:85] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 5
W20260709 13:04:17.211042  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.211252  8672 pending_rounds.cc:107] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:04:17.183939 14152 consensus_queue.cc:237] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.103318  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:17.215027  8827 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:17.196544 14096 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:17.053399  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:17.220088  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 166825us
W20260709 13:04:17.115022 14118 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.095s	user 0.005s	sys 0.005s
I20260709 13:04:17.075515 14126 consensus_queue.cc:237] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.034890 13370 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:17.192870  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:17.193037  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:17.192478  8754 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:17.192907  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:17.223573  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 120913us
I20260709 13:04:17.178129 14106 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:17.196740  8678 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:17.203549  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.211742  8672 raft_consensus.cc:2889] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:17.190738 14062 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.222240  8793 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:17.228421  8672 pending_rounds.cc:107] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:04:17.239701  8827 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:17.276620  8672 raft_consensus.cc:2889] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:17.277887  9355 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:17.278277  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 56142.5us
I20260709 13:04:17.280016  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:17.281844  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:17.284737  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:17.215131 14018 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:17.289489  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:17.289696  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:17.290988  8754 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:17.291700  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:17.195242 14167 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.152s
I20260709 13:04:17.294787  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:17.300355  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 74378.3us
I20260709 13:04:17.222880 13847 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:17.311842  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:17.315155  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 399169us
I20260709 13:04:17.207340 14166 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 13:04:17.277386 14034 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:17.316208  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:17.323416  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:17.287981 14161 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:17.227310 14151 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:04:17.331445  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:17.331521  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:17.304103 14014 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:17.340348  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:17.306641 14160 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:17.340443  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:17.347839  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:17.356662  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:17.356906  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:17.357498  8752 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:17.306802 14168 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:17.362537  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:17.359169  8722 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:17.367045  8647 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:17.346601 14096 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:17.379640 14182 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:17.379781 14096 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:17.380246 14182 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:17.380419 14096 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:17.381305 14182 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.354121 13900 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:04:17.383100 14179 log.cc:927] Time spent T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:04:17.376577 13927 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:17.344266 14014 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.348491 13726 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:17.378743 14170 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:17.391588  8752 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:17.393760 14014 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:17.363688 14160 raft_consensus.cc:697] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:17.394829 14169 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.359622 14164 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.189s
I20260709 13:04:17.395213 14177 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:04:17.411273  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:17.347306 14034 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:17.353346 14139 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.188s
W20260709 13:04:17.432821  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 68635us
W20260709 13:04:17.437053  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 148240us
I20260709 13:04:17.403157 14178 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:17.396116 14173 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:17.396668 13726 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:17.469871  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2698) took 1964 ms. Trace:
I20260709 13:04:17.404614 14170 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.441128  8827 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:04:17.476567  8827 pending_rounds.cc:85] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 9
I20260709 13:04:17.476877  8827 pending_rounds.cc:107] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:04:17.477130  8827 raft_consensus.cc:2889] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:17.443930  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:17.444736  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 305810us
I20260709 13:04:17.434453  8572 catalog_manager.cc:5697] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 6 to 12, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 12 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:17.397351 13585 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:17.485503  8825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2147) took 1816 ms (client timeout 1513 ms). Trace:
I20260709 13:04:17.489328  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:17.446481  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:17.491067  8826 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:17.493795  8824 raft_consensus.cc:2393] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:17.453639  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:17.435463 14084 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.503s	user 0.015s	sys 0.001s
I20260709 13:04:17.470914  8679 rpcz_store.cc:276] 0709 13:04:15.505753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.623530 (+117777us) service_pool.cc:224] Handling call
0709 13:04:15.623989 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:15.632561 (+  8572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:15.632623 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:15.649632 (+ 17009us) log.cc:844] Serialized 26 byte log entry
0709 13:04:15.819851 (+170219us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:15.857704 (+ 37853us) log.cc:844] Serialized 14 byte log entry
0709 13:04:15.857957 (+   253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.858037 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:16.336601 (+478564us) spinlock_profiling.cc:243] Waited 6.56 ms on lock 0x7b18004e0fa0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:17.114280 (+777679us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:17.456268 (+341988us) raft_consensus.cc:1685] finished
0709 13:04:17.456304 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:17.469783 (+ 13479us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51432,"prepare.queue_time_us":37643,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":10775,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":13169,"spinlock_wait_cycles":26044160,"thread_start_us":92498,"threads_started":1,"wal-append.queue_time_us":212008}
I20260709 13:04:17.478587 14018 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.503525 14185 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.504187 14185 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:17.505967 14018 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:17.504848  8572 catalog_manager.cc:5697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 9 to 11, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 11 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:17.506594 14174 raft_consensus.cc:2764] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:04:17.473649 14126 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:17.508278 14126 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:04:17.440337  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 81260us
I20260709 13:04:17.508966  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:17.510193  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:04:17.510447  9388 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 993) took 1895 ms (client timeout 1806 ms). Trace:
I20260709 13:04:17.482700  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:17.496069  8827 raft_consensus.cc:2376] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13. Re-sending same reply.
W20260709 13:04:17.486114  8825 rpcz_store.cc:269] 0709 13:04:15.668877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.744409 (+ 75532us) service_pool.cc:224] Handling call
0709 13:04:17.485097 (+1740688us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b5400126e54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:17.485388 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1175824640}
I20260709 13:04:17.513684  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:17.510880  9388 rpcz_store.cc:269] 0709 13:04:15.592314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.592472 (+   158us) service_pool.cc:224] Handling call
0709 13:04:17.487117 (+1894645us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400510254. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:17.487302 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2365276544}
I20260709 13:04:17.456226 14167 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:17.475543 13726 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:17.536535 14167 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.536922  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:17.539892  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:17.544679  8827 raft_consensus.cc:2376] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13. Re-sending same reply.
I20260709 13:04:17.537849  8830 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:17.506709 14018 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.477700 14161 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:17.499635 14068 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:17.542230  8754 raft_consensus.cc:2410] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:17.545856  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:17.555002  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:17.553720  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:17.480196 14170 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:17.557109  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:17.551795  8830 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:17.552297  9384 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:17.508633 14126 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:17.435192 13370 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:04:17.555791  8824 raft_consensus.cc:2376] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15. Re-sending same reply.
I20260709 13:04:17.562938  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:17.476933 14053 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:17.553385  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:17.560779  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:17.500075 14151 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:17.530797 14195 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.569499 14151 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:17.569664 14195 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.572412  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:17.573118 14195 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:17.573511  8674 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:17.572412  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:17.432114 14169 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:17.544263 14167 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:17.635038  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:17.635998  8822 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:17.617806 14106 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:17.642980  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:17.643790  8677 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
W20260709 13:04:17.619699 14181 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:04:17.620289 14176 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:17.564364 14178 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:17.558677 14174 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:17.671128 14204 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:17.671957 14174 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:17.671806 14204 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.672474 14174 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:17.672973 14174 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:17.567444  8825 raft_consensus.cc:2376] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14. Re-sending same reply.
I20260709 13:04:17.627640 13370 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:17.675004 14178 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:17.675279 13370 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:17.675722 14178 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:17.676414 13370 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.676824 14178 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.677927 13370 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:17.628113 13585 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:17.554311 14160 consensus_queue.cc:237] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:17.629794  8674 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:17.630687  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:17.551668 14139 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:17.679584  8679 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:17.674293 14204 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:17.679965  8752 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:17.653617  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:17.554142 13900 raft_consensus.cc:697] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:17.557024 14018 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:17.557719 14068 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:17.494575 14173 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:17.685060  8755 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:17.686288  8822 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
W20260709 13:04:17.685927  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.688555  9355 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:17.688715  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:17.695274  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:17.696041  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:17.697423  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.660521 14195 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:17.560519 14161 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.696069  9380 raft_consensus.cc:2376] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11. Re-sending same reply.
I20260709 13:04:17.682978 14139 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.708997  8796 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:17.686313 14068 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:17.697695 14102 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.806s	user 0.003s	sys 0.013s
W20260709 13:04:17.722419  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.727439  8679 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:17.695843 13698 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:17.725606 14131 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.730019 14131 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:17.730568 14192 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:04:17.732841 14131 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:17.734319 14099 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:17.736829 14197 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.737469 14197 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:17.739094  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.741212 14197 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:17.747048 13900 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:17.755240 14150 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.978s	user 0.013s	sys 0.003s
I20260709 13:04:17.755977 14150 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:17.756095 13977 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.756837 13977 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:17.710548 14139 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:17.680219 14176 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:17.834100  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:17.784197  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 417171us
I20260709 13:04:17.729104 14204 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:04:17.796558  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.837507  8647 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:17.720098 14195 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.843479  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:17.845319  8647 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:17.841106  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:17.786881 13963 consensus_meta.cc:287] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.604s	user 0.012s	sys 0.003s
W20260709 13:04:17.818797 14175 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.308s	user 0.000s	sys 0.002s
I20260709 13:04:17.850566 13963 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.853858 13963 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:17.699116 14126 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:17.839049 14154 consensus_meta.cc:287] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.658s	user 0.012s	sys 0.001s
I20260709 13:04:17.782593 13698 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:17.820089 14161 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:17.785686 14150 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:17.863885  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:17.811481 13977 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:17.879105  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 150240us
I20260709 13:04:17.847051 14106 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.728415 14102 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:17.882827  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:17.847527 14207 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:17.849701 14131 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:17.855697 14034 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:17.822057 13726 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.989902  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:17.992288  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:17.994271  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:17.900409  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:17.901062  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:17.900085  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:18.000805  8755 raft_consensus.cc:2410] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:17.917662  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:17.947306  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46646.7us
I20260709 13:04:18.003680  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:17.920614  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:18.004884  8672 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:17.885758 14053 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:17.961227 14106 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.046975 14018 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:18.047768 14018 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:18.048425 13726 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:18.049927  8678 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:18.050148 14096 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.669s	user 0.015s	sys 0.000s
I20260709 13:04:18.050738 14096 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.053130 14096 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:18.037169  8681 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 8: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:18.053759 13698 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.057159 14102 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:18.059648  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:17.860471 14154 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:17.898864 13900 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:18.036296  8647 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:18.089906  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:18.046357 14106 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:18.091801  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 190777us
W20260709 13:04:18.176215  8722 leader_election.cc:336] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.557s (SENT)
I20260709 13:04:18.176855  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:18.097806  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 61603.5us
I20260709 13:04:18.178388  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:18.090451  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2755) took 1665 ms. Trace:
I20260709 13:04:18.100697  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:18.127563  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.179730  8676 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:18.179455  8748 rpcz_store.cc:276] 0709 13:04:16.424881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.428124 (+  3243us) service_pool.cc:224] Handling call
0709 13:04:18.089865 (+1661741us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b5400156dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:18.090363 (+   498us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":650502912}
I20260709 13:04:18.219285  8677 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:04:18.221580  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 148906us
W20260709 13:04:18.225814  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.268188  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.271838  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 237478us
I20260709 13:04:18.273348  8793 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:18.273609 14120 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.218003 14096 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:18.278908  8680 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:18.287983 14184 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.255s
I20260709 13:04:18.289710 14213 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:18.295534 14201 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.237s
I20260709 13:04:18.300534 14205 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:18.358552  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 85293.9us
W20260709 13:04:18.373518  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.419706  8828 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 13:04:18.422518  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 244235us
I20260709 13:04:18.424064  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:18.360024  8793 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:18.426072  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 66178.2us
I20260709 13:04:18.060914 14169 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:17.801110  8572 catalog_manager.cc:5697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:18.449754 14208 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:18.450456  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:18.451272  8822 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:18.454195 14220 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:18.455660 14176 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:18.456167 14154 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:17.899371 14150 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.460363  8572 catalog_manager.cc:5697] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 5 to 8, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 8 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:18.467075  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 290883us
I20260709 13:04:18.472580  8824 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 13:04:18.431234 14160 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.021s	sys 0.012s
W20260709 13:04:18.462580 14068 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.735s	user 0.007s	sys 0.001s
I20260709 13:04:18.493973 14034 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:18.274333 14120 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:18.497388  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 393586us
I20260709 13:04:18.475533 14220 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.474864 14207 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:18.520357 13370 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.838s	user 0.013s	sys 0.001s
I20260709 13:04:18.498306 14176 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.529173  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:18.562220  8824 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
W20260709 13:04:18.536113  9355 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.040s	sys 0.083s
I20260709 13:04:18.434288 14161 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.480214 14208 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.574536 14161 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.574770 14208 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.576830 14161 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:18.442423 14106 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:18.578751 14135 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:18.448733 14096 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.086s
W20260709 13:04:18.521019 14160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.022s	sys 0.012s
I20260709 13:04:18.519181  8822 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:18.504709 14131 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.511047 14152 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:04:18.565088  9355 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.040s	sys 0.083s
I20260709 13:04:18.301373 14205 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.568033 14111 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:18.568601  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2266) took 1673 ms. Trace:
I20260709 13:04:18.569687  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2185) took 1127 ms. Trace:
I20260709 13:04:18.573745 13847 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.530041 14201 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:18.582525  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:18.535337 14154 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.535542 14184 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:18.596693  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:18.566452 14214 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:18.610026  9355 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:18.544948 14220 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:18.614347  8826 rpcz_store.cc:276] 0709 13:04:16.894791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.901273 (+  6482us) service_pool.cc:224] Handling call
0709 13:04:17.490859 (+589586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.491640 (+   781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.568340 (+1076700us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b5400126e50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:18.568528 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1354966784}
I20260709 13:04:18.614537  8828 rpcz_store.cc:276] 0709 13:04:17.441926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.442100 (+   174us) service_pool.cc:224] Handling call
0709 13:04:17.442542 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:18.420435 (+977893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.569383 (+148948us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b540016bf54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:18.569617 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993444736}
I20260709 13:04:18.625090  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:18.626283  8824 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:18.602690 14161 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.562273 14213 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:18.629940  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:18.636493  8647 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.694s (SENT) [suppressed 19 similar messages]
I20260709 13:04:18.579533 14135 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.629062 13847 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.614514 14195 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:18.638424  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.639127 14120 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:18.583487 14176 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.543149 14068 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.650552  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:18.686436  8828 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:18.687975  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:18.627274  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:18.660053  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:18.658321  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:18.653847  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:18.545536 14034 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.612452 14223 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:18.672889  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.693279  9380 raft_consensus.cc:2410] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
W20260709 13:04:18.693662  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.659657 14222 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.683344  9388 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 9.
I20260709 13:04:18.691046  8822 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:18.663537 13698 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:18.693848  8827 raft_consensus.cc:2393] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 9: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:04:18.694501  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.625839 14131 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.699069  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:18.700853  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 64378.1us
I20260709 13:04:18.707844  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:18.712539  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.692349 14014 consensus_meta.cc:287] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.257s	user 0.009s	sys 0.007s
I20260709 13:04:18.715000  8824 raft_consensus.cc:2393] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:18.715364  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:18.718784  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:18.719441 14150 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:18.686318 14195 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.723536  9389 raft_consensus.cc:2368] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15.
I20260709 13:04:18.724071  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:18.727963  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:18.722590  8828 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:04:18.734858  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:18.735037  9355 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:18.735641  8828 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:18.644867 14201 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.646899 14154 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:18.700743 14174 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.027s	user 0.015s	sys 0.000s
I20260709 13:04:18.739833  8822 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:18.666950 14161 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:18.720037 14229 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:04:18.748369  8828 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [16.9-18.10]
I20260709 13:04:18.748548  8822 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:18.755558  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2342) took 1588 ms. Trace:
I20260709 13:04:18.727391 14014 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.756232  8753 rpcz_store.cc:276] 0709 13:04:17.166986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.167120 (+   134us) service_pool.cc:224] Handling call
0709 13:04:17.167398 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.167644 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:17.204623 (+ 36979us) log.cc:844] Serialized 14 byte log entry
0709 13:04:18.754996 (+1550373us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b180048b080. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e20a14bc 00007f32df3ef48b 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287
0709 13:04:18.755117 (+   121us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:18.755187 (+    70us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:18.755259 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.755338 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:18.755486 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40699,"prepare.queue_time_us":122620,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":6075,"raft.queue_time_us":52090,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":8681,"spinlock_wait_cycles":15394688,"thread_start_us":159424,"threads_started":3,"wal-append.queue_time_us":222678,"wal-append.run_cpu_time_us":2599,"wal-append.run_wall_time_us":1254191}
I20260709 13:04:18.719985  8572 catalog_manager.cc:5697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 10 to 13, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 13 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:18.759559  8572 catalog_manager.cc:5697] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:04:18.762408 14216 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:04:18.761700  8572 catalog_manager.cc:5697] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:18.764495  8572 catalog_manager.cc:5697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:04:18.765808 14201 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:18.742213 13847 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:18.766114  8572 catalog_manager.cc:5697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:04:18.766966  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:18.659039 14214 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.749676 14174 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.770921 14214 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:18.771556 14214 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:18.770558  8572 catalog_manager.cc:5697] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 9 to 11, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 11 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:18.772042 14214 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.706451 14223 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.773234  8572 catalog_manager.cc:5697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 11, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 11 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:18.768191 14034 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:18.776062  8572 catalog_manager.cc:5697] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 9 to 10, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 10 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:04:18.777756 14176 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:18.749118 14195 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.778241  8572 catalog_manager.cc:5697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:18.721045 13698 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:18.780521 14131 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:18.780278 13698 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.671763 14213 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.658811 14184 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.780121  8572 catalog_manager.cc:5697] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 11, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 11 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:18.712499 14222 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.783592  8572 catalog_manager.cc:5697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 10 to 13, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 13 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:18.786844 14129 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:18.787838 14068 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:18.770925 14174 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:18.767642 14135 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:18.794139  8828 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:18.758265  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:18.785211 14170 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.198s	user 0.011s	sys 0.000s
I20260709 13:04:18.803650  8827 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:18.808161  8647 leader_election.cc:400] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15.
I20260709 13:04:18.808666  8647 leader_election.cc:403] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:04:18.798285 14221 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:18.813540 14221 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.813983 14221 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:18.806589 14170 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:18.797382 14167 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.135s	user 0.014s	sys 0.000s
I20260709 13:04:18.823549 14167 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.806985 14068 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:18.824484 14124 raft_consensus.cc:3055] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:18.824872 14068 raft_consensus.cc:697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:18.826095 14167 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:18.820416  8825 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:04:18.823834 14223 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:18.828030 14139 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.032s	user 0.016s	sys 0.001s
I20260709 13:04:18.844756 14139 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.828228 14177 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:18.826053 14068 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:18.837008  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:18.841741  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.824934 14124 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:18.862295 14124 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.863746 14124 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:18.822762 14186 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.117s	user 0.009s	sys 0.013s
I20260709 13:04:18.843612 14131 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:18.871035 14131 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.874073 14131 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:18.848615  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 140828us
I20260709 13:04:18.858529  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:18.877759  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:18.877900  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:18.878871  8822 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:18.880398  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.896297  8750 raft_consensus.cc:2410] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:04:18.898756  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:18.847162 14174 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:18.902369  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.901780  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:18.857591 14214 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:18.899736 14084 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.900131 14167 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:18.903645 14024 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.918829  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:18.917622  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:18.920156  8824 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:18.928640  8822 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:18.937542  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:18.923774 14214 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:18.920887 14024 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.922328 14236 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:18.919049 14084 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.927433 14139 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:18.943677 14233 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:18.952548  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:18.938452  8822 raft_consensus.cc:2376] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19. Re-sending same reply.
I20260709 13:04:18.969367 14169 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:18.970523  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:18.972414 14139 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:18.972849 14139 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:18.973251 14139 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:18.975229 14233 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:18.979530  8824 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:18.986078  8822 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:19.032579  8824 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:04:19.038010 14169 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:18.980970 14236 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.987108 14167 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.995067 14152 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.033174 14124 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.035848 14223 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.035362 14170 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:18.978940 14246 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
W20260709 13:04:19.062361  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.075232  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.045538 14169 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.054230 14236 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:19.078704 14250 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:04:19.083940 14169 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:19.084486 14236 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:19.089267 14161 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:19.091496  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2350) took 1897 ms. Trace:
I20260709 13:04:19.117403  8752 rpcz_store.cc:276] 0709 13:04:17.193750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.565807 (+372057us) service_pool.cc:224] Handling call
0709 13:04:17.566318 (+   511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.680416 (+114098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.091115 (+1410699us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b540008ff54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:19.091422 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260343040}
I20260709 13:04:19.092545  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2336) took 1729 ms. Trace:
I20260709 13:04:19.055102 14167 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:19.118791  8678 rpcz_store.cc:276] 0709 13:04:17.363366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.363527 (+   161us) service_pool.cc:224] Handling call
0709 13:04:17.363917 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.050424 (+686507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.092237 (+1041813us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b54004303d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:19.092485 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1577014016}
I20260709 13:04:19.120402  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:19.121248  8678 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 14: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:19.122208  9385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 801) took 2362 ms. Trace:
I20260709 13:04:19.123349 14016 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:19.095993  8825 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:04:19.126127  8825 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:04:19.099453 14024 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:19.103471 14245 log.cc:927] Time spent T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:04:19.104771 14247 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:04:19.129846 14167 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:19.130566  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:19.108147  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:19.133213  8674 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:19.133720  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:19.109019  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:19.137099  8681 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
W20260709 13:04:19.109670 14240 log.cc:927] Time spent T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:04:19.110348 14177 raft_consensus.cc:3055] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:19.110889  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:19.154031  8680 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:19.076967 14223 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.099517  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2347) took 1906 ms. Trace:
I20260709 13:04:19.058540 14152 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:19.113888  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:19.157021  8754 rpcz_store.cc:276] 0709 13:04:17.192989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.339832 (+146843us) service_pool.cc:224] Handling call
0709 13:04:19.099251 (+1759419us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b540031cc54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:19.099443 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423291136}
I20260709 13:04:19.159979  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.122340  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:19.162086  8678 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:19.122758  9385 rpcz_store.cc:276] 0709 13:04:16.759564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.759724 (+   160us) service_pool.cc:224] Handling call
0709 13:04:16.772006 (+ 12282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:16.972059 (+200053us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:17.194429 (+222370us) log.cc:844] Serialized 14 byte log entry
0709 13:04:17.315072 (+120643us) log.cc:844] Serialized 14 byte log entry
0709 13:04:17.315252 (+   180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:17.315301 (+    49us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:17.315345 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.570775 (+1255430us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b18003e3b00. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f
0709 13:04:18.572085 (+  1310us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b48000cc3d8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:04:19.121939 (+549854us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54001ccd54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:19.121991 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.122115 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":117725,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":12250,"raft.queue_time_us":8320,"raft.run_cpu_time_us":18917,"raft.run_wall_time_us":1261921,"spinlock_wait_cycles":511999616,"thread_start_us":333502,"threads_started":2,"wal-append.queue_time_us":481962,"wal-append.run_cpu_time_us":1487,"wal-append.run_wall_time_us":1001490}
I20260709 13:04:19.158524  9388 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:04:19.166000  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:19.166932  8681 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:19.167650  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.168283  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:19.168406  8753 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:19.169170  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.169785  8752 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:19.169932  8754 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:19.172490  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:19.169500 13849 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:04:19.173014 14233 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:19.177556  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.171144 13472 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:19.180157 13472 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.168284  8674 raft_consensus.cc:2410] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:04:19.183398  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.072866 14124 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.184952  8754 raft_consensus.cc:2410] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:04:19.126194 14053 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.199728  8752 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:19.204747  8756 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:04:19.156880 14177 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:19.206712  8824 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:19.183647 13472 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:19.205755 14220 consensus_meta.cc:287] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.003s
I20260709 13:04:19.195726 14016 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.220341  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:19.222564  8754 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:04:19.223013  8752 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:19.220906  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.226080  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.226917  8752 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 17: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:19.223754  8751 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:19.229133  8752 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:04:19.230517  8755 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:19.227494 14205 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:19.243774 14177 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:19.248790 14223 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:19.226809 14159 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:19.228093 14068 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:19.250937  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:19.236536  8753 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:04:19.253278  8753 pending_rounds.cc:85] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 8
I20260709 13:04:19.253518  8753 pending_rounds.cc:107] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:04:19.253749  8753 raft_consensus.cc:2889] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:19.205668 14161 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:04:19.221813 14220 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:19.260653 14220 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:19.262913  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:19.266070 14062 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:19.266665  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:19.268174  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:19.211186 14053 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.247941 14221 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.263171 14124 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:19.233022 14174 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15.
I20260709 13:04:19.271517  8826 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:19.285676  8826 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:19.251060 14159 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.280154  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1074) took 2398 ms. Trace:
I20260709 13:04:19.286408  8823 rpcz_store.cc:276] 0709 13:04:16.881679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.881847 (+   168us) service_pool.cc:224] Handling call
0709 13:04:16.882218 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:16.934117 (+ 51899us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:16.934173 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:16.934239 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:16.934298 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.279837 (+2345539us) spinlock_profiling.cc:243] Waited 51.5 ms on lock 0x7b540016b554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:19.279895 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.280051 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118521216}
I20260709 13:04:19.287093  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:19.287909  9382 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:19.290429  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:19.282135  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:19.292778  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:19.293303 14208 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:19.294250 14159 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:19.260720 14084 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:19.297250 14173 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:19.299095  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.299028  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:19.307502  9389 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:19.221951 14016 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.301630  8826 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:19.323391  8826 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:19.305271  9382 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:19.308301  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:19.331326  8823 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:19.309485  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:19.348802  8821 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:19.355567  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:19.357052  8754 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:19.271090 14068 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:19.376356  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:19.378818  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:55168 (request call id 1072) took 2528 ms. Trace:
I20260709 13:04:19.379338  8821 rpcz_store.cc:276] 0709 13:04:16.821387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.822418 (+  1031us) service_pool.cc:224] Handling call
0709 13:04:19.348015 (+2525597us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b4c00065d40. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e07c5a20 00007f32e07d6a29 00007f32e07d713f 00007f32e079b2ab 00007f32e07c8f3c 00007f32e07c93fe 00007f32e0a5688b 00007f32e2455495 00007f32e24a630f 00007f32e24f0a55 00007f32e74c0cd0
0709 13:04:19.349401 (+  1386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79221120}
I20260709 13:04:19.326676 14208 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:19.373245  8747 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 8.
I20260709 13:04:19.382772  8676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:55714
I20260709 13:04:19.383514  8676 raft_consensus.cc:606] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:19.384088  8676 raft_consensus.cc:614] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:19.382112  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:19.385721 14131 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:19.386268 14131 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.387007 14131 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:19.391499  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:19.384351 14099 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:19.422353 14099 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:19.422976 14099 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:19.388868 14174 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:04:19.391100 14014 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:19.387887 14176 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:19.381206 14139 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.453182  8752 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:19.454293 14139 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:19.389546 14208 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:19.455430  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.456645  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.457472  8752 raft_consensus.cc:2393] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:19.405711  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:19.410825 14221 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:19.412736  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.412736  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:19.459785  8751 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:19.459903  8825 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:19.413426 13585 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:19.415521  8829 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:04:19.464602  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 73285.6us
W20260709 13:04:19.416251  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.420181  9385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:19.382438 14164 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.465425  9385 raft_consensus.cc:606] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:19.465476  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:19.465966  9385 raft_consensus.cc:614] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:19.433357  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:19.436869  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:19.466738  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.439215  8755 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 13:04:19.467377  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.466794  8753 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:19.440451  8826 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:04:19.468426  8826 pending_rounds.cc:85] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 8
I20260709 13:04:19.468925  8826 pending_rounds.cc:107] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:04:19.446746 14176 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:19.448478  8748 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:19.449673 14016 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:19.452003 14068 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:19.458204  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.463426  8680 raft_consensus.cc:2410] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:04:19.460860  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:19.478351  8645 leader_election.cc:336] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.691s (SENT)
W20260709 13:04:19.468461  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.478957  8645 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:19.441428  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:19.442869 14261 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:04:19.469404  8826 raft_consensus.cc:2889] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:19.469668  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 64035.2us
I20260709 13:04:19.471675  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.474437  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:19.485469  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.486794  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.489677  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:19.491322  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.481545 14208 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:19.492908 14208 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:19.493423 14208 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.493817 14208 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:19.494879 14185 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:19.462168 14053 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:19.503773  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.505673  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:19.460862 14204 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.516s	user 0.017s	sys 0.009s
I20260709 13:04:19.480376  8754 raft_consensus.cc:2410] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:19.511771  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.475203 13849 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:19.514752  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.473717 14176 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:19.509850 14168 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 13:04:19.527536  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 61757.1us
W20260709 13:04:19.515853 14204 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.658s	user 0.018s	sys 0.009s
I20260709 13:04:19.487723 14016 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:19.527750  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.531787  8754 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:04:19.532423  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.539120  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:19.514143 13878 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:04:19.542243 13878 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:19.542764 13878 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:19.543744 13878 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.545271 13878 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:19.473564 14164 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:19.535018  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.479852 13698 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:04:19.524268 13849 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:19.524433 14120 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:04:19.550132 13849 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:19.553845 14182 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:19.555755  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:19.557430  8747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2345) took 2365 ms (client timeout 2127 ms). Trace:
I20260709 13:04:19.510294 14221 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:04:19.557282  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:19.558382  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.558668  8755 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:19.561640 14095 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:19.525394 14265 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:04:19.567883  8747 rpcz_store.cc:269] 0709 13:04:17.192372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.192512 (+   140us) service_pool.cc:224] Handling call
0709 13:04:19.557235 (+2364723us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b540034bf54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:19.557358 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4877800576}
W20260709 13:04:19.571733 14248 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.604s	user 0.003s	sys 0.000s
I20260709 13:04:19.572570  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:19.572724  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:19.530226  8647 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:19.547950  8753 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
W20260709 13:04:19.588522  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 58527.2us
I20260709 13:04:19.590363  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:19.591303  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:19.591287  8748 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
W20260709 13:04:19.597411  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 99586.1us
I20260709 13:04:19.583120  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:19.592602  8680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35e078746b204711a1e15f221c16e31a"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:55714
I20260709 13:04:19.602078  8825 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
I20260709 13:04:19.604197  8680 raft_consensus.cc:606] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:19.604614  8680 raft_consensus.cc:614] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:19.593379 14174 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:19.607661  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 129335us
I20260709 13:04:19.581406 14016 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:19.539098 14204 raft_consensus.cc:697] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:19.617844  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:19.618429  8755 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:19.581936 14234 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:19.621030  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:19.607055 13927 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:19.624607 14099 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:19.632051 14151 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:19.633803  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1086) took 1004 ms. Trace:
I20260709 13:04:19.640564  8824 rpcz_store.cc:276] 0709 13:04:18.629371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.072631 (+443260us) service_pool.cc:224] Handling call
0709 13:04:19.074468 (+  1837us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.212293 (+137825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.633308 (+421015us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54002689d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:19.633691 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302963456}
I20260709 13:04:19.635571 14268 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:19.642426 14268 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:19.643544 14268 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:19.643759 14016 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:19.636489 14244 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.691s	user 0.002s	sys 0.000s
W20260709 13:04:19.641798  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.645835  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.648711  8645 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:19.655383  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:19.656312 13878 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:19.621151 14174 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:19.651978 13927 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:19.664906 14014 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:19.665870 14168 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:04:19.668251  8750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2346) took 2475 ms (client timeout 2158 ms). Trace:
W20260709 13:04:19.668875  8750 rpcz_store.cc:269] 0709 13:04:17.192670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.192800 (+   130us) service_pool.cc:224] Handling call
0709 13:04:19.667942 (+2475142us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b540034ab54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:19.668147 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3842257664}
I20260709 13:04:19.670578  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:19.672250 14152 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:19.672374 14014 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:19.674649  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:19.675593 14014 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:19.677699  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:19.648557 14267 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:19.681062  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:19.687765  8794 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:19.691419  8677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:55714
I20260709 13:04:19.692306  8677 raft_consensus.cc:606] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:19.692688  8677 raft_consensus.cc:614] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:19.695096 14267 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:19.694996 14204 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.703730 14184 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:19.649614 14102 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:04:19.716687 14102 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:19.717172 14102 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:19.705224 14234 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:19.719064 14174 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:19.728259 14266 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:19.689116 13927 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:19.693588 14168 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:04:19.770973  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.771703  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.735886 14014 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:19.718180 14102 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.745939 14168 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:19.784897 13927 raft_consensus.cc:697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:19.794770 14168 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:19.806026 13927 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.795984 14014 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:19.813263 14014 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:19.815156  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:19.816143  9383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 728) took 1218 ms. Trace:
I20260709 13:04:19.816483  9383 rpcz_store.cc:276] 0709 13:04:18.597755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.597903 (+   148us) service_pool.cc:224] Handling call
0709 13:04:19.815019 (+1217116us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:04:19.816049 (+  1030us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5079424}
I20260709 13:04:19.820109 14132 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.821002 14132 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:19.809912  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 160979us
I20260709 13:04:19.824478  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:19.824771  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2204) took 1394 ms. Trace:
I20260709 13:04:19.825229  8828 rpcz_store.cc:276] 0709 13:04:18.430607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.750113 (+319506us) service_pool.cc:224] Handling call
0709 13:04:18.750477 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.800264 (+ 49787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.824480 (+1024216us) spinlock_profiling.cc:243] Waited 42.8 ms on lock 0x7b54003dd6d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:19.824698 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98530176}
I20260709 13:04:19.786917 14176 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.802459 14195 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:19.832202 14208 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:19.835819 14164 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:19.837234 14176 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:19.809978 14102 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:19.842765  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.829784 14131 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:19.849690 14176 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:19.846287 13849 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:19.845978  8749 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:19.898509  9381 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:19.860041  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 201288us
I20260709 13:04:19.827898 14095 consensus_queue.cc:1059] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:04:19.853372  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:19.863013  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 239442us
I20260709 13:04:19.908595  8756 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:19.878892  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.912710  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 59473.4us
W20260709 13:04:19.914608  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.850189 14131 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:19.842896 14208 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:19.852298  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 723) took 2849 ms. Trace:
I20260709 13:04:19.844492 14055 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:04:19.897823  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 249144us
W20260709 13:04:19.921931  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.922679  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:19.924633  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:19.925518  8825 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 17: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:19.920863  9384 rpcz_store.cc:276] 0709 13:04:17.003060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.003532 (+   472us) service_pool.cc:224] Handling call
0709 13:04:17.021025 (+ 17493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.558944 (+537919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.851929 (+2292985us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b540061c1d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:19.852206 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1200387840}
I20260709 13:04:19.928897  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:19.924072  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:19.929754  8829 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:19.930035  8824 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:19.940879  8822 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:19.915577 13726 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:19.915809 14174 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:19.945863  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:19.949813  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:19.951205  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:19.952705  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:19.953685  8825 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:19.947285  8829 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:19.926239 14055 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:19.944514 14131 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:19.919267 14266 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:04:19.948763 14174 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.977012  8825 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
W20260709 13:04:19.999114  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.972324  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:20.006419  8822 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:20.020414  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.009768 14095 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.002985 14174 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:20.035725  9380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 730) took 1383 ms (client timeout 1820 ms). Trace:
W20260709 13:04:20.036417  9380 rpcz_store.cc:269] 0709 13:04:18.653634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.656715 (+  3081us) service_pool.cc:224] Handling call
0709 13:04:20.035447 (+1378732us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b5400469954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:20.035646 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7470336}
I20260709 13:04:20.003885 14176 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:20.015198 14276 raft_consensus.cc:3055] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:20.007246 14140 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:20.040191 13927 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.025064 14095 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:20.051420 13927 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.047678 14140 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:20.057595 14140 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.056246  9387 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:20.058894 14140 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:20.059243  9387 raft_consensus.cc:606] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Received request to transfer leadership to TS 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:20.042395 14276 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:04:20.051203 14284 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.171s	user 0.000s	sys 0.002s
I20260709 13:04:20.064363  8822 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:20.043617 14176 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:20.070318  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 144840us
I20260709 13:04:20.063297 13927 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:20.077421  8645 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:20.078651 14291 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:20.079478 14291 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.080911  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 984) took 1983 ms. Trace:
I20260709 13:04:20.081436  8679 rpcz_store.cc:276] 0709 13:04:18.097030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.218206 (+121176us) service_pool.cc:224] Handling call
0709 13:04:18.218861 (+   655us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:18.305759 (+ 86898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:18.305811 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:18.432591 (+126780us) log.cc:844] Serialized 26 byte log entry
0709 13:04:19.669121 (+1236530us) spinlock_profiling.cc:243] Waited 8.92 ms on lock 0x7b1800409900. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:04:19.914506 (+245385us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:19.914570 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.914796 (+   226us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.080339 (+165543us) raft_consensus.cc:1685] finished
0709 13:04:20.080372 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.080790 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91112,"raft.queue_time_us":201457,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":9523,"spinlock_wait_cycles":20570112,"thread_start_us":245132,"threads_started":1,"wal-append.queue_time_us":1608531}
I20260709 13:04:20.083094 14293 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.083715 14293 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.086769 14293 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:20.082396 14291 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:20.089537 14095 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:20.092288  8645 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:20.070176 13849 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:20.096422 14276 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:20.098028 14276 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:20.075230 14131 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.099453 14131 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.100157 14168 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.101915 14131 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:20.075462 14266 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.074007 14222 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:20.053198 14055 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.102823 14266 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.103574 14055 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:20.105321 14266 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:20.108467 14176 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:20.110221  8678 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:04:20.111023  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2721) took 1008 ms. Trace:
I20260709 13:04:20.111333  8678 rpcz_store.cc:276] 0709 13:04:19.102510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.164437 (+ 61927us) service_pool.cc:224] Handling call
0709 13:04:20.109652 (+945215us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:04:20.109900 (+   248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:20.110713 (+   813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.110960 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4664192}
I20260709 13:04:20.117122  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2711) took 2485 ms. Trace:
I20260709 13:04:20.094408 14095 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:20.118819 14168 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:20.117672  8673 rpcz_store.cc:276] 0709 13:04:17.631811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.639145 (+  7334us) service_pool.cc:224] Handling call
0709 13:04:17.639599 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:17.725991 (+ 86392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:17.726027 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:17.726575 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 13:04:18.792652 (+1066077us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b18003d9780. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:04:18.914669 (+122017us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:18.980264 (+ 65595us) log.cc:844] Serialized 14 byte log entry
0709 13:04:18.980526 (+   262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.116656 (+1136130us) spinlock_profiling.cc:243] Waited 66.3 ms on lock 0x7b54001ff9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:20.116721 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.116780 (+    59us) raft_consensus.cc:1685] finished
0709 13:04:20.116797 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.117009 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19570,"prepare.queue_time_us":69712,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":59,"raft.queue_time_us":19939,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":8880,"spinlock_wait_cycles":205502464,"thread_start_us":180323,"threads_started":2,"wal-append.queue_time_us":1218640}
W20260709 13:04:20.121862  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:20.124948  8644 connection.cc:256] RPC call timeout handler was delayed by -1.572109724000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:04:20.125914 13726 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.127832  8678 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:20.126541  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.128916  8673 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:20.144634  8679 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 10. Current term is 11. Ops: []
W20260709 13:04:20.144973  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 48875.3us
I20260709 13:04:20.147315  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:20.147703  8672 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:20.148893  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:20.150276  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.151798  8676 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:20.151301  8674 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:20.154158  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:20.155018  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:20.156066  8672 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:20.156984  8675 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:20.158361  8673 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:20.158838  8677 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:04:20.159282  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:20.159965  8678 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:20.160449  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.169500 14126 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:20.174778  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.177119  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:20.126482 13726 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.094559 14016 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:04:20.137252 14291 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:20.186690  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.156417 14265 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:20.161254 14095 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:20.190007  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:20.191056  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:20.191677  8680 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:20.124969 14236 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:04:20.166113 14266 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:20.176285 14293 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:20.208042  8673 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:20.209551  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.208732  8678 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:20.189132 14016 consensus_queue.cc:1243] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:20.212401  8679 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:20.187319 14291 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:20.215327  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.214857  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.211683 14095 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:20.211102  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.269068  8676 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 6.
I20260709 13:04:20.276073  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.276654  8678 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:20.269024 14291 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:20.228085 14154 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:20.280014  8676 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 17: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:20.233485 14177 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:20.244701 13726 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:20.283562  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 106552us
I20260709 13:04:20.221186 14265 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:04:20.287088  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.288892  8676 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:20.289956  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.291052  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.291759  8676 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:04:20.295322  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.296178  8676 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:04:20.216032 14293 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:20.298416  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.300144  8676 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:20.297147  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:20.301080 14174 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.303704  8674 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:20.299400  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:20.304720  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:20.305308  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.305369  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.305848  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:20.306286  8676 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:20.304664  8678 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:20.301906  8680 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:20.314173 14276 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:20.275951 14095 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:20.354355  8675 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:20.356302  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.369223  9355 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:20.370051  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:20.358901  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.359141  8678 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:20.359478  8677 raft_consensus.cc:2410] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 16 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:04:20.307874  8680 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:20.360013  8676 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:20.358924  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 61864.4us
I20260709 13:04:20.377333  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:20.377413  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:20.308540 14296 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:20.385569  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:20.390898  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:20.391307  8678 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:20.391760  8821 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:20.354221 14106 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.390789  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:20.390537  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:20.359964 14168 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:20.397253  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:20.372702 14154 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.394273 14014 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.399042 14106 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.403848 14184 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:20.405181 14184 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:20.405617 14184 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:20.406698  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.385640 14145 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:20.401351  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:20.403829 14154 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.397363  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:20.409289  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:20.400154  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.402526  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:20.412446  8678 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:20.394198 14204 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:20.414358 14167 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:20.414577 14204 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.414978 14123 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:20.414742 14095 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:20.416503 14174 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:20.420193  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:20.411763 14286 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:20.424017  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.423774 14286 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.427057 14276 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:20.427630 14276 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:20.428052 14276 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:20.428927  8681 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:04:20.429742  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:20.430207  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.421597 14129 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.431603 14129 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.432471 14286 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:20.433504 14300 consensus_queue.cc:1059] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:20.434485 14300 raft_consensus.cc:3055] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:20.431946 14140 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:20.435029 14184 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:20.436048 14140 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:20.436446  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:20.436942 14140 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.437671 14184 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:20.438166 14140 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:20.435827 14300 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:04:20.430526  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.385905 13973 consensus_queue.cc:1059] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
W20260709 13:04:20.445892  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.414580  8719 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:20.434785  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:20.448148  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.453166  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:20.430471  8681 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:04:20.460253  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:20.461200  8756 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:20.462159  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:20.462406  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:20.462972  8754 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:20.466722 14213 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:20.418143 14236 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:04:20.462939  8821 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:20.470458  8750 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:04:20.471225  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 56740.8us
I20260709 13:04:20.443140  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:20.476689  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:20.477900  8750 raft_consensus.cc:2410] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:04:20.413054 14145 raft_consensus.cc:697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:20.474169  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:20.485216  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:20.483479  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:20.488467  8821 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:20.483865  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:20.489060  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:20.489681  8753 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:20.485961  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:20.495826  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:20.469825 14016 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:20.476629 14278 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:20.470109 14222 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:20.496660 14095 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:20.507278 14278 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:20.507949 14095 raft_consensus.cc:697] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:20.507452 14222 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:20.508177 13726 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:20.508586 14278 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.508952 14095 consensus_queue.cc:237] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.477455 14055 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:04:20.511451 14123 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:20.408430 14168 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.508644 13726 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.519843  8748 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
W20260709 13:04:20.503497 14297 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:04:20.478026 14213 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:20.510313 14222 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:20.501663  8681 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:20.404079 14014 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.496519 14172 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:20.531776 13726 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:20.536098 14300 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:20.538723 14168 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:20.544968 14291 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:20.537016  8673 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:20.512428 14154 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:20.495143 14129 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:20.499869 14293 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:20.500236 14174 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:20.484503 14184 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:04:20.641790 14132 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:20.649984 14106 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:20.653861 14204 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:20.660159  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:20.661237  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:20.636860 14172 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:20.663105  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:20.666981  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:20.704926  8673 raft_consensus.cc:2376] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11. Re-sending same reply.
W20260709 13:04:20.676887  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 243619us
W20260709 13:04:20.678987  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.682121  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:20.512665 14145 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:20.687887  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:20.601040 14213 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:20.698012  8680 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:20.710913  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:20.662143  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:20.632128 14299 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.717001 14172 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.703158 14300 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:20.700601 14174 raft_consensus.cc:697] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:20.738001  9382 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:20.708614 14184 consensus_queue.cc:1243] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:04:20.745201  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 259295us
I20260709 13:04:20.747308  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:20.714203 14213 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:20.720965 14299 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.760113  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:20.950980  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 203741us
I20260709 13:04:20.951485  8748 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:20.760874  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:20.952819  8821 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:20.728451 14172 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:20.761152  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:20.957738  8754 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:20.763826  8794 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:20.748462 14126 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.775607  8824 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:20.776490  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:20.960189  8753 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:20.779605  8678 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 13:04:20.961360  8678 pending_rounds.cc:85] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 13
I20260709 13:04:20.961671  8678 pending_rounds.cc:107] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 13:04:20.961979  8678 raft_consensus.cc:2889] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:20.781265  8751 raft_consensus.cc:2410] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:04:20.931967 14291 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:20.873261  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.771430 14102 consensus_queue.cc:1059] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 13:04:20.899039  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:20.905575  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 411160us
I20260709 13:04:20.909132 14014 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:20.893013 14174 consensus_queue.cc:237] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:20.943990  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:20.946717  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 287604us
W20260709 13:04:20.949291  8827 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2281) took 2471 ms (client timeout 2247 ms). Trace:
W20260709 13:04:20.871518  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.033712  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:21.033454  8827 rpcz_store.cc:269] 0709 13:04:18.472502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.720489 (+247987us) service_pool.cc:224] Handling call
0709 13:04:20.943892 (+2223403us) spinlock_profiling.cc:243] Waited 50.5 ms on lock 0x7b54002292d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:20.944047 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116145536}
I20260709 13:04:21.036442  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:21.039433  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 67715.4us
I20260709 13:04:21.038261  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:21.041402  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:21.041755  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:21.042346  8827 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:21.045159  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:20.969139  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:20.946310 14295 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.000s	sys 0.017s
W20260709 13:04:21.078634  8823 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.007s	sys 0.041s
W20260709 13:04:20.971943  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.031s	sys 0.032s
W20260709 13:04:21.079084  8823 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.007s	sys 0.042s
I20260709 13:04:20.973987  9381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 725) took 3967 ms. Trace:
I20260709 13:04:21.079274  8827 raft_consensus.cc:2393] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:04:21.029263  8749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2364) took 3129 ms (client timeout 1691 ms). Trace:
I20260709 13:04:20.958253  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:21.028416  8821 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
W20260709 13:04:21.080832  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 122708us
I20260709 13:04:21.081875  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:21.008605 14232 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 2.245s	user 0.020s	sys 0.004s
I20260709 13:04:21.079555  9381 rpcz_store.cc:276] 0709 13:04:17.006703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.008360 (+  1657us) service_pool.cc:224] Handling call
0709 13:04:17.008788 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.905924 (+2897136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.973606 (+1067682us) spinlock_profiling.cc:243] Waited 2.89 s on lock 0x7b54000762d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:20.973893 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6644735744}
W20260709 13:04:21.080317  8749 rpcz_store.cc:269] 0709 13:04:17.899994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.900145 (+   151us) service_pool.cc:224] Handling call
0709 13:04:21.028996 (+3128851us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54002fe9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:21.029173 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4111362304}
W20260709 13:04:21.079274  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.031s	sys 0.032s
W20260709 13:04:21.086668  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.099598  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 935457us
W20260709 13:04:21.088752  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.100356  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:21.096203  8755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.016s	sys 0.017s
I20260709 13:04:21.101691  8645 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:21.081595  8719 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:21.104343  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:21.101599  8755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.016s	sys 0.017s
W20260709 13:04:21.088129  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 56425.4us
I20260709 13:04:20.961375 14314 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:21.109694  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:21.111023 13370 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:21.113498  8821 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:21.114729  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:21.115477  8821 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:21.114856 13370 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:21.116314  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:21.116633  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:21.116531 13370 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:04:21.001972 14276 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.001s
W20260709 13:04:21.111843  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 383514us
I20260709 13:04:21.021098 14299 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:21.079018 14295 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.000s	sys 0.018s
I20260709 13:04:21.114881  8719 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:21.118050  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:21.119778  8645 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:21.147105  8719 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:21.147665  8821 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
I20260709 13:04:21.148886  8821 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:21.061914 14068 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:21.121589  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:21.150970  8827 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:21.151098  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:21.153611  8821 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:21.153761  8827 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:21.118527 13370 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:21.123328  9384 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 9. Current term is 10. Ops: []
W20260709 13:04:21.142557  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 426984us
I20260709 13:04:20.973016 14126 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.160653  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:21.120147  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.170604  8821 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:21.172147  8821 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:21.168567  8828 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:21.103370 14313 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:21.173107  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2292) took 1255 ms. Trace:
I20260709 13:04:21.124698 14291 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:21.244551  8828 rpcz_store.cc:276] 0709 13:04:19.918185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.918363 (+   178us) service_pool.cc:224] Handling call
0709 13:04:21.160402 (+1242039us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:04:21.173045 (+ 12643us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2713984}
W20260709 13:04:21.229537  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.177630  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 57969.6us
I20260709 13:04:21.175961  8827 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:21.178020  8824 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:04:21.260361  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:21.142433 14300 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:21.173637  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:21.279493  9381 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:21.159399 14314 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:21.279623  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 106096us
I20260709 13:04:21.158550 14184 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:21.289410  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:21.289366  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 248874us
I20260709 13:04:21.149822 14068 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:21.288661  8647 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:21.140034 14158 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.132s
I20260709 13:04:21.259232 14278 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:21.366711  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2199) took 2938 ms. Trace:
I20260709 13:04:21.324898 13370 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.176991 14299 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:21.373847 14299 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:21.376359 14299 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:21.376964 14299 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.377441 14299 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.293877 14267 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:21.379038 14267 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.379484 14267 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:21.370635  8830 rpcz_store.cc:276] 0709 13:04:18.428350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.428491 (+   141us) service_pool.cc:224] Handling call
0709 13:04:18.428893 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:18.443450 (+ 14557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:18.443498 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:18.462912 (+ 19414us) log.cc:844] Serialized 26 byte log entry
0709 13:04:18.598901 (+135989us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:18.598947 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.599025 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.944879 (+2345854us) spinlock_profiling.cc:243] Waited 9.58 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:21.322238 (+377359us) raft_consensus.cc:1685] finished
0709 13:04:21.322275 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.366600 (+ 44325us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":64466,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":109,"spinlock_wait_cycles":22043392,"thread_start_us":135351,"threads_started":1,"wal-append.queue_time_us":225644}
I20260709 13:04:21.296401 14184 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.383795  9388 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:21.384136 14184 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:21.387645 14068 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.156317 14276 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.404232  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:21.275141 14291 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.428452 13370 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:21.436816  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 277833us
I20260709 13:04:21.364511 14055 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:21.447531  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.432097 13726 consensus_meta.cc:287] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.748s	user 0.012s	sys 0.003s
I20260709 13:04:21.424888 14158 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:21.432327 14208 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:21.423177 14068 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:21.433537 14291 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:21.465252 13726 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.460088 14158 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:21.446244 14055 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.484320 13726 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:21.494691  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.460563 14314 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:21.506489 14172 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.000s
W20260709 13:04:21.508078  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 69741.9us
I20260709 13:04:21.511466 14158 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.513764  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1102) took 1140 ms. Trace:
I20260709 13:04:21.580821  8823 rpcz_store.cc:276] 0709 13:04:20.373314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.373482 (+   168us) service_pool.cc:224] Handling call
0709 13:04:20.373919 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:20.385510 (+ 11591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:20.385546 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:20.390466 (+  4920us) log.cc:844] Serialized 26 byte log entry
0709 13:04:21.082781 (+692315us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:21.082833 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.082920 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:21.513382 (+430462us) raft_consensus.cc:1685] finished
0709 13:04:21.513416 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.513667 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11974,"spinlock_wait_cycles":9190656,"thread_start_us":660913,"threads_started":1,"wal-append.queue_time_us":1047696}
I20260709 13:04:21.520247 14276 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:21.488071 14055 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:21.533151 14299 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:21.470311 14208 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:21.183851  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.597815  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 496814us
W20260709 13:04:21.598677  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.175562  8821 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:04:21.578563  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:21.569401 14313 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:21.604538  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.606057  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.610159  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 321594us
I20260709 13:04:21.614913  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:21.616169  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:21.616974  8679 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:21.608412  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:21.618453  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:21.604490 14208 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:21.618211 14172 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:21.609809 14158 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:21.627219  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:21.608211 14313 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:21.631431  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:21.631915 13370 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:21.636569 13370 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:21.637302  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:21.638200  8677 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:21.615620 14299 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.640843 13370 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:21.626319  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 479266us
I20260709 13:04:21.753931  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:21.757452  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.758213  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 120985us
I20260709 13:04:21.777917  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:21.760998  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:21.767251  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 158319us
I20260709 13:04:21.784155  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:21.785210  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:21.632247 14172 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:21.801738 14172 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:21.687192  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 108760us
I20260709 13:04:21.789088 13370 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:21.805766 13370 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:21.650517 14099 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:21.810014 14312 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:21.621677 14127 consensus_queue.cc:1059] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:04:21.759116  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:21.659199 14184 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.817783 14184 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:21.820138  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 988) took 1646 ms. Trace:
I20260709 13:04:21.705513 14208 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:21.663190 14213 consensus_meta.cc:287] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.691s	user 0.008s	sys 0.008s
I20260709 13:04:21.823547 14213 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:21.828696  8681 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:04:21.715998 14266 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.495s	user 0.014s	sys 0.026s
I20260709 13:04:21.635465 14276 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:21.830793 14266 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.610s	user 0.014s	sys 0.026s
I20260709 13:04:21.831391 14266 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:21.831121 14276 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:21.832777 14347 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.833616 14347 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:21.821400  8680 rpcz_store.cc:276] 0709 13:04:20.173196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.371790 (+198594us) service_pool.cc:224] Handling call
0709 13:04:20.391555 (+ 19765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:20.709675 (+318120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.819242 (+1109567us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b54003cf854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:21.819782 (+   540us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":704000896}
I20260709 13:04:21.835912 14213 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:21.836802  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:21.837153 14347 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:21.840236  9382 raft_consensus.cc:3055] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:21.677369 14267 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.648083 14299 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:21.715047 14268 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.469s	user 0.009s	sys 0.036s
W20260709 13:04:21.658921 14317 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:04:21.717965 14319 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.558s	user 0.004s	sys 0.007s
W20260709 13:04:21.719197 14323 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.114s	user 0.000s	sys 0.002s
I20260709 13:04:21.672091 14332 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:21.898947 14349 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.612885 14333 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:04:21.900298 14349 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:21.709106 14278 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:21.904242  8679 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:21.911746 14267 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:21.902417 14126 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:21.919323  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2301) took 1543 ms. Trace:
I20260709 13:04:21.921314  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:21.919977  8822 rpcz_store.cc:276] 0709 13:04:20.375619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.375745 (+   126us) service_pool.cc:224] Handling call
0709 13:04:20.376060 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:20.386410 (+ 10350us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:20.698882 (+312472us) log.cc:844] Serialized 14 byte log entry
0709 13:04:21.718496 (+1019614us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:21.871426 (+152930us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:21.872494 (+  1068us) log.cc:844] Serialized 26 byte log entry
0709 13:04:21.872835 (+   341us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:21.872892 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.873104 (+   212us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:21.918718 (+ 45614us) raft_consensus.cc:1685] finished
0709 13:04:21.918895 (+   177us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.919074 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":912675,"prepare.queue_time_us":495737,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":897237,"raft.run_cpu_time_us":421,"raft.run_wall_time_us":7786,"spinlock_wait_cycles":4997376,"thread_start_us":575410,"threads_started":3,"wal-append.queue_time_us":853238}
I20260709 13:04:21.909730  8675 raft_consensus.cc:2393] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:04:21.922731 14351 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.064s	user 0.002s	sys 0.000s
I20260709 13:04:21.806730 13370 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:21.705576 14291 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:21.928298  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1002) took 1643 ms. Trace:
W20260709 13:04:21.885350 14268 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.633s	user 0.009s	sys 0.037s
I20260709 13:04:21.928906  8678 rpcz_store.cc:276] 0709 13:04:20.285113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.438509 (+153396us) service_pool.cc:224] Handling call
0709 13:04:20.779206 (+340697us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:20.962547 (+183341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.927733 (+965186us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b5400257bd0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:21.928045 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":770447232}
I20260709 13:04:21.929948 14291 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:21.840842  9382 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:21.844276 14014 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.846427 14208 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:21.931375 14014 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.847646 14154 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:21.848598 13973 raft_consensus.cc:993] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: : Instructing follower 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:04:21.849973  9352 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:21.838191  8680 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
W20260709 13:04:21.851487  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 66555.1us
W20260709 13:04:21.859656  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.834477 14276 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:21.938563 14208 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:21.931912  9382 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:21.939265  8680 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:04:21.939736 14129 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:21.939543 14208 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:21.940313 14129 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:21.941151  8680 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:21.941778 14129 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.846138 14106 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:21.944868 14106 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:21.944994 14208 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:21.945168  8751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2771) took 2484 ms (client timeout 1756 ms). Trace:
I20260709 13:04:21.945549 14106 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:21.910734 14313 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:21.945803  8751 rpcz_store.cc:269] 0709 13:04:19.460194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.460519 (+   325us) service_pool.cc:224] Handling call
0709 13:04:21.944574 (+2484055us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54002ad354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:21.944847 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2992644352}
I20260709 13:04:21.948439 14349 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:21.934468 14154 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.950002 14154 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:21.871799  8674 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:21.846241 13927 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:21.894132  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1009) took 1407 ms. Trace:
I20260709 13:04:21.895535  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2302) took 1519 ms. Trace:
I20260709 13:04:21.895776  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2300) took 1520 ms. Trace:
W20260709 13:04:21.912237  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 1.14826e+06us
I20260709 13:04:21.905261  8677 raft_consensus.cc:2410] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:04:21.833408 14266 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:21.951666  8680 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:21.914882  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:21.936189  8719 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:21.953123  8679 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:21.953284 13927 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.954430  8719 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:21.955075  8678 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:04:21.937458 14014 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:21.844355  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 900186us
W20260709 13:04:21.942571  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.864511 14213 raft_consensus.cc:2764] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:21.951282  8755 rpcz_store.cc:276] 0709 13:04:20.486656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.490888 (+  4232us) service_pool.cc:224] Handling call
0709 13:04:20.514093 (+ 23205us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:20.514517 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:20.514546 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:20.515203 (+   657us) log.cc:844] Serialized 26 byte log entry
0709 13:04:21.107332 (+592129us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:21.107396 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.107498 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:21.893408 (+785910us) raft_consensus.cc:1685] finished
0709 13:04:21.893455 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.893770 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":3555,"thread_start_us":569633,"threads_started":1,"wal-append.queue_time_us":1012622}
I20260709 13:04:21.951812  8825 rpcz_store.cc:276] 0709 13:04:20.375250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.375393 (+   143us) service_pool.cc:224] Handling call
0709 13:04:20.375675 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:20.375934 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:20.660355 (+284421us) log.cc:844] Serialized 14 byte log entry
0709 13:04:21.837083 (+1176728us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b18002ddde0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:04:21.860788 (+ 23705us) log.cc:844] Serialized 14 byte log entry
0709 13:04:21.895042 (+ 34254us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.895123 (+    81us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:21.895221 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.895359 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.895589 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111777,"prepare.queue_time_us":403337,"prepare.run_cpu_time_us":158,"prepare.run_wall_time_us":9282,"raft.queue_time_us":989076,"raft.run_cpu_time_us":579,"raft.run_wall_time_us":580,"spinlock_wait_cycles":54946560,"thread_start_us":476596,"threads_started":2,"wal-append.queue_time_us":249356}
I20260709 13:04:21.962719 14314 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:21.963677  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:21.963797 14314 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.951664  8829 rpcz_store.cc:276] 0709 13:04:20.375934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.376134 (+   200us) service_pool.cc:224] Handling call
0709 13:04:20.376432 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:20.399039 (+ 22607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:20.399078 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:20.399621 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 13:04:20.744780 (+345159us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:20.744826 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.744920 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:21.717315 (+972395us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:21.894743 (+177428us) raft_consensus.cc:1685] finished
0709 13:04:21.894782 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.895215 (+   433us) inbound_call.cc:177] Queueing success response
0709 13:04:21.907907 (+ 12692us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b48000cd158. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
Metrics: {"mutex_wait_us":11217,"raft.queue_time_us":11409,"raft.run_cpu_time_us":51069,"raft.run_wall_time_us":1517470,"spinlock_wait_cycles":244192896,"thread_start_us":274756,"threads_started":1,"wal-append.queue_time_us":549641}
I20260709 13:04:21.965512 14126 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:21.965885  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:21.968317 14158 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:21.969089 14158 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.969636 14158 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:21.975852  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:21.979418  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 740) took 1592 ms. Trace:
I20260709 13:04:21.980041  9384 rpcz_store.cc:276] 0709 13:04:20.386389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.386566 (+   177us) service_pool.cc:224] Handling call
0709 13:04:20.386938 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.157043 (+770105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.978429 (+821386us) spinlock_profiling.cc:243] Waited 736 ms on lock 0x7b54005b0754. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:21.979137 (+   708us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1692835200}
I20260709 13:04:21.981004 14126 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.981904 14126 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.982910 14213 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.983755 14213 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:21.984982 14126 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:21.986164 14126 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:21.986966 14126 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:21.989603 14126 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:21.993235  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:22.000546 14126 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:22.003070 14126 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:22.003836 13927 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:22.004449 14184 raft_consensus.cc:3055] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:22.007447 14349 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.007922 14184 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:22.008311 14167 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:22.007448  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:22.008688 14349 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.009087 14167 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.009488 14184 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.004423 14316 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:22.011300 14316 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:22.011721 14316 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:22.012548  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:22.013062 14316 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.014523  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:22.015419  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 165809us
I20260709 13:04:21.906140 14299 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.018676  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:22.019207 13973 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:22.019678 13973 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:22.020045 14163 raft_consensus.cc:1081] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Signalling peer 0e9b0850145a4c66bfe983ffdfa93a57 to start an election
I20260709 13:04:22.020253 13973 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:22.016602 14184 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.011799  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:22.031993 14349 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.030504 14314 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.033433 14129 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:22.016944 14316 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:22.035342 14324 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:22.036780 14324 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:22.036547 14014 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:22.037185 14324 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:22.019549 13927 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:22.040633  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.034472  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:22.029585  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:21.986466 14213 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.054909  8794 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:22.057787 14167 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:22.057933  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 92218.9us
I20260709 13:04:22.059134  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:22.060101 14354 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:22.061187  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:22.073146  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.075188  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:22.075714  8755 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:22.038378 14324 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.077837 14324 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:22.079877  8828 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:22.087764 13973 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.088194  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:22.088805 14354 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:22.090411 14352 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.091194  9382 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.091485  8679 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:22.093302  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:22.079684  8755 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:22.095842 14353 raft_consensus.cc:2764] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:22.055858 14313 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:22.095129  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:22.117944  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.078292 14106 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:22.120266  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 76725us
I20260709 13:04:22.105957 14172 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.131248 14235 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:22.131916 14235 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:22.132381  8672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2358) took 1972 ms (client timeout 1905 ms). Trace:
I20260709 13:04:22.132596  8676 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:22.133023  8672 rpcz_store.cc:269] 0709 13:04:20.160072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.160212 (+   140us) service_pool.cc:224] Handling call
0709 13:04:22.131868 (+1971656us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b540033aed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:22.132102 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4520837376}
I20260709 13:04:22.134303  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:22.135222  8673 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:22.135402  8674 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:04:22.136111 14172 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:22.136327 14168 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:22.135244  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 991) took 1961 ms. Trace:
I20260709 13:04:22.137229 14168 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.137377  8676 rpcz_store.cc:276] 0709 13:04:20.174140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.379345 (+205205us) service_pool.cc:224] Handling call
0709 13:04:22.132156 (+1752811us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.133359 (+  1203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.135000 (+  1641us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b540033aed0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:22.135173 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3996189952}
W20260709 13:04:22.138834 14068 consensus_meta.cc:287] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.018s
I20260709 13:04:22.140254 14106 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:22.141324 13927 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:22.141776 13927 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:22.142192 13927 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:22.149149  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.150065  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.132555 14355 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:22.150568  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.106683 14353 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:22.176323  8676 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:04:22.150823 14355 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:22.150255 14055 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.546s	user 0.015s	sys 0.006s
I20260709 13:04:22.132400 14235 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:22.156163 14014 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:22.182436  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:22.187059  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1104) took 1749 ms. Trace:
I20260709 13:04:22.187500 14168 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:22.188978  8825 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:22.187692  8826 rpcz_store.cc:276] 0709 13:04:20.380250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.380504 (+   254us) service_pool.cc:224] Handling call
0709 13:04:20.380834 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:20.381085 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:20.762488 (+381403us) log.cc:844] Serialized 14 byte log entry
0709 13:04:22.066504 (+1304016us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:22.104993 (+ 38489us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:22.109734 (+  4741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.109817 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:22.129256 (+ 19439us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":826054,"prepare.queue_time_us":493712,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":1764,"raft.queue_time_us":832966,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":312,"thread_start_us":796529,"threads_started":3,"wal-append.queue_time_us":670132}
W20260709 13:04:22.190188  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.180996 14355 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:22.190991 14055 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.191637 13973 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.193818 14055 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:22.207199  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 112214us
I20260709 13:04:22.208287  8793 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:22.209728 14168 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:22.210274 14168 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.210743 14168 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.210969  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:22.212984  8794 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:22.213291 14068 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.206837 14343 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
W20260709 13:04:22.219842  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.221135 13370 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.226358  8823 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:22.213012 14356 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:22.230809 14299 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.231319  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:22.227797 14055 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.236680 14334 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:22.236807 14055 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.237637 14334 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.238754 14299 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.239881 14278 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:22.235656 14106 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:22.231937 14349 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:22.239985 14291 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:22.243639  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:22.243819  9382 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:22.242851  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:22.241113  8679 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.240785 14106 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.253515  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:22.256306  8751 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:22.252669 14349 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.260021  8749 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:22.261217 14313 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.261693 14349 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.261937 14313 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.262867  9382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 1011) took 1222 ms. Trace:
I20260709 13:04:22.263275  9382 rpcz_store.cc:276] 0709 13:04:21.040596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.041134 (+   538us) service_pool.cc:224] Handling call
0709 13:04:21.839604 (+798470us) spinlock_profiling.cc:243] Waited 81.8 ms on lock 0x7b54001a43a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2470dc0 00007f32e24fce18 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6
0709 13:04:22.022594 (+182990us) spinlock_profiling.cc:243] Waited 80.9 ms on lock 0x7b580086a2d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24550aa 00007f32e24d7447 00007f32e24effe7 00007f32e24f17de 00007f32e24fd27c 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c
0709 13:04:22.262766 (+240172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374035072}
I20260709 13:04:22.264567 14313 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.219560 14068 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.262377  9384 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:22.174116 14353 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.241029 14356 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:04:22.277153  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 56543.6us
I20260709 13:04:22.267383  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:22.287469 13370 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:22.268198 14106 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:22.297062  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:22.299865  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.240715 14278 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.305289  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:22.306810  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:22.307482  8673 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:22.302506  8794 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:22.311154  8674 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:22.313313  8675 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: []
W20260709 13:04:22.317672 14307 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:04:22.288865 14184 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.311184 13973 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:04:22.318886 14184 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.319144 14163 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:22.306563  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:22.325229  9388 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:22.327867 14168 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.330230 14168 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:22.344664 14068 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:22.348781 14357 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:04:22.346369  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:22.281898  8747 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
 from {username='slave'} at 127.0.0.1:35702
I20260709 13:04:22.362639 14172 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.363106  8747 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.363637  8747 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.363571 14172 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.366184 14353 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:22.371109 14356 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.371467 14154 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.371873  8674 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:22.373605  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 81736.2us
I20260709 13:04:22.335844  8754 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:22.321344 14184 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:22.381387  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.383179  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 58031.9us
I20260709 13:04:22.389966 14151 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:22.390518 14151 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:22.395283 14068 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:22.412855 14151 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.375172 14163 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:04:22.413147 14068 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.414300 14125 raft_consensus.cc:3055] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:22.414842 14125 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:22.416138 14068 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:22.384991 14299 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.398571 14055 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.400671 14286 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:22.417834 14286 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:22.418321 14286 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.416198 14125 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.382453 14324 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:04:22.422173 14125 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:22.402293 14158 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.427176 14354 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:22.428587 14158 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.429276  8679 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:22.431387  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:22.432511  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:22.407112  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 106734us
I20260709 13:04:22.390969 14014 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:22.420894 14163 consensus_queue.cc:1059] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:22.434377 14014 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:22.435001 13370 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.435119 14014 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.435701 13370 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.437749 14014 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:22.442224  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:22.438462 14014 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:04:22.450081  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:22.444785  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:22.444944  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.451423  8754 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:22.445972  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:22.453963  9386 raft_consensus.cc:2376] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14. Re-sending same reply.
I20260709 13:04:22.435003 13849 raft_consensus.cc:3055] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:22.448386  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:22.455127 13849 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:22.439023 14305 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:04:22.456439 13849 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.457448 14204 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:22.457945 14204 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:22.442708 14334 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:22.388871  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:22.412186 14278 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.437791 13370 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.380630 14172 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.472257  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:22.489553  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.467978  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.464844  9384 raft_consensus.cc:2410] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 13:04:22.498384  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.499801  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 112124us
I20260709 13:04:22.455008  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:22.467664  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:22.462391  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 181669us
I20260709 13:04:22.489758  8674 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:04:22.491871  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:22.505528 14307 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:22.505760  8749 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:22.506247  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:22.506402 14307 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:22.507036  8755 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:22.459179 14204 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:22.491963  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.505621  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:22.463646  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:22.511058  8751 raft_consensus.cc:2410] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:04:22.512058  8754 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:22.516050  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:22.502195  8747 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.507792 14307 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.501886  9382 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:22.458262  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:22.479739  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:22.458330 13849 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:22.530500  8748 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:22.536492  8674 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:22.537402  8673 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:22.542368  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:22.542816  8672 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:22.470772 14154 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:22.543344  9387 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:22.553120  8747 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:22.553618  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:22.557865  9384 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:04:22.477303 14355 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.555634  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:22.552536  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:22.570950  9383 raft_consensus.cc:2393] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:22.559871 14125 consensus_queue.cc:1059] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.008s
W20260709 13:04:22.579284  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.581138  9380 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:04:22.579326 14125 raft_consensus.cc:3055] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:22.583241 14125 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:22.584554 14125 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.588182 14204 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:22.591992 14172 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:22.560268  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:22.573935  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:22.574553 14163 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:22.597110  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:22.597872  9382 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:22.550963 14278 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.586354 14125 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:22.603027  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.682056  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 115844us
W20260709 13:04:22.638927  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 300828us
I20260709 13:04:22.702030  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:22.707007  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.707720  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.701435  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:22.712313  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.770886  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:22.712775  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 152577us
I20260709 13:04:22.774749  8674 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 7.
W20260709 13:04:22.778434  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.792732  8753 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:22.796334 14356 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.797179 14356 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:22.799170  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 264504us
W20260709 13:04:22.799830 14235 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.005s
I20260709 13:04:22.800151  8796 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:22.800276 14235 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.801182 14168 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:22.801736 14168 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:22.802815  9381 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:22.803440  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:22.776650 14278 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.804301  8755 raft_consensus.cc:2393] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:22.805727 14278 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.788193 14172 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.812871  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:22.813423  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:22.813771  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:22.814328  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:22.815065  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:22.815325  8747 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:22.816190  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:22.817337  8754 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:22.818310 14353 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:22.820150 14355 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:22.813930  8748 raft_consensus.cc:2393] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:22.821450  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:22.822294  8672 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:22.811002 14172 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.828176 14172 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.835903 14356 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:22.836867 14278 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.837589 14278 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.839977 14278 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.791160 14324 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:22.849184 14324 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.849550 14324 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:22.850981  9382 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:22.852123 14235 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:22.788589 14349 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.855259 14349 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.857726 14349 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:22.793036  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:22.859290  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 66354us
I20260709 13:04:22.859985 14349 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:22.862708 14349 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.863268 14349 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:22.863610  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:22.864539  8752 raft_consensus.cc:2393] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:22.815410  8750 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 12. Current term is 13. Ops: [10.8-12.9]
I20260709 13:04:22.884169  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:22.882357  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:22.887638  8753 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:22.890728  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:22.885522 14168 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:22.897050 14106 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.004s
I20260709 13:04:22.897650 14106 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.898763 14354 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:04:22.874181 13927 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.732s	user 0.015s	sys 0.000s
I20260709 13:04:22.789839 14163 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:04:22.879444 14352 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.788s	user 0.009s	sys 0.004s
I20260709 13:04:22.877516 14353 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:22.942899  8755 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:22.907547 14163 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:22.951016 14163 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:22.874364 14278 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:04:22.953454  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:22.988123  8748 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:22.927863 14151 consensus_meta.cc:287] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.005s
I20260709 13:04:22.922308 14352 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:22.875946 14286 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.974373  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:22.910061 13927 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:22.928808 14354 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:04:23.006762  8755 raft_consensus.cc:1275] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 6. (term mismatch)
I20260709 13:04:22.975242 14154 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:04:23.007421  8755 pending_rounds.cc:85] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 5
I20260709 13:04:22.998032 14106 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:22.997566 14305 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:23.008014  8752 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:23.010408  8755 pending_rounds.cc:107] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 11.6
W20260709 13:04:23.022382  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.020848  8755 raft_consensus.cc:2889] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:23.026201  8755 pending_rounds.cc:107] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:04:23.028366  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:23.028849  8755 raft_consensus.cc:2889] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:23.030025  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 135498us
I20260709 13:04:23.027433  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:22.996475 14151 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:23.025461 14163 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:23.056213 14305 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:23.060192 14278 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.061002 14278 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:23.061723  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:23.062749  8752 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:04:23.063292 14174 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.063926 14174 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:23.078559 13927 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:23.079403 14174 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:23.091308  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 575419us
I20260709 13:04:23.099782 14278 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:23.108290  8752 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:04:23.115794  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.121032  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:23.123194  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 279267us
I20260709 13:04:23.124993  9381 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:23.126809  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 281056us
I20260709 13:04:23.129626 14235 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:23.130999 14235 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:23.131891  8752 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:23.132309  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:23.133329  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:23.133275 14174 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:23.134233  8749 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:23.135696 14343 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:23.136119 14343 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:23.136962  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.137161 14343 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:23.113936 14324 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:23.138931 14172 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.139546 14172 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:23.139994 14324 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:23.143901 14354 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:23.110466  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:23.142251 14172 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:23.157124  8674 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:23.158201  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:23.150098  8748 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:23.165344  8750 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:04:23.145049 13849 consensus_meta.cc:287] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.575s	user 0.003s	sys 0.008s
I20260709 13:04:23.168469  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:23.170805  8747 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:23.170785  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:23.173219  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:23.173779  8679 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:04:23.174796  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:23.179114  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:23.183380  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:23.193987  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.195559  8748 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:04:23.205296  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:23.206259  8751 raft_consensus.cc:2368] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14.
I20260709 13:04:23.205315  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:23.208727 14349 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:23.200465  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:23.190697 14343 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:23.225904  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.234138  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:23.200006 14305 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.238837  8824 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:23.176527 14354 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:23.225715  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:23.241291  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.245196  8748 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:23.268278  8752 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:23.268296  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:23.257845  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:23.284658  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:23.285742  8672 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:23.253463 14353 raft_consensus.cc:3055] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:23.289673  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:23.290823  8751 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:23.302476  8750 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:23.306000  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:23.267629 14286 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:23.318673  8828 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 16: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:23.321388 14331 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.322324 14331 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:23.325472 14331 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:23.331470  8681 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 13:04:23.293520 14353 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:23.269830 14305 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:23.362561  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:23.358878  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:23.375895  8823 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:23.376056  8676 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:23.376942  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:23.377823  8754 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:23.297226 14354 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:23.386391  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.390944  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:23.412744  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:23.416791  8825 raft_consensus.cc:2368] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15.
I20260709 13:04:23.414143  8755 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:23.421976  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:23.423156  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 501256us
I20260709 13:04:23.425696  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:23.429263  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:23.430405  8824 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:23.435940  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:23.442994  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:23.459990  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.414634 14307 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.014s	sys 0.036s
W20260709 13:04:23.473722  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 82729us
I20260709 13:04:23.477852  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:23.479673 14168 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:23.482522  8827 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:23.483178 14014 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:04:23.484560 14366 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.144s	user 0.001s	sys 0.001s
I20260709 13:04:23.487360 14353 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:23.489524  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:23.496001 14305 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:23.501165  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.469223 14151 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:23.472924 14307 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.015s	sys 0.036s
I20260709 13:04:23.579198  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:23.532181 14349 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:23.602770  8722 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.066s	sys 0.100s
W20260709 13:04:23.603389  8722 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.066s	sys 0.100s
W20260709 13:04:23.603797  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 816782us
W20260709 13:04:23.615839  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.616508  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.616927  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.593160 14382 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.569492 13973 consensus_queue.cc:1059] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
W20260709 13:04:23.646559  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.636193  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.621080  8722 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:23.646957  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 109143us
I20260709 13:04:23.603605  8678 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
W20260709 13:04:23.649798  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.467397  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:23.623939  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 491212us
W20260709 13:04:23.623355  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 494248us
I20260709 13:04:23.652199  8828 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:23.654175  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:23.654881  8750 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:23.647505  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 169827us
I20260709 13:04:23.681095  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:23.681685  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:23.682332  8754 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:23.618253 14172 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.664691  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:23.690191  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:23.692466  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:23.695371  8826 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:23.699503  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:23.694479  8645 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.923s (SENT) [suppressed 33 similar messages]
I20260709 13:04:23.712101 14312 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:23.712599  8823 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:23.702359  8754 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:04:23.717212  8754 pending_rounds.cc:85] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 9
I20260709 13:04:23.717571  8754 pending_rounds.cc:107] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:04:23.717876  8754 raft_consensus.cc:2889] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:23.684971 14353 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:23.738876  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.738116  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1119) took 1137 ms. Trace:
W20260709 13:04:23.750602  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.748956  8829 rpcz_store.cc:276] 0709 13:04:22.600470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.604204 (+  3734us) service_pool.cc:224] Handling call
0709 13:04:22.701430 (+ 97226us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:22.871977 (+170547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:22.872023 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:22.946238 (+ 74215us) log.cc:844] Serialized 26 byte log entry
0709 13:04:23.438241 (+492003us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:23.438291 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.438371 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:23.737871 (+299500us) raft_consensus.cc:1685] finished
0709 13:04:23.737894 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.738047 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67622,"raft.queue_time_us":88960,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"spinlock_wait_cycles":18304,"thread_start_us":441422,"threads_started":1,"wal-append.queue_time_us":669645}
I20260709 13:04:23.710364 14331 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.666240 14382 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:23.761355  8756 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:23.765018 14354 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:23.766337  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:23.767381  8824 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:23.767931 14373 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:23.720527 14172 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:23.734206 14378 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:04:23.736122 14388 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.770982 14388 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:23.774890  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:23.809134  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 67844.8us
I20260709 13:04:23.815572 14172 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:23.817397  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:23.820510  8823 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:23.820892  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:23.821516  8825 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:23.819114  8821 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:23.817734  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:23.825135  8829 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:23.735785 14381 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:23.826232 14381 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:23.828588 14381 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:23.820513 14172 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:23.819074  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:23.740821  8750 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:23.837486  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:23.895423  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.891906  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:23.901021  8826 raft_consensus.cc:1275] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:04:23.902904  8750 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:04:23.904184  8647 leader_election.cc:336] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.720s (SENT)
I20260709 13:04:23.907035 14381 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.907723 14381 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:23.910344 14381 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:23.910750  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:23.911538  8824 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
W20260709 13:04:23.914564  8672 consensus_meta.cc:287] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.610s	user 0.020s	sys 0.007s
I20260709 13:04:23.915160  8672 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
W20260709 13:04:23.918074 14163 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.847s	user 0.016s	sys 0.010s
I20260709 13:04:23.904919  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:23.947269  8676 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.557s	user 0.027s	sys 0.029s
I20260709 13:04:23.948143  8676 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:04:23.895035 14172 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:23.952327  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 334247us
I20260709 13:04:23.813159 14331 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:23.963718  8750 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:23.968317  9383 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:04:23.969403  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:23.971513  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:23.976513  8754 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:23.978440  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:23.951973 14381 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:23.982007  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.985292  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:23.986060  8754 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:23.988044  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:23.988996  9381 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:23.991183  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 284237us
I20260709 13:04:23.992244  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:23.981410 14343 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.733s	user 0.016s	sys 0.012s
I20260709 13:04:23.994642  8750 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:24.000604  9352 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:23.981384 14163 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:24.011595  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 107469us
I20260709 13:04:23.979566 14388 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:24.015385  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 120703us
W20260709 13:04:24.015153  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 56619.2us
I20260709 13:04:24.017683  8821 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:04:23.988713 14382 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:24.018589  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.022450  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:24.028180  8824 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:24.031018  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.033753  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:24.036844  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:24.037779  8722 leader_election.cc:336] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.508s (SENT)
I20260709 13:04:24.042645  8753 raft_consensus.cc:2393] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:24.021049 14394 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:24.059391 14374 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:04:24.064718  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 57935.5us
I20260709 13:04:24.019219 14163 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.069725  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:24.069872  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:24.068125  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.071218  8753 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:24.054920 14373 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:24.073310  8722 leader_election.cc:336] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.564s (SENT)
I20260709 13:04:24.076500  8826 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:24.073823  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:24.078198 14343 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:24.078883 14343 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:24.079250  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.039311 14172 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.079833 14343 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.081804 13702 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:24.063710 14394 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.086872 14394 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:24.087256 14331 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:24.087700  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:24.088397  8830 raft_consensus.cc:2376] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15. Re-sending same reply.
I20260709 13:04:24.089359  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:24.090888  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.091307  8830 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:24.091579  8824 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:24.092697 14404 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:24.093187  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.094106  8821 raft_consensus.cc:2410] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:04:24.094890  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 78685.4us
I20260709 13:04:24.097360  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:24.067562 14311 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:24.096118  9388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 1023) took 1547 ms. Trace:
I20260709 13:04:24.101876  8674 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:04:24.107170  8674 pending_rounds.cc:85] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 8
I20260709 13:04:24.107542  8674 pending_rounds.cc:107] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:04:24.107885  8674 raft_consensus.cc:2889] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:24.094278  8722 leader_election.cc:336] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.612s (SENT)
I20260709 13:04:24.108656  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:24.111758  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.112548  8753 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:24.112896  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.113521  8825 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
W20260709 13:04:24.115259  8647 leader_election.cc:336] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.247s (SENT)
I20260709 13:04:24.114430  8829 raft_consensus.cc:2393] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:04:24.103509 14389 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:04:24.118701 13702 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:24.104745  9388 rpcz_store.cc:276] 0709 13:04:22.549223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.549412 (+   189us) service_pool.cc:224] Handling call
0709 13:04:22.565444 (+ 16032us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:22.602166 (+ 36722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:22.602201 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:22.640940 (+ 38739us) log.cc:844] Serialized 26 byte log entry
0709 13:04:22.957588 (+316648us) spinlock_profiling.cc:243] Waited 7.16 ms on lock 0x7b1800028120. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:04:23.103426 (+145838us) spinlock_profiling.cc:243] Waited 56 ms on lock 0x7b1800028120. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32e63cd58f 00007f32df40c571 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd
0709 13:04:23.424642 (+321216us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:23.424707 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.424817 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.095889 (+671072us) raft_consensus.cc:1685] finished
0709 13:04:24.095913 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.096042 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5863,"raft.queue_time_us":133819,"raft.run_cpu_time_us":9130,"raft.run_wall_time_us":72696,"spinlock_wait_cycles":145306112,"thread_start_us":458775,"threads_started":1,"wal-append.queue_time_us":1070335}
I20260709 13:04:24.127933 13702 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:23.951272  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:24.130683  8575 catalog_manager.cc:5697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 8 to 9, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 9 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:24.135107  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.135649  8793 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:24.135993  8821 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:24.136615 14373 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:24.137024  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:24.137277 14373 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:24.131031  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.131526 14380 raft_consensus.cc:3055] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:24.144229 14380 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:24.145432 14380 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.115025 14311 raft_consensus.cc:697] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:24.148010 14394 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.138505 14373 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:24.153832  8825 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:24.153669  8828 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:24.148785 14311 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:24.149570 14168 consensus_meta.cc:287] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.669s	user 0.028s	sys 0.007s
I20260709 13:04:24.159108 14168 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:24.161437  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:24.122445 14404 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:24.164077 14405 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.167518 14380 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:24.164263 14163 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:24.171030 14354 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.175241  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:24.176278  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:24.176681  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:24.185729  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.185768  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.200682  8828 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:24.201112  8827 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:24.202155  8756 raft_consensus.cc:2393] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 7: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:24.198853  8677 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:24.203130  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:24.203907  8750 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:24.207082  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:24.208168  8751 raft_consensus.cc:2376] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15. Re-sending same reply.
I20260709 13:04:24.214107  8674 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:04:24.219671  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2804) took 1073 ms. Trace:
W20260709 13:04:24.224618  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 130370us
W20260709 13:04:24.226231  8722 leader_election.cc:336] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.641s (SENT)
I20260709 13:04:24.226752  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:24.237713  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:24.177130 14235 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
W20260709 13:04:24.253852  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.251557 14394 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:24.256711  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:24.258576  8750 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:24.216687 14404 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:24.262549  9381 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:24.262447  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:24.220181  8749 rpcz_store.cc:276] 0709 13:04:23.145988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.146249 (+   261us) service_pool.cc:224] Handling call
0709 13:04:23.154829 (+  8580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.389701 (+234872us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.389752 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:23.485641 (+ 95889us) log.cc:844] Serialized 26 byte log entry
0709 13:04:23.718034 (+232393us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:23.718100 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.718438 (+   338us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.219404 (+500966us) raft_consensus.cc:1685] finished
0709 13:04:24.219430 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.219597 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11834,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":16884,"thread_start_us":466087,"threads_started":2,"wal-append.queue_time_us":412568}
I20260709 13:04:24.262552  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:24.264081  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 197093us
I20260709 13:04:24.268838  8673 raft_consensus.cc:2393] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:24.274050  8647 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:24.275151 14396 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:24.275995 14388 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:24.276525 14388 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.275357  8647 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:24.276960 14388 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:24.277688 14396 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:24.278313 14396 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:24.281380  8647 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:24.279688 14396 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:24.282852 14381 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:24.283393 14381 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.283777 14381 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:24.284989 14168 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:24.285975 14168 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.258076 14172 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:24.290205  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:24.290381 14404 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.291290  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:24.292335 14349 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:24.292804 14349 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:24.293030 14404 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:24.293249 14349 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:24.181699 14405 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.295641 14405 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:24.210289 14354 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:24.298781 14331 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:24.299880 14354 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:24.300446 14154 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.300554 14168 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:24.301043 14154 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.306403 14278 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:24.299535 14331 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.309492  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:24.309894  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:24.319273  8756 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:24.321539 14168 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.325449  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:24.326509  9381 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:24.329464  8796 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:24.331486  8647 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:24.324903  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:24.335016  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 108797us
W20260709 13:04:24.337435  8722 leader_election.cc:336] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.704s (SENT)
I20260709 13:04:24.322927 14405 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:04:24.312090 14154 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:24.353008  8722 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:24.325091 14174 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:24.363559  9383 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:24.313002 14278 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:04:24.350019 14353 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:24.385330 14353 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.385809 14353 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:24.322456 14168 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:24.352756 14391 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:24.392375  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:24.381947  8751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:04:24.393410  8751 raft_consensus.cc:606] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:24.384711  9381 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:24.321623 14381 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:24.329704 14349 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:24.371026 14415 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:24.368274 14392 consensus_queue.cc:1048] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:04:24.394166 14391 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.421886 14391 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:24.395514 14174 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 13:04:24.428521  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 91117.7us
I20260709 13:04:24.397135 14312 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:24.431106 14312 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.431512 14312 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:24.435621 14307 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.416338 14349 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:24.417456  8751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:04:24.418186 14168 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:24.419209 14416 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:04:24.406826 14354 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:04:24.446825 14154 consensus_queue.cc:1059] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:24.448236  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:24.451586  8751 raft_consensus.cc:606] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:24.480837  8751 raft_consensus.cc:614] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:24.480952  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:24.459755  9381 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:24.467597  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:24.461180  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 122161us
W20260709 13:04:24.457502  8722 leader_election.cc:336] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.720s (SENT)
I20260709 13:04:24.476109  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:24.484223  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:24.488520  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.484826  8754 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:24.451501 14396 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.498587  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.499387  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:24.480189 14354 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:24.495401  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.506950  8827 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
W20260709 13:04:24.508764  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51321.3us
I20260709 13:04:24.500355  8755 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:04:24.462133 14373 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.495427  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:24.509606  8722 leader_election.cc:336] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.721s (SENT)
I20260709 13:04:24.489530  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:24.516790  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:24.519335  8823 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:04:24.473697 14392 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:04:24.493737 14349 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:24.514714  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:24.461916 14307 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.529917  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:24.494079 14154 consensus_queue.cc:1243] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:24.505950 14354 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:24.536674 14422 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:24.542696 14163 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:24.543481 14331 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:24.549430 14174 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:04:24.560321  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 50742.1us
I20260709 13:04:24.550873  8645 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:24.561149 14174 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:24.555521 14163 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.562242 14174 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:24.562873 14163 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:24.568014  9383 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:24.568920  9383 raft_consensus.cc:606] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:24.550493 14388 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:24.573096 14388 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:24.574141 14388 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:24.575321 14418 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 13:04:24.574802  8722 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.809s (SENT)
I20260709 13:04:24.578934  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:24.552273  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 65494.8us
I20260709 13:04:24.585758  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:24.512537 14278 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:04:24.528136 14349 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:24.534575 14415 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:24.604620  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:24.609498  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:24.609402  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 74321.4us
I20260709 13:04:24.596911 14388 consensus_queue.cc:1059] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:04:24.614676 13702 raft_consensus.cc:2955] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Committing config change with OpId 13.11: config changed from index 4 to 11, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:24.554724 14393 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:04:24.618170 14278 consensus_queue.cc:1243] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:24.619773  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.618178  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:24.578635 14163 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.513295 14396 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.630231  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:24.625536  8824 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:24.631028  8753 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:24.625427  8829 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:24.522953 14373 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:24.634729  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:24.636123  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 76181.6us
I20260709 13:04:24.609851 14349 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:24.639494  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:24.640529  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:24.641215  8673 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:24.641485  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:24.642231  8677 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 7.
I20260709 13:04:24.642468  8672 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:04:24.643168  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68410.7us
W20260709 13:04:24.643936  8722 leader_election.cc:336] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.900s (SENT)
I20260709 13:04:24.641384  8753 raft_consensus.cc:2955] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 4 to 11, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:24.610332 14415 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.611210 14381 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:24.652796  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.656699 14427 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:24.633082 14278 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:24.666806  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:24.668147  8677 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:24.668537 14154 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.669112 14154 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.670343  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:24.671202  8672 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:24.676487  8674 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:04:24.682036  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:24.682265  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 45207.2us
W20260709 13:04:24.681870  8722 leader_election.cc:336] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.964s (SENT)
I20260709 13:04:24.650840 14407 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.684392  8824 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: [12.9-14.10]
I20260709 13:04:24.649350 14388 raft_consensus.cc:3055] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:24.691147  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:24.692029  8645 leader_election.cc:400] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14.
I20260709 13:04:24.692544  8645 leader_election.cc:403] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:04:24.693636  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:24.653604 14415 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:24.663021 14278 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.634087 14353 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.661651 13973 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:24.703212  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:24.712141  8755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:04:24.713150  8755 raft_consensus.cc:606] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:24.658221 14429 raft_consensus.cc:3055] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:24.680203 14154 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:24.720389  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.690158 14174 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:24.625344 14423 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 13:04:24.695925 14388 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:24.724117  8824 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:24.685442 14396 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:24.723450  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:24.753778  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:24.755506  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.756879  8821 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 17: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:24.756654  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:24.717702 14278 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:24.718030 13973 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.697654 14407 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.735672  8673 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:04:24.691694 14392 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:04:24.658903 14354 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.757730  8676 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:24.762775  8677 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:24.758574  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.766093  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:24.742159  8672 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:24.719146 14429 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:04:24.769940  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 50163.6us
I20260709 13:04:24.776851  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:24.778097  8676 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:24.781206  8755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:04:24.781949  8755 raft_consensus.cc:606] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:24.782393  8755 raft_consensus.cc:614] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:24.762755 14396 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:24.786283  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.787202  8823 raft_consensus.cc:2368] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15.
I20260709 13:04:24.765242 13973 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:24.752959 14381 consensus_queue.cc:1048] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:04:24.796638  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:24.797645  8676 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:24.799525  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:24.801550  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 197075us
I20260709 13:04:24.804152  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:24.802850  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:24.808427  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:24.808264  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:24.805176  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:24.809341  8824 raft_consensus.cc:2376] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15. Re-sending same reply.
I20260709 13:04:24.813233  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.813788  8645 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:24.790575 14397 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.795547 14354 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:24.820003  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:24.827499  8679 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:24.829957 14432 raft_consensus.cc:993] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: : Instructing follower 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:04:24.830694 14432 raft_consensus.cc:1081] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Signalling peer 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:04:24.831063 14428 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:04:24.831822 14435 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:04:24.820459 14174 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:04:24.833374 14435 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:24.834090 14435 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:24.839157  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:24.840055 14431 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:04:24.840850  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:24.841342 14436 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:04:24.842900 14436 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:24.843250 14436 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:24.844044 14436 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.853065  8676 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:24.813378  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:24.788460 14391 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:24.864995  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:24.864033  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:24.868507  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 55199.2us
I20260709 13:04:24.845342 14436 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:24.880209  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:24.880712  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.880991  8755 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:24.882876  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.885860  8824 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:24.886723  8823 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:24.886828  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:24.887012  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 98598us
I20260709 13:04:24.887557  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.887684  8827 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:24.891196  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:24.892125  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:24.893504 14168 raft_consensus.cc:3055] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:24.893942 14168 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:24.897487 14427 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:24.897921 14427 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:24.905345 14388 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:24.907537 14312 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.911890  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.913991  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.917104  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.918155  8828 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:24.919117  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.918963 14427 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:24.919417  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:24.919847  8822 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:24.920559 14427 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:24.921100  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.922950  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.923228  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.924470  8824 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:04:24.924875  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:24.925555  8825 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:24.927182  8674 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:24.928164  8829 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
W20260709 13:04:24.885807  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.862493 14438 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:24.862723 14163 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:24.882301 14392 raft_consensus.cc:2764] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:24.938156  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:24.940500  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 49369.3us
I20260709 13:04:24.946864  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:24.886173 14429 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:24.950464 14174 consensus_queue.cc:1175] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e814ed2013244fa6be42623ea5c0c7b9" responder_term: 12 status { last_received { term: 12 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 12 index: 9 } }
I20260709 13:04:24.952696 14353 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:24.954205  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:24.954537 14450 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:24.955040 14450 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:24.955461 14450 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:24.927263 14373 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:24.959998 14391 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:24.936419 14438 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:24.968000 14438 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:24.969676 14438 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:24.925784 14388 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:24.847838 14397 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.976796  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:24.984277  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 46258.4us
I20260709 13:04:24.985973  8794 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:24.987102  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 173368us
W20260709 13:04:24.988801  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 303259us
I20260709 13:04:24.991695  8825 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:24.865113 14437 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.124s
I20260709 13:04:24.992841  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:24.995746  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.996551  8825 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:24.992365 14422 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:24.997790  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:24.998697  8823 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:24.999317  8645 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:25.000705  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:25.001890  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:25.002725 14440 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:25.003602 14440 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:25.004096 14440 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:25.005200 14440 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:25.008780  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:25.010761  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:25.011782  8679 raft_consensus.cc:2368] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16.
I20260709 13:04:25.011962  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:25.014176  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:24.973412 14439 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:04:25.015101  8676 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:25.016314 14381 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:25.017184 14381 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:25.017643  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:24.949075 14444 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:04:25.019445 14444 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:25.020392  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:25.022945  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:25.023133 14443 raft_consensus.cc:3055] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:25.023627 14443 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:25.023736  8680 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:25.024740 14443 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.025560  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:25.026120 14443 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:24.992681  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:24.972124 14168 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:24.997946 14354 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:24.942785 14432 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:04:25.037022  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:24.991955 14353 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:24.989363 14373 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:25.040170  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.041630  8825 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
W20260709 13:04:25.042557  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 51543.3us
I20260709 13:04:25.043280  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:25.043763  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:25.044740  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:25.039978  8722 leader_election.cc:336] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.012s (SENT)
I20260709 13:04:25.047287  8679 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:25.029099 14422 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:25.036787 14428 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:25.056861  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:25.068656  8824 raft_consensus.cc:2376] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9. Re-sending same reply.
I20260709 13:04:25.057232  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:25.056890  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.059034  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:25.061077  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.071738  8827 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:25.051537  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.072458  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.072933  8825 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:25.073542  8829 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:25.067777  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:25.075599  8675 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:25.079133  8828 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:04:25.050580 14452 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:25.079155  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.087172  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.077356 14168 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:25.089731  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:25.080842 14373 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:25.091303 14373 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:25.090982 14422 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:25.091935 14373 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:25.108834  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 114644us
I20260709 13:04:25.110258  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:25.110836  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 70819us
W20260709 13:04:25.115325  8722 leader_election.cc:336] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.132s (SENT)
I20260709 13:04:25.115435 14439 consensus_queue.cc:1059] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:04:25.116719  8722 leader_election.cc:336] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.160s (SENT)
I20260709 13:04:25.115330 14278 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.119066  8676 raft_consensus.cc:2368] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15.
I20260709 13:04:25.119884 14397 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:25.124092  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 79404.3us
I20260709 13:04:25.125882  8794 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:25.132728  8675 raft_consensus.cc:2410] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:04:25.133453  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:25.135859  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:25.134377  8826 raft_consensus.cc:2376] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15. Re-sending same reply.
I20260709 13:04:25.078047 13973 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:25.140483  8794 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:25.141636  9383 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:25.141976  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:25.143354  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:25.109392 14452 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:25.131491 14382 raft_consensus.cc:3055] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:25.148185  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.152377  8828 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:25.149942  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:25.135144 14392 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.159569 14331 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:25.160210 14331 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:25.161325 14331 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:25.162987  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:25.163714  8676 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:25.148545 14394 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:04:25.154714 14235 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:04:25.144500 14353 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:25.173120 14353 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.173563 14353 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:25.179299 14278 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:25.183207 13702 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.566s	user 0.013s	sys 0.002s
W20260709 13:04:25.184715  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.181636 14396 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14.
I20260709 13:04:25.188295  8678 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:04:25.111953 14349 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.190008  8678 pending_rounds.cc:85] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 12
I20260709 13:04:25.199299  8678 pending_rounds.cc:107] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:04:25.197867  8676 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:04:25.200677  8755 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8.
I20260709 13:04:25.158005 14392 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.199731  8678 raft_consensus.cc:2889] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:25.215850  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 72577.5us
I20260709 13:04:25.218081  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:25.218806  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.218953  8676 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:25.215471  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:25.219854  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.220695  8828 raft_consensus.cc:2368] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17.
I20260709 13:04:25.221691  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.222991  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:25.223873  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.224429  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:25.224128 14452 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.225127  8675 raft_consensus.cc:2368] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16.
I20260709 13:04:25.214543 14437 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:25.227013  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.223121  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:25.246289  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:25.250329  8676 raft_consensus.cc:2368] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15.
I20260709 13:04:25.151616 14382 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:25.231264 14455 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:25.214807 14415 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.137148 14172 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 13:04:25.238862  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.239049  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:25.244997  8673 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:25.274335  8827 raft_consensus.cc:2410] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:04:25.226001  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:25.246765  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.275913  8828 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:25.248510  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:25.277254  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.278175  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:25.255330  8672 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:25.250396  8826 raft_consensus.cc:2376] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13. Re-sending same reply.
I20260709 13:04:25.279029  8825 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:25.142939 14354 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:25.282902  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:25.274641  8821 raft_consensus.cc:2376] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15. Re-sending same reply.
W20260709 13:04:25.286257  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 176182us
I20260709 13:04:25.200381 14168 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:25.290992  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:25.292758  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:25.202189 14454 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:25.293459  8827 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:25.289079  8719 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:25.293804  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:25.296288  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:25.296557  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:25.296985  8676 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:25.297250  8753 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:25.298053  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.301847  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:25.302819  8753 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:04:25.303992  8719 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:25.306651  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:25.309361  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:25.313560  8825 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:25.314592  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:25.310117  8751 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:25.317623  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:25.318321  8748 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:25.319234  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:25.315368 14172 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:25.322080 14435 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:25.322912  8677 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:25.327334  8754 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:04:25.327803  8754 pending_rounds.cc:85] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 9
I20260709 13:04:25.328049  8754 pending_rounds.cc:107] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:04:25.328295  8754 raft_consensus.cc:2889] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:25.316977  8827 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:25.331835  8719 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:25.333195  8676 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:25.334632  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:25.335722  8676 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:25.300827 14436 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:25.343163  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:25.310792 14448 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:25.348028  8748 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
W20260709 13:04:25.354812  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.378628  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:25.379844  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.397783  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.337394 14427 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:25.379187 14450 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.337126 13973 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:25.341619 14435 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:25.408610  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 65545us
I20260709 13:04:25.336843 14451 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:25.344841 14436 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.409541  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:25.348915 14448 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:25.411334  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:25.402272 14465 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:04:25.455358  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 44060.8us
I20260709 13:04:25.455349  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.408464 13702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:58628 (request call id 10) took 2229 ms. Trace:
W20260709 13:04:25.460747  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 128988us
I20260709 13:04:25.459580  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:25.462291  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:25.467547  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:25.413234 14436 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:25.414166 14396 raft_consensus.cc:2764] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:04:25.412196 14172 raft_consensus.cc:697] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:25.401918 14354 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:25.477344  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.494669  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.496407  8719 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:25.473278 14445 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:25.343703 14382 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.500267  8822 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:25.416179 14278 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:25.461815 14427 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:25.510049  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 360124us
I20260709 13:04:25.514648  8676 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:25.503050  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:25.519097  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.520109  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:25.520233  8822 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:25.501305 14451 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:25.499740 14465 raft_consensus.cc:2764] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:04:25.525003 14415 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:25.525475 14448 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:25.519095 14349 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:25.558485  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.521950 13973 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:25.557225  8719 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:25.578821  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.576212  8719 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:25.583468  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.584508  8822 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:04:25.555109 14451 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:25.540136 14373 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.594247  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:25.591326  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.574497 14392 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:25.575088 14448 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:25.579058 14415 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:25.582851 14467 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:25.560249 14468 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:04:25.625684  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:25.627569  8822 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:04:25.612768 14438 consensus_meta.cc:287] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.641s	user 0.006s	sys 0.010s
I20260709 13:04:25.577131 13973 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:25.461117 13702 rpcz_store.cc:276] 0709 13:04:23.116669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.122396 (+  5727us) service_pool.cc:224] Handling call
0709 13:04:23.191924 (+ 69528us) log.cc:844] Serialized 494 byte log entry
0709 13:04:23.824231 (+632307us) spinlock_profiling.cc:243] Waited 6.24 ms on lock 0x7b54006cbea0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2472142 00007f32e2452eaa 00007f32e2459d53 00007f32e2459cd9 00007f32e2459c69 00007f32e2459c31 00007f32e2458dfd 000000000038f354 000000000038f199 00007f32df40c475
0709 13:04:24.004947 (+180716us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b180041df40. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2479b26 00007f32e247349a 00007f32e2472714
0709 13:04:24.365477 (+360530us) spinlock_profiling.cc:243] Waited 6.74 ms on lock 0x7b54006cbea0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2479b26 00007f32e247349a 00007f32e2472714 00007f32e2452eaa 00007f32e2459d53 00007f32e2459cd9 00007f32e2459c69 00007f32e2459c31 00007f32e2458dfd 000000000038f354
0709 13:04:25.345811 (+980334us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1406,"raft.run_cpu_time_us":11038,"raft.run_wall_time_us":1018609,"spinlock_wait_cycles":68407680,"thread_start_us":348307,"threads_started":1,"wal-append.queue_time_us":538213,"wal-append.run_cpu_time_us":2135,"wal-append.run_wall_time_us":1024779}
I20260709 13:04:25.635226  8719 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:25.621801 14467 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:25.638394  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.629415 14415 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:25.643698  8822 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:25.590044 14172 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.647912 14373 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:25.651558  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.652725 14388 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.678s	user 0.013s	sys 0.000s
I20260709 13:04:25.641842 14465 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:25.658828 14469 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:25.659931 14469 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:25.660465 14469 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:25.660683  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.662185  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.663762  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:25.665302 13973 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:25.667510 14428 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:25.668020 14428 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:25.645448 14307 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:25.672209  8677 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:04:25.673511  8677 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:25.674927 14451 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.687769  8677 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:04:25.697263  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.639665 14438 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:25.608356 14427 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.697525  8675 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
W20260709 13:04:25.708408  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.709379  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:25.710048  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.710160  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:25.643042 14468 raft_consensus.cc:3055] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:25.715377  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:25.639443 14467 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:25.718673  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:25.716629  8751 raft_consensus.cc:2393] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:25.729369  8677 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 13:04:25.730768  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:25.731288  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:25.731635  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:25.644660 14353 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.661461 14469 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.715245  8794 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:25.771539  9355 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:25.782051  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 66870.7us
I20260709 13:04:25.784718  8829 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 17: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:25.785574  8752 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:25.711844 14438 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.790094  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:25.785284  8749 raft_consensus.cc:2376] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15. Re-sending same reply.
I20260709 13:04:25.789119 14354 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:25.794212 14354 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:04:25.779465 14473 log.cc:927] Time spent T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:04:25.798281  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:25.800274  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:25.760685 14445 raft_consensus.cc:697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:25.794643 14354 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:25.801739 14470 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:04:25.803692 14463 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:25.805258 14463 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:25.805843 14463 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:25.796571 13973 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:25.807080 13973 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:25.772591 13702 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.812881  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:25.813900  8751 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:25.781463 14468 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:25.815459  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:25.801124  8749 raft_consensus.cc:2376] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15. Re-sending same reply.
I20260709 13:04:25.809372  8821 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:25.807564 13973 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:25.802431 14445 consensus_queue.cc:237] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.797847 14415 raft_consensus.cc:697] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:25.839408 14429 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:25.841614  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:25.839759  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:25.792831 14438 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:25.848994  8756 raft_consensus.cc:2376] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13. Re-sending same reply.
I20260709 13:04:25.842314  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:25.842772  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:25.856393  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:25.857426  8829 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:25.859972  8829 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:25.861631  8829 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:25.863025  8829 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 13:04:25.843786 14444 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:25.865690 14457 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:25.889238  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:25.891187  8753 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 13:04:25.894203  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.866386 13702 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:25.866885  8566 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 11)
I20260709 13:04:25.904659  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:25.907049  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.901271 14475 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:04:25.897985 14373 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:25.929472 14407 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:25.937362  8722 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.633s (SENT)
I20260709 13:04:25.938118  8752 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:25.943513  8751 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
W20260709 13:04:25.943966  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.954094  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:25.955785  8752 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:25.934518 14461 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.463s	user 0.000s	sys 0.002s
I20260709 13:04:25.958138  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:25.959452  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:25.971747  8753 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 16: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:25.886948 14444 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:25.980610  8752 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:25.888618 14468 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.959893 14481 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:25.971563  8566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 210247us
I20260709 13:04:25.977764 14464 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:25.896003 14457 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:25.976850 14353 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:25.910187 14415 consensus_queue.cc:237] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:25.957469 14373 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.036571  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:26.028527  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:26.028951  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:26.095250  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:26.085994  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:26.103973  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 67505.9us
I20260709 13:04:26.111145  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:26.027890 14483 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
W20260709 13:04:26.030931 14482 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
W20260709 13:04:26.112653  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 257324us
I20260709 13:04:26.120361  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:26.121333  9384 raft_consensus.cc:2410] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:26.039047 14468 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:26.014532  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:26.124432  8753 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:26.017376 14444 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:26.015465 14484 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:04:26.015125 14463 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:26.038774 14407 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.112713  8722 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:26.004195 14432 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:26.117162 14427 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:26.106469 14163 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:26.014205 14481 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.119028 14354 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:26.194504 14439 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.327251  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2759) took 1152 ms. Trace:
I20260709 13:04:26.328140  8673 rpcz_store.cc:276] 0709 13:04:25.175121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.678789 (+503668us) service_pool.cc:224] Handling call
0709 13:04:25.679109 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.679476 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.679507 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:25.679971 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 13:04:25.767889 (+ 87918us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:25.767938 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.768019 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:26.326964 (+558945us) raft_consensus.cc:1685] finished
0709 13:04:26.326989 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.327176 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1587,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":87498,"threads_started":1,"wal-append.queue_time_us":159487}
I20260709 13:04:26.193473 14427 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:26.262171 14444 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.261685 14492 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:26.356905 14449 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:26.356581 14492 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.357487 14449 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.357955 14449 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:26.358680 14465 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:26.359227 14465 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:26.374158  8753 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:26.195242 14407 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.254091 14432 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.321242 14163 raft_consensus.cc:697] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:26.326475 14353 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:26.382701  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.355258 14427 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:26.380501  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:26.431185 14432 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:26.401077 14492 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:26.370537 14444 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:26.404371 14481 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:26.408291 14354 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:26.414950 13702 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:26.416354 14457 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:26.436455 14448 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.821s	user 0.005s	sys 0.010s
I20260709 13:04:26.428134 14499 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.428900 14485 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:26.435868  8753 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
W20260709 13:04:26.370808 14436 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.863s	user 0.015s	sys 0.000s
W20260709 13:04:26.378212 14443 consensus_meta.cc:287] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.348s	user 0.017s	sys 0.004s
W20260709 13:04:26.456344  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 343663us
W20260709 13:04:26.431246 14429 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.580s	user 0.009s	sys 0.003s
I20260709 13:04:26.427408  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2359) took 1187 ms. Trace:
I20260709 13:04:26.445521 14462 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.481384  8824 rpcz_store.cc:276] 0709 13:04:25.239896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.240177 (+   281us) service_pool.cc:224] Handling call
0709 13:04:25.241091 (+   914us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.620484 (+379393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.620521 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:25.636169 (+ 15648us) log.cc:844] Serialized 26 byte log entry
0709 13:04:25.751461 (+115292us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:25.751518 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.751623 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:26.427102 (+675479us) raft_consensus.cc:1685] finished
0709 13:04:26.427130 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.427322 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33335,"raft.queue_time_us":101296,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":56079,"thread_start_us":380267,"threads_started":2,"wal-append.queue_time_us":82441}
I20260709 13:04:26.484134  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:26.484894  8824 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
W20260709 13:04:26.485205 14428 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.817s	user 0.002s	sys 0.010s
I20260709 13:04:26.485723 14428 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.468227 14504 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:26.486969  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:26.487110 14504 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.488806  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:26.489609  8753 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:26.490155 14504 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:26.480502 14436 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:26.493198 14436 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:26.464278 14432 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:26.501920  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:26.465970  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.376753 14439 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:26.460176 14464 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:26.477630 14485 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.498525  8679 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:26.501372  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:26.462517 14438 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.596s	user 0.013s	sys 0.000s
I20260709 13:04:26.507681  9355 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:26.438362 14353 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:26.514272  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.514178  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:26.519724  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:26.521065  8674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2400) took 1731 ms (client timeout 1962 ms). Trace:
I20260709 13:04:26.522060  8830 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
W20260709 13:04:26.521642  8674 rpcz_store.cc:269] 0709 13:04:24.789404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.789602 (+   198us) service_pool.cc:224] Handling call
0709 13:04:26.520829 (+1731227us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54003a2ad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:26.520990 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280696192}
W20260709 13:04:26.527328  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.498939 14428 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:26.528973  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:26.515692 14373 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:26.530253  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1142) took 1031 ms. Trace:
I20260709 13:04:26.520792 13702 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:26.514710 14438 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:26.525126 14443 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:26.524852 14163 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.535118 14503 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:26.539489  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2283) took 1184 ms. Trace:
I20260709 13:04:26.479583 14499 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.547510 14499 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:26.533602  8822 rpcz_store.cc:276] 0709 13:04:25.499282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.195067 (+695785us) service_pool.cc:224] Handling call
0709 13:04:26.310065 (+114998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.310332 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:26.325998 (+ 15666us) log.cc:844] Serialized 14 byte log entry
0709 13:04:26.529872 (+203874us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.529932 (+    60us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:26.529989 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.530056 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.530187 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5789,"prepare.queue_time_us":15992,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":7394,"thread_start_us":197597,"threads_started":1}
I20260709 13:04:26.548218 14430 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:26.549046  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:26.549453  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.551338 14511 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:26.550449  8722 leader_election.cc:400] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17.
I20260709 13:04:26.552803  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:26.553218  8722 leader_election.cc:403] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:04:26.554200  9353 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:26.554845 14430 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17.
I20260709 13:04:26.549479  8829 rpcz_store.cc:276] 0709 13:04:25.354938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.864182 (+509244us) service_pool.cc:224] Handling call
0709 13:04:25.864500 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.864729 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:26.258671 (+393942us) log.cc:844] Serialized 14 byte log entry
0709 13:04:26.538579 (+279908us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.538756 (+   177us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:26.538868 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.539007 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.539364 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13043,"prepare.queue_time_us":28918,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":9356,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":575627,"threads_started":2}
I20260709 13:04:26.554419  8722 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:26.555648 14481 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:26.556591 14481 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.556710 14430 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:26.556919  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:26.557010 14481 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:26.557860  8822 raft_consensus.cc:2410] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:04:26.559409  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:26.556874  8829 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:04:26.560127  8829 pending_rounds.cc:85] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 9
I20260709 13:04:26.560403  8829 pending_rounds.cc:107] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:04:26.560648  8829 raft_consensus.cc:2889] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:26.560743  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1145) took 1044 ms. Trace:
I20260709 13:04:26.561460  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1144) took 1059 ms. Trace:
I20260709 13:04:26.561717 14430 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:26.561224  8822 rpcz_store.cc:276] 0709 13:04:25.516201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.559482 (+1043281us) service_pool.cc:224] Handling call
0709 13:04:26.559836 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.560032 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.560057 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.560107 (+    50us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:26.560148 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.560208 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.560344 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.563848  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1146) took 1044 ms. Trace:
I20260709 13:04:26.564970  8829 rpcz_store.cc:276] 0709 13:04:25.501652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.556010 (+1054358us) service_pool.cc:224] Handling call
0709 13:04:26.556442 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.561172 (+  4730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.561393 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.566432  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:26.564313  8822 rpcz_store.cc:276] 0709 13:04:25.519878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.562956 (+1043078us) service_pool.cc:224] Handling call
0709 13:04:26.563310 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.563506 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.563529 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.563572 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:26.563602 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.563654 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.563783 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.537268 14407 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:26.573107 14407 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:26.573617 14407 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.573997 14485 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:26.540575 13973 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.688s	user 0.007s	sys 0.006s
I20260709 13:04:26.574857 13973 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.577558 13973 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:26.577425 14405 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:26.578650  9381 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:26.536173 14464 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.580547  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:26.581362  8748 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:26.512135 14462 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:26.513129 14432 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:26.584853  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:26.587806  8748 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:26.589563  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:26.591665  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:26.541016 13702 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.594826  8751 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:26.593734  8722 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:26.596588  8793 leader_election.cc:400] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16.
I20260709 13:04:26.598591 14423 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:26.600376  8822 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:26.574071 14407 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:26.609087  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:26.611032 14491 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:26.611994  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:26.600378 13702 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:26.614012  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:26.593956 14485 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:26.616284 14353 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:26.616976 14353 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:26.617229 14467 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.820s	user 0.010s	sys 0.003s
I20260709 13:04:26.617722 14467 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.619174 14444 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:26.620464 14450 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:26.620990 14443 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:26.622051 14443 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:26.607949  8748 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:26.628962  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:26.633319  8748 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:26.541505 14505 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.093s
I20260709 13:04:26.523494 14465 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:26.605949 14457 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:26.606449  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:26.660419  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54275.6us
W20260709 13:04:26.659065  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 91100.3us
I20260709 13:04:26.676199 14513 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:26.660243 14457 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.680142 14467 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:26.681413 14457 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:26.682428  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2754) took 1509 ms. Trace:
I20260709 13:04:26.682891  8676 rpcz_store.cc:276] 0709 13:04:25.173483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.515635 (+342152us) service_pool.cc:224] Handling call
0709 13:04:25.529237 (+ 13602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.557285 (+ 28048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.557341 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:25.571816 (+ 14475us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:25.576776 (+  4960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.682114 (+1105338us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b54000da2d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:26.682184 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.682356 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63858048}
W20260709 13:04:26.690900  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 136755us
W20260709 13:04:26.695971  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 82032.7us
I20260709 13:04:26.696708  9353 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:26.696913  8793 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:26.698434  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.702963 14457 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.703685 14457 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:26.709517  8793 leader_election.cc:400] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15.
I20260709 13:04:26.689584 14444 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:26.723800  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2398) took 1947 ms. Trace:
I20260709 13:04:26.718550  8748 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:26.724963  8672 rpcz_store.cc:276] 0709 13:04:24.776340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.784965 (+  8625us) service_pool.cc:224] Handling call
0709 13:04:24.785419 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.279113 (+493694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.723403 (+1444290us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b5400176454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:26.723695 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080144256}
I20260709 13:04:26.725695  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:26.726451  8748 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:26.706410 14457 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:26.720444 14394 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:26.734107 14462 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:26.743132 14467 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.743989 14467 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:26.747869 14468 consensus_meta.cc:287] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.615s	user 0.011s	sys 0.002s
I20260709 13:04:26.714558 14154 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:26.749720 14432 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:26.751502  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2435) took 1040 ms. Trace:
W20260709 13:04:26.765285  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 55964.9us
I20260709 13:04:26.765825  8751 rpcz_store.cc:276] 0709 13:04:25.712262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.959057 (+246795us) service_pool.cc:224] Handling call
0709 13:04:26.751241 (+792184us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b540034ba54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:26.751398 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1458529664}
I20260709 13:04:26.728992 14444 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:26.770107  8793 leader_election.cc:400] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16.
I20260709 13:04:26.771469  8793 leader_election.cc:400] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15.
I20260709 13:04:26.777613  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:26.778067  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1038) took 1658 ms. Trace:
I20260709 13:04:26.778352  8677 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:26.778594  8751 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 17: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:04:26.778064  8680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2748) took 1753 ms (client timeout 2081 ms). Trace:
I20260709 13:04:26.778497  8678 rpcz_store.cc:276] 0709 13:04:25.119261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.131767 (+ 12506us) service_pool.cc:224] Handling call
0709 13:04:25.143531 (+ 11764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.217493 (+ 73962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.777807 (+1560314us) spinlock_profiling.cc:243] Waited 44.4 ms on lock 0x7b540033aed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:26.778003 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102027008}
W20260709 13:04:26.780272  8680 rpcz_store.cc:269] 0709 13:04:25.024808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.025287 (+   479us) service_pool.cc:224] Handling call
0709 13:04:26.777833 (+1752546us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54002562d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:26.777999 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640262528}
I20260709 13:04:26.751595 14465 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:26.707638 14449 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:26.784904 14312 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:26.772385 14432 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.795348  8748 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:26.783859  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:26.771006 14468 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.773257  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:26.808138  8751 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:26.809376  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2447) took 1075 ms. Trace:
I20260709 13:04:26.809722  8751 rpcz_store.cc:276] 0709 13:04:25.734238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.807401 (+1073163us) service_pool.cc:224] Handling call
0709 13:04:26.807849 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.809062 (+  1213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.809317 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:26.810483  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 93699.5us
I20260709 13:04:26.810956  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:26.751559 13702 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.774693 14444 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:26.827929  8722 leader_election.cc:336] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.796s (SENT)
I20260709 13:04:26.830332  8722 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:26.805069 14174 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:26.835023  8754 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:26.834731 14174 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.835898  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2448) took 1078 ms. Trace:
I20260709 13:04:26.836289  8754 rpcz_store.cc:276] 0709 13:04:25.757597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.807655 (+1050058us) service_pool.cc:224] Handling call
0709 13:04:26.834725 (+ 27070us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.835542 (+   817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.835832 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:26.837009 14521 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:04:26.837226 14174 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:26.837903  8754 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:26.842480  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2451) took 1082 ms. Trace:
I20260709 13:04:26.843309  8754 rpcz_store.cc:276] 0709 13:04:25.760190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.837157 (+1076967us) service_pool.cc:224] Handling call
0709 13:04:26.837613 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.842126 (+  4513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.842406 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.831524 14443 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:26.841885 14505 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:26.833596 14468 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:26.857442 14481 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.857475 14427 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.842561 14467 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:26.832204 14465 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:26.852916 14354 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:26.865636 14353 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:26.866294  8793 connection.cc:256] RPC call timeout handler was delayed by -1.2251267910000934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
I20260709 13:04:26.868099  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:26.872277 14460 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.465s	user 0.005s	sys 0.008s
I20260709 13:04:26.806969 14432 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:26.893728 14354 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:26.883775  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:26.896173 14354 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:26.896508 14312 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:04:26.859494 14510 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.259s	user 0.002s	sys 0.000s
I20260709 13:04:26.898742  8678 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:26.900322  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:26.901175  8680 raft_consensus.cc:2368] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16.
I20260709 13:04:26.882530  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:26.892314  8793 connection.cc:256] RPC call timeout handler was delayed by -1.1549267910000935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16870000000000002s for a response
I20260709 13:04:26.892992 14469 consensus_queue.cc:1059] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:26.920241 14481 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:26.923696 14449 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:26.926882 13702 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:26.927228  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:26.928041  8825 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:26.928242 14427 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:26.929052  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:26.929845  8825 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:26.930501 14468 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:26.931315  8827 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:04:26.932998  8827 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:04:26.934079  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:26.935473  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2766) took 1758 ms. Trace:
I20260709 13:04:26.935902  8673 rpcz_store.cc:276] 0709 13:04:25.177260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.360972 (+1183712us) service_pool.cc:224] Handling call
0709 13:04:26.381833 (+ 20861us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.382269 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.382310 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:26.437370 (+ 55060us) log.cc:844] Serialized 26 byte log entry
0709 13:04:26.626209 (+188839us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:26.626261 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.626336 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:26.935214 (+308878us) raft_consensus.cc:1685] finished
0709 13:04:26.935240 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.935407 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":126,"thread_start_us":124440,"threads_started":1,"wal-append.queue_time_us":303271}
W20260709 13:04:26.937680  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.939473  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:26.940994  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:26.943200  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 75155.7us
I20260709 13:04:26.900381  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1139) took 1402 ms. Trace:
W20260709 13:04:26.948671  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.909304  8794 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:26.944466  8828 rpcz_store.cc:276] 0709 13:04:25.498396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.888978 (+390582us) service_pool.cc:224] Handling call
0709 13:04:25.899612 (+ 10634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.108308 (+208696us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.108358 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.108425 (+    67us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:26.108478 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.900038 (+791560us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54003dead4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:26.900114 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.900294 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":479078784}
I20260709 13:04:26.952581  8794 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:26.956301  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:26.957203  8829 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:04:26.957115  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.892061 14465 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.960703 14449 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:26.962076 14449 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:26.952587 14354 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:26.952769 14505 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:26.968552  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:26.969306  8827 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:26.969476  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.959450 14442 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:26.960928 14353 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:26.960882 14465 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:26.975821  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2828) took 1266 ms. Trace:
I20260709 13:04:26.976476  8747 rpcz_store.cc:276] 0709 13:04:25.709015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.709533 (+   518us) service_pool.cc:224] Handling call
0709 13:04:25.709847 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.710115 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:25.734442 (+ 24327us) log.cc:844] Serialized 14 byte log entry
0709 13:04:26.915635 (+1181193us) spinlock_profiling.cc:243] Waited 9.25 ms on lock 0x7b1800119e00. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:04:26.975506 (+ 59871us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.975565 (+    59us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:26.975606 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.975672 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.975768 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22878,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":60,"spinlock_wait_cycles":21280128,"thread_start_us":59596,"threads_started":1}
I20260709 13:04:26.986831  8827 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:26.967609 14354 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:26.962218 14407 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:26.998806  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:27.007474  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:27.008605  8749 raft_consensus.cc:2410] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 13:04:27.010658  8830 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2274) took 1954 ms (client timeout 1796 ms). Trace:
W20260709 13:04:27.011142  8830 rpcz_store.cc:269] 0709 13:04:25.056180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.056977 (+   797us) service_pool.cc:224] Handling call
0709 13:04:27.010350 (+1953373us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b540002b7d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:27.010497 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3272290048}
I20260709 13:04:27.012452  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:27.013422  8830 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:27.019287  8825 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:26.972496 14485 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:04:26.976167 14442 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:26.994947  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:27.047489  8747 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:27.047986  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:27.048686  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:27.040074  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:26.994748 14505 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.000481 14354 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:26.973062 14447 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:04:26.991097 14353 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:27.049877  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:27.068542  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:27.081617  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:27.082732  8749 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:04:27.067147 14505 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.066838 14394 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:27.065613 14174 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:27.076650 14432 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:27.078640 14443 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:27.111553 14407 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:27.111595  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:27.114636  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:27.118819  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2452) took 1357 ms. Trace:
I20260709 13:04:27.119478  8754 rpcz_store.cc:276] 0709 13:04:25.761232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.844736 (+1083504us) service_pool.cc:224] Handling call
0709 13:04:26.874860 (+ 30124us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.875298 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.875332 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:26.875850 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 13:04:26.937591 (+ 61741us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:26.937661 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.937766 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:27.118461 (+180695us) raft_consensus.cc:1685] finished
0709 13:04:27.118513 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.118730 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34850,"raft.queue_time_us":6083,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":25602,"thread_start_us":61111,"threads_started":1,"wal-append.queue_time_us":117424}
W20260709 13:04:27.123804  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 152147us
I20260709 13:04:27.097410 14467 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.127853  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:27.078459 14517 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:04:27.128814  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:27.130538  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:27.127576 14467 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.128170  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:27.079016 14519 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:04:27.127745  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:27.127607  8722 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:27.135357  9384 raft_consensus.cc:2368] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15.
I20260709 13:04:27.135787  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:27.107937 14443 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:27.108866 14174 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.140973  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:27.145387  8747 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:27.140973  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:27.142635  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:27.146904  8754 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:27.125247 14394 raft_consensus.cc:697] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:27.153338  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:27.154426  9384 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:27.158036  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:27.158519  8830 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:27.159248  8827 raft_consensus.cc:2410] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 15 because replica has last-logged OpId of term: 14 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:04:27.159843  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:27.164726  8827 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:04:27.143332 14443 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:27.178165  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:27.179541  8830 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:27.144778 14485 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:27.130209 14353 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:27.151098 14432 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:27.159462 14476 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:27.190366  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.160048 14505 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:27.176246 14511 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:27.206369  8747 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:04:27.189311 14485 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:27.208710  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:27.209437 14485 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:27.210961  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:27.212129  8747 raft_consensus.cc:2410] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:04:27.189007 14432 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.214016 14432 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:27.133163 14442 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:27.187058 14466 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:27.208702  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:27.225872  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:27.229332  8645 connection.cc:256] RPC call timeout handler was delayed by -1.7873305410000739s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19140000000000001s for a response
W20260709 13:04:27.231215  8645 leader_election.cc:336] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.798s (SENT)
I20260709 13:04:27.231822  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:27.185034 14468 consensus_queue.cc:1059] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:04:27.190996 14469 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:04:27.240423  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:27.241274  8754 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:27.190807 14393 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:04:27.243698  9353 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:27.244935 14536 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:27.245410  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:27.245853 14536 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:27.250887 14397 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:27.251921 14397 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:27.253306 14397 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.260324  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2449) took 1502 ms. Trace:
W20260709 13:04:27.240201 14404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.833s	user 0.023s	sys 0.028s
I20260709 13:04:27.266889 14536 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:27.261392  8751 rpcz_store.cc:276] 0709 13:04:25.758354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.810914 (+1052560us) service_pool.cc:224] Handling call
0709 13:04:26.811377 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.831833 (+ 20456us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:26.973373 (+141540us) log.cc:844] Serialized 14 byte log entry
0709 13:04:27.168118 (+194745us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.185311 (+ 17193us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.185506 (+   195us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:27.185552 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.185651 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:27.259939 (+ 74288us) raft_consensus.cc:1685] finished
0709 13:04:27.259979 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.260215 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5815,"prepare.queue_time_us":172084,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":7051,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":22236,"thread_start_us":258168,"threads_started":2,"wal-append.queue_time_us":264611}
W20260709 13:04:27.267524 14404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.023s	sys 0.028s
I20260709 13:04:27.271522 14404 raft_consensus.cc:3055] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:27.271991 14404 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:27.272409 14437 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.967s	user 0.025s	sys 0.037s
W20260709 13:04:27.272954 14437 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.975s	user 0.025s	sys 0.037s
I20260709 13:04:27.273393 14437 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:27.273087 14404 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:27.274447 14404 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:27.274546 14437 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:27.281366 14449 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:27.200627 14476 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.287853  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:27.239153 14517 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:27.289837  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:27.216459 14444 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.293229  8751 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:27.313539  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:27.227427 14511 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.321069  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:27.325804 14532 raft_consensus.cc:476] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Not starting pre-election -- already a leader
I20260709 13:04:27.328787 14442 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:27.329433 14444 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:27.329869 14438 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:27.330120 14174 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:27.234316 14394 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.334966 14476 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:27.341449  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:27.341921  8749 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:27.343288  9353 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:27.344424  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:27.345742  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 118246us
I20260709 13:04:27.347936  8722 leader_election.cc:400] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15.
I20260709 13:04:27.348652  8722 leader_election.cc:403] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:27.350864  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:27.317557 14517 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:27.367089  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:27.362485  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:27.369244  8749 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:27.374517  8796 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:27.380185  8679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2751) took 2290 ms (client timeout 1609 ms). Trace:
W20260709 13:04:27.387831  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:27.380769  8679 rpcz_store.cc:269] 0709 13:04:25.075500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.075653 (+   153us) service_pool.cc:224] Handling call
0709 13:04:27.364862 (+2289209us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400175554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:27.364996 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3237870336}
I20260709 13:04:27.349772 14468 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:27.394219  8755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2819) took 2099 ms (client timeout 2127 ms). Trace:
I20260709 13:04:27.390239  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:27.356364 14519 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:04:27.394794  8755 rpcz_store.cc:269] 0709 13:04:25.295222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.295401 (+   179us) service_pool.cc:224] Handling call
0709 13:04:27.393983 (+2098582us) spinlock_profiling.cc:243] Waited 970 ms on lock 0x7b540034bf54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:27.394143 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2231068928}
I20260709 13:04:27.377645 14154 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:27.357067 14404 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:27.398061  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:27.401904  8827 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:27.359273 14174 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:27.358741 14427 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15.
I20260709 13:04:27.408654  8751 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:27.412760  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 181561us
W20260709 13:04:27.414240  8645 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.844s (SENT)
I20260709 13:04:27.417433  8749 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:27.420516  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:27.397511 14467 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:27.428511  8645 connection.cc:256] RPC call timeout handler was delayed by -1.1669941320000135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
W20260709 13:04:27.429190  8645 connection.cc:256] RPC call timeout handler was delayed by -1.1615941320000047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19230000000000003s for a response
W20260709 13:04:27.432569  8645 connection.cc:256] RPC call timeout handler was delayed by -1.1138941319999276s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
W20260709 13:04:27.433089  8645 connection.cc:256] RPC call timeout handler was delayed by -1.0202941320000036s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20800000000000002s for a response
I20260709 13:04:27.438889  8827 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:27.407867 14538 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:27.418730 14454 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.012s	user 0.008s	sys 0.019s
I20260709 13:04:27.407760 14519 raft_consensus.cc:697] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:27.451390 14174 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.443426 14517 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.452788 14174 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.455570 14174 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:27.456561 14549 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:27.457312 14549 raft_consensus.cc:697] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:27.457109 14354 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.449048 14539 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.458288 14467 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:27.424083 14465 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:27.458746 14539 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.458930 14467 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:27.461246 14539 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:27.461516  8748 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:04:27.462426 14354 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:27.406282 14154 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:27.516562  8675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2760) took 2341 ms (client timeout 1916 ms). Trace:
I20260709 13:04:27.517592  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:27.518110  8675 rpcz_store.cc:269] 0709 13:04:25.175415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.225715 (+ 50300us) service_pool.cc:224] Handling call
0709 13:04:27.516351 (+2290636us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b540033a9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:27.516484 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":872759680}
I20260709 13:04:27.519639  9388 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:27.447271 14538 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:27.452080 14454 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.102s	user 0.008s	sys 0.020s
I20260709 13:04:27.540686 14454 raft_consensus.cc:697] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:27.542217 14454 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:27.529969  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 111405us
W20260709 13:04:27.543395  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.544731  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:27.545231  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 124817us
I20260709 13:04:27.536890  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:27.529366 14174 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:27.547068  8647 connection.cc:256] RPC call timeout handler was delayed by -1.2019951099999844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
W20260709 13:04:27.547518  8647 connection.cc:256] RPC call timeout handler was delayed by -1.2019951099999844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
W20260709 13:04:27.547708  8719 leader_election.cc:336] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.905s (SENT)
I20260709 13:04:27.548323  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:27.562350  9381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 833) took 1589 ms. Trace:
I20260709 13:04:27.567099  9381 rpcz_store.cc:276] 0709 13:04:25.972925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.973312 (+   387us) service_pool.cc:224] Handling call
0709 13:04:25.973645 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.579212 (+605567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.561986 (+982774us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b54005e9554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:27.562263 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390695808}
I20260709 13:04:27.522805 14393 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.555161  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:27.571128  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 121498us
I20260709 13:04:27.572927  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:27.573594  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:27.556712  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:27.575915  8827 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:27.533175 14435 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:27.548435 14537 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:27.535395 14538 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:27.560159 14154 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:27.600323 14552 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 13:04:27.603765 14501 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.182s	user 0.000s	sys 0.004s
I20260709 13:04:27.608337  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:27.568553 14548 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:27.614857  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1143) took 2115 ms. Trace:
I20260709 13:04:27.615267 14548 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.615398  8824 rpcz_store.cc:276] 0709 13:04:25.499523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.485733 (+986210us) service_pool.cc:224] Handling call
0709 13:04:26.486088 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.529176 (+ 43088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.529213 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.529262 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:26.529296 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.614572 (+1085276us) spinlock_profiling.cc:243] Waited 42.8 ms on lock 0x7b540032df54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:27.614648 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.614794 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98489728}
I20260709 13:04:27.617306  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:27.610230  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:27.552603 14544 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:27.619254  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2446) took 1903 ms. Trace:
I20260709 13:04:27.593988  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:27.642979  8678 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:53352
I20260709 13:04:27.706269  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2453) took 1944 ms. Trace:
I20260709 13:04:27.697172  8756 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:27.623595  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:27.623714  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:27.640120  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2268) took 2586 ms (client timeout 1633 ms). Trace:
I20260709 13:04:27.708282  8679 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:27.708710  8681 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:04:27.636924  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:27.666320  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:27.644009  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2764) took 2467 ms. Trace:
I20260709 13:04:27.712198  8825 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:27.619462  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:27.712044  8747 rpcz_store.cc:276] 0709 13:04:25.761499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.212948 (+1451449us) service_pool.cc:224] Handling call
0709 13:04:27.219052 (+  6104us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.219345 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:27.236816 (+ 17471us) log.cc:844] Serialized 14 byte log entry
0709 13:04:27.558270 (+321454us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b18003183a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:04:27.705842 (+147572us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:27.705908 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:27.705960 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.706036 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.706195 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15552,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":22424,"raft.queue_time_us":100,"spinlock_wait_cycles":6508416,"thread_start_us":147204,"threads_started":1}
I20260709 13:04:27.713268  8676 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:27.704490  8752 rpcz_store.cc:276] 0709 13:04:25.715829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.988937 (+273108us) service_pool.cc:224] Handling call
0709 13:04:25.989979 (+  1042us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.003612 (+ 13633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.003683 (+    71us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:26.008717 (+  5034us) log.cc:844] Serialized 26 byte log entry
0709 13:04:26.316989 (+308272us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:26.317060 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.317182 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:27.602980 (+1285798us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:27.618916 (+ 15936us) raft_consensus.cc:1685] finished
0709 13:04:27.618938 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.619185 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13284,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":107,"spinlock_wait_cycles":29115136,"thread_start_us":199929,"threads_started":1,"wal-append.queue_time_us":316038}
I20260709 13:04:27.717013  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:27.717207  8752 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:27.685379  8827 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:27.724072  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:27.726159  8680 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:27.721479  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:27.711050  8677 rpcz_store.cc:276] 0709 13:04:25.176655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.797771 (+621116us) service_pool.cc:224] Handling call
0709 13:04:26.778160 (+980389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.786825 (+  8665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.643740 (+856915us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b54002562d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:27.643933 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2155155328}
I20260709 13:04:27.726662  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:27.727487  8673 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:27.727834  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:27.708153  8823 rpcz_store.cc:269] 0709 13:04:25.054065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.071690 (+ 17625us) service_pool.cc:224] Handling call
0709 13:04:27.639729 (+2568039us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540016ba54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:27.639965 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3731714304}
I20260709 13:04:27.729024  8676 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:27.729315  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:27.726130  8825 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:27.732388  8672 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:04:27.730909  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:27.734356  8676 raft_consensus.cc:2410] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:04:27.721613  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1058) took 1619 ms. Trace:
I20260709 13:04:27.628952 14382 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:27.642601 14449 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:27.736613  8752 rpcz_store.cc:276] 0709 13:04:26.101949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.716526 (+1614577us) service_pool.cc:224] Handling call
0709 13:04:27.716955 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.721227 (+  4272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.721522 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:27.741640  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:27.741863  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:27.623885 14550 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.582445 14393 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:27.739184  8677 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:04:27.568573 14174 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.735074  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:27.740506  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:27.744509  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:27.747231  8681 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:27.747359  8680 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:27.737985  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:27.618482 14548 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:27.637530 14422 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:27.748472  8674 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:27.748844  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:27.614423 14443 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.742432  8679 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:27.755307  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:27.652298 14439 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:27.591317 14435 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.750818  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:27.773054  8673 raft_consensus.cc:2376] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14. Re-sending same reply.
I20260709 13:04:27.590523 14519 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.815445  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2456) took 2053 ms. Trace:
W20260709 13:04:27.815936  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.817694 14443 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:27.819952  8828 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:27.821029  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:27.751560  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:27.742434  8823 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:27.750532  8752 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:27.758337  8756 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:27.815990  8751 rpcz_store.cc:276] 0709 13:04:25.762318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.315055 (+1552737us) service_pool.cc:224] Handling call
0709 13:04:27.315523 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.409144 (+ 93621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.815109 (+405965us) spinlock_profiling.cc:243] Waited 92.9 ms on lock 0x7b540031ae54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:27.815370 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213635840}
I20260709 13:04:27.748075 14550 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:27.749089 14432 consensus_meta.cc:287] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.520s	user 0.014s	sys 0.000s
I20260709 13:04:27.762054  8676 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:27.764513  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2360) took 2520 ms. Trace:
I20260709 13:04:27.764751  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:27.755312  8825 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:27.858182  8825 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:27.620186 14537 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.765148  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:27.871376  8681 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 18: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:04:27.769883  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.842562  8821 rpcz_store.cc:276] 0709 13:04:25.240290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.287747 (+ 47457us) service_pool.cc:224] Handling call
0709 13:04:25.299289 (+ 11542us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.844284 (+544995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.760134 (+1915850us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b540016a654. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:27.760445 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1172345216}
W20260709 13:04:27.750870  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 177374us
I20260709 13:04:27.854256  8680 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 8.
I20260709 13:04:27.776158 14435 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:27.776983  8678 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:27.776113 14174 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:27.779918  8673 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:27.875573  8678 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:27.790501  8674 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:27.793814 14439 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:27.795815 14480 consensus_queue.cc:1059] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:27.796953  8719 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:27.898468  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2450) took 2136 ms. Trace:
I20260709 13:04:27.763059 14548 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:27.829638  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1060) took 1718 ms. Trace:
I20260709 13:04:27.829792  8751 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:27.900635  8752 rpcz_store.cc:276] 0709 13:04:26.110908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.743573 (+1632665us) service_pool.cc:224] Handling call
0709 13:04:27.743959 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.825620 (+ 81661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.829570 (+  3950us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:27.901065  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1063) took 1715 ms. Trace:
I20260709 13:04:27.901463  8751 rpcz_store.cc:276] 0709 13:04:26.185469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.825155 (+1639686us) service_pool.cc:224] Handling call
0709 13:04:27.829603 (+  4448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.900782 (+ 71179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.901004 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:27.902007  8752 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:27.830333  8672 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:27.902601  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1065) took 1543 ms. Trace:
I20260709 13:04:27.761109 14422 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.832406  8679 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:27.833388  8747 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:27.834074  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1064) took 1516 ms. Trace:
I20260709 13:04:27.758370  8647 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:27.908658  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 150376us
I20260709 13:04:27.909619  8647 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:27.728340 14534 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:04:27.860095  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:27.771921  8677 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:27.774137  9382 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:27.881747  8822 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:27.884837  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:27.892496  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:27.923468  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 188436us
I20260709 13:04:27.907147  8756 rpcz_store.cc:276] 0709 13:04:26.318065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.833102 (+1515037us) service_pool.cc:224] Handling call
0709 13:04:27.833453 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.833671 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.833707 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:27.833757 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:27.833801 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.833875 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.834000 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:27.924865  9355 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:27.871872 14432 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:27.819748 14556 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.121s
W20260709 13:04:27.894061  8826 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2370) took 2634 ms (client timeout 3089 ms). Trace:
I20260709 13:04:27.896154 14537 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:27.896997 14524 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 1.042s	user 0.003s	sys 0.000s
I20260709 13:04:27.767524 14394 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.104s
I20260709 13:04:27.582787 14549 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.898903  8748 rpcz_store.cc:276] 0709 13:04:25.758651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.828162 (+1069511us) service_pool.cc:224] Handling call
0709 13:04:26.828625 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.462044 (+633419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.890307 (+428263us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b54002fdad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:27.894568 (+  4261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1454778112}
I20260709 13:04:27.945492  8748 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:27.946235  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2835) took 1394 ms. Trace:
I20260709 13:04:27.946563  8748 rpcz_store.cc:276] 0709 13:04:26.551484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.944887 (+1393403us) service_pool.cc:224] Handling call
0709 13:04:27.945285 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.945927 (+   642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.946173 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:27.948029  8748 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:27.948679  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2837) took 1396 ms. Trace:
I20260709 13:04:27.949010  8748 rpcz_store.cc:276] 0709 13:04:26.552219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.947465 (+1395246us) service_pool.cc:224] Handling call
0709 13:04:27.947830 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.948427 (+   597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.948621 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:27.903317  8752 rpcz_store.cc:276] 0709 13:04:26.359080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.901534 (+1542454us) service_pool.cc:224] Handling call
0709 13:04:27.901836 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.902377 (+   541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.902549 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:27.904364  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 107503us
I20260709 13:04:27.959188  8752 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:27.959900  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2838) took 1352 ms. Trace:
I20260709 13:04:27.960224  8752 rpcz_store.cc:276] 0709 13:04:26.607700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.958396 (+1350696us) service_pool.cc:224] Handling call
0709 13:04:27.958805 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.959621 (+   816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.959831 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:27.902969  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1066) took 1543 ms. Trace:
I20260709 13:04:27.912389  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:27.961210  8751 rpcz_store.cc:276] 0709 13:04:26.359517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.902175 (+1542658us) service_pool.cc:224] Handling call
0709 13:04:27.902463 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.902681 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.902706 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:27.902743 (+    37us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:27.902775 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.902824 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.902918 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:27.912760  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:27.963129  8680 raft_consensus.cc:2376] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 8. Re-sending same reply.
I20260709 13:04:27.916656 14550 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:27.876400 14539 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:27.928912  9381 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:27.933449  8821 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:04:27.931041  8826 rpcz_store.cc:269] 0709 13:04:25.259790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.279428 (+ 19638us) service_pool.cc:224] Handling call
0709 13:04:27.893810 (+2614382us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b540016ab54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:27.893995 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4789519616}
W20260709 13:04:27.931428  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 188433us
I20260709 13:04:27.929395  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:27.943224  8722 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:27.924521 14423 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:27.966060  8755 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:27.931581 14435 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:27.988046  8751 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:04:27.989550  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46459.8us
I20260709 13:04:28.000804  8752 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:27.926784 14560 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:27.938602 13702 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:28.000420  8719 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:27.941543 14538 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.944002 14548 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:04:27.966001 14537 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
W20260709 13:04:28.101450  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.003371 14454 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:28.156038  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 271319us
I20260709 13:04:28.156733  8828 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:28.069537  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:28.060726  8679 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:28.157953  8830 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:28.019382  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:28.043967  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 100992us
W20260709 13:04:28.071440  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 141962us
I20260709 13:04:28.047794  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2830) took 2201 ms. Trace:
I20260709 13:04:28.041507  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:28.161149  8749 rpcz_store.cc:276] 0709 13:04:25.846339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.421549 (+1575210us) service_pool.cc:224] Handling call
0709 13:04:27.424680 (+  3131us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.440728 (+ 16048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.440774 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.441460 (+   686us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.702036 (+260576us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:27.702086 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.702167 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.047486 (+345319us) raft_consensus.cc:1685] finished
0709 13:04:28.047515 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.047692 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18995,"raft.queue_time_us":17060,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":4901,"thread_start_us":177347,"threads_started":1,"wal-append.queue_time_us":444673}
I20260709 13:04:28.073011  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:27.986054 14423 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:28.169303  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 168973us
I20260709 13:04:28.044129  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:28.172072  8821 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:28.159922  8680 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:28.125869  8678 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:27.988996 14480 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.003513 14558 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:28.036123 14486 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 2.155s	user 0.000s	sys 0.019s
I20260709 13:04:26.041458  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:28.180934  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:28.211376 14511 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:28.180157 14423 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:28.212402  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:28.214061  9387 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:04:28.214708  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 200661us
I20260709 13:04:28.219897  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:28.221094  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:28.213603  8575 catalog_manager.cc:5697] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:28.048594 14560 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:04:28.384068  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 226644us
I20260709 13:04:28.440526  8645 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:28.402644  8827 raft_consensus.cc:2410] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:04:28.230820  8830 raft_consensus.cc:2410] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:04:28.032768 14435 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:28.178764 14540 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:28.124073 14550 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:28.477852  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:28.041522 14439 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.181s
W20260709 13:04:28.479238  8753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 1055) took 2428 ms (client timeout 2078 ms). Trace:
I20260709 13:04:28.035130 14519 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.350s
W20260709 13:04:28.465364 14453 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 3.298s	user 0.002s	sys 0.008s
I20260709 13:04:28.530969 14438 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:04:28.479882  8753 rpcz_store.cc:269] 0709 13:04:26.014542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.490325 (+475783us) service_pool.cc:224] Handling call
0709 13:04:28.442115 (+1951790us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54002fdfd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:28.442299 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3408375040}
I20260709 13:04:28.385993 14457 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:28.512915 14540 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:28.219367 14480 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:28.534869  9353 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.023s	sys 0.058s
W20260709 13:04:28.543762  9353 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.023s	sys 0.058s
I20260709 13:04:28.495460  8826 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:28.517325 14511 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:28.537688 14422 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:28.564810  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:28.535526 14457 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:28.561609 14511 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.567641 14511 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:28.570827  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:28.572333 14457 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:28.537195 14540 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:28.506701  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:28.578222  8677 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:28.485190 14566 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:04:28.697021  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 84444.1us
I20260709 13:04:28.699753  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:28.711815 14572 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.712644 14572 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:28.716192 14454 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.001s	sys 0.008s
W20260709 13:04:28.716933 14454 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.001s	sys 0.008s
I20260709 13:04:28.717319 14454 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:28.718582 14454 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:28.719599 14539 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:28.721328 14560 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:28.724916 14382 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.985s	user 0.003s	sys 0.007s
I20260709 13:04:28.516677 14550 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:28.537113 14480 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:28.739248  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:28.562562 14567 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:28.747478  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2833) took 2196 ms. Trace:
I20260709 13:04:28.609503 14540 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.748253  8756 rpcz_store.cc:276] 0709 13:04:26.550927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.936716 (+1385789us) service_pool.cc:224] Handling call
0709 13:04:27.937196 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.937513 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:28.260816 (+323303us) log.cc:844] Serialized 14 byte log entry
0709 13:04:28.746966 (+486150us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:28.747046 (+    80us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:28.747112 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.747206 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.747394 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":771052,"thread_start_us":791953,"threads_started":2}
I20260709 13:04:28.771941  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:28.562871 14519 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.179s
I20260709 13:04:28.786065  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 1054) took 1003 ms. Trace:
I20260709 13:04:28.794169  8677 rpcz_store.cc:276] 0709 13:04:27.782284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.771587 (+989303us) service_pool.cc:224] Handling call
0709 13:04:28.785950 (+ 14363us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:28.741308 14394 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.013s	sys 0.025s
I20260709 13:04:28.614421 14563 consensus_queue.cc:1059] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.177s
I20260709 13:04:28.740996 14432 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:28.762353 14567 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.709227 14439 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:04:28.802904  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:28.811390  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 1048) took 1030 ms. Trace:
I20260709 13:04:28.812065  8677 rpcz_store.cc:276] 0709 13:04:27.780516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.802445 (+1021929us) service_pool.cc:224] Handling call
0709 13:04:28.811287 (+  8842us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:28.813117  8750 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:28.813552  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:28.814783  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 1056) took 1031 ms. Trace:
W20260709 13:04:28.816128  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 375668us
I20260709 13:04:28.815984  8677 rpcz_store.cc:276] 0709 13:04:27.782854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.813118 (+1030264us) service_pool.cc:224] Handling call
0709 13:04:28.814695 (+  1577us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:28.742442 14382 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
W20260709 13:04:28.799199 14394 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.013s	sys 0.025s
I20260709 13:04:28.817430  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:28.823671  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 1052) took 1041 ms. Trace:
W20260709 13:04:28.796609 14548 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.020s	sys 0.004s
I20260709 13:04:28.820043  8645 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:28.824311  8677 rpcz_store.cc:276] 0709 13:04:27.781771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.817142 (+1035371us) service_pool.cc:224] Handling call
0709 13:04:28.823573 (+  6431us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:28.825271  8721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.016s	sys 0.016s
I20260709 13:04:28.800691 14566 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.818150 14478 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.026s	sys 0.028s
I20260709 13:04:28.837872  8645 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:28.819471  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 599606us
W20260709 13:04:28.827857  8721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.017s	sys 0.017s
W20260709 13:04:28.841094  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.805452 14538 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:28.815954 14505 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.809715 14567 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:28.785593 14540 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:28.819940 14575 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.829926 14548 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.020s	sys 0.005s
W20260709 13:04:28.836467 14478 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.026s	sys 0.029s
I20260709 13:04:28.773550 14573 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:04:28.803447 14571 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:04:28.833493 14432 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:04:28.888988  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 190370us
I20260709 13:04:28.856761 14539 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:28.889014  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 484758us
I20260709 13:04:28.821322 14423 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:28.890282  8719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.015s	sys 0.030s
W20260709 13:04:28.897082  8719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.015s	sys 0.030s
I20260709 13:04:28.898717  8756 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:04:28.899269  8756 pending_rounds.cc:85] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 12
I20260709 13:04:28.899504  8756 pending_rounds.cc:107] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 17.13
W20260709 13:04:28.899664 14449 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.156s	user 0.013s	sys 0.000s
I20260709 13:04:28.898416 14573 raft_consensus.cc:3055] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:28.903285 14573 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:28.904429 14573 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:28.905927 14573 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:28.899736  8756 raft_consensus.cc:2889] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:28.908810 14174 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:28.909742 14558 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:28.912340  8680 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:28.898554  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 727645us
I20260709 13:04:28.886512 14572 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:28.922942  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 85202.9us
W20260709 13:04:28.897207  9353 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 2.078s (SENT) [suppressed 51 similar messages]
I20260709 13:04:28.920639  8719 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:28.921046  8749 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:04:28.921999  8753 raft_consensus.cc:3055] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:28.943228  8753 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:28.810489 14435 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:28.928330  8748 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:28.921094  8754 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:28.946930  8753 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:28.948335  8753 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:28.897941 14580 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:28.879981 14505 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:28.966364  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 45794.6us
I20260709 13:04:28.968465  8719 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:28.970017  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 46099.5us
I20260709 13:04:28.973048  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:28.894483 14432 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:28.944972 14439 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.951550  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 58409.5us
I20260709 13:04:28.941484  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2432) took 1164 ms. Trace:
I20260709 13:04:28.975198  8679 rpcz_store.cc:276] 0709 13:04:27.776956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.912003 (+135047us) service_pool.cc:224] Handling call
0709 13:04:28.941169 (+1029166us) spinlock_profiling.cc:243] Waited 99 ms on lock 0x7b5400299354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:28.941383 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227725056}
I20260709 13:04:28.959280  8825 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
W20260709 13:04:28.964458 14582 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:04:28.922168 14558 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.849645 14566 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:28.976879 14560 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:28.974916  8645 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:28.978271  8719 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:28.980440  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1045) took 1202 ms. Trace:
I20260709 13:04:28.984757  8680 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
W20260709 13:04:28.989030  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.985725  8753 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:04:28.989627  8753 pending_rounds.cc:85] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 11
I20260709 13:04:28.989914  8753 pending_rounds.cc:107] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:04:28.990171  8753 raft_consensus.cc:2889] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:28.933784 14572 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:04:28.889824 14575 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:28.988222  8677 rpcz_store.cc:276] 0709 13:04:27.778156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.826679 (+1048523us) service_pool.cc:224] Handling call
0709 13:04:28.827178 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:28.860709 (+ 33531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:28.860823 (+   114us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:28.888900 (+ 28077us) log.cc:844] Serialized 26 byte log entry
0709 13:04:28.980035 (+ 91135us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:28.980079 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.980154 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.980201 (+    47us) raft_consensus.cc:1685] finished
0709 13:04:28.980212 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.980374 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49855,"raft.queue_time_us":65872,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":31660,"thread_start_us":53823,"threads_started":1,"wal-append.queue_time_us":53594}
I20260709 13:04:28.969548 14583 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:28.995126  8677 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:29.176059  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1047) took 1397 ms. Trace:
I20260709 13:04:29.176707  8677 rpcz_store.cc:276] 0709 13:04:27.778776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.994440 (+1215664us) service_pool.cc:224] Handling call
0709 13:04:28.994864 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.175597 (+180733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.175967 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.006721 14599 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:29.182417  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 207553us
I20260709 13:04:29.009778  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:29.010169  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.193427  9384 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
W20260709 13:04:29.195674  9352 connection.cc:256] RPC call timeout handler was delayed by -1.3594135319999623s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
I20260709 13:04:28.966965 14580 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.033219 14478 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:29.068704 14601 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:04:29.181360 14585 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.215s	user 0.000s	sys 0.000s
I20260709 13:04:29.195751  8677 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:29.261220  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1049) took 1480 ms. Trace:
I20260709 13:04:29.174965 14572 consensus_queue.cc:1243] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:04:29.240974  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55576.4us
W20260709 13:04:29.240693 14511 consensus_meta.cc:287] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.672s	user 0.012s	sys 0.001s
I20260709 13:04:29.261869  8677 rpcz_store.cc:276] 0709 13:04:27.780924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.195011 (+1414087us) service_pool.cc:224] Handling call
0709 13:04:29.195506 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.260718 (+ 65212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.261114 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.015609  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:29.175979 14550 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:29.202407 14560 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:29.286542  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1050) took 1505 ms. Trace:
I20260709 13:04:29.175524 14590 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:29.178156 14454 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:04:29.246913 14534 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:29.287549  8677 rpcz_store.cc:276] 0709 13:04:27.781233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.284977 (+1503744us) service_pool.cc:224] Handling call
0709 13:04:29.285746 (+   769us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.286016 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.286057 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:29.286121 (+    64us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:29.286176 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.286253 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.286448 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.299222  8677 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:29.300045  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1051) took 1518 ms. Trace:
I20260709 13:04:29.300446  8677 rpcz_store.cc:276] 0709 13:04:27.781491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.298243 (+1516752us) service_pool.cc:224] Handling call
0709 13:04:29.298731 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.299713 (+   982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.299969 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:28.985288 14439 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:28.994894 14558 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.259835 14465 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:29.305110  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1163) took 1128 ms. Trace:
I20260709 13:04:29.323412  8830 rpcz_store.cc:276] 0709 13:04:28.176585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.442147 (+265562us) service_pool.cc:224] Handling call
0709 13:04:28.442667 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:28.443142 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:28.443183 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:28.443847 (+   664us) log.cc:844] Serialized 26 byte log entry
0709 13:04:28.745033 (+301186us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:28.745081 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.745154 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.304743 (+559589us) raft_consensus.cc:1685] finished
0709 13:04:29.304777 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.305016 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1961,"raft.queue_time_us":977,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":19673,"thread_start_us":223284,"threads_started":1,"wal-append.queue_time_us":307288}
I20260709 13:04:29.302506  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1053) took 1520 ms. Trace:
I20260709 13:04:29.327241  8677 rpcz_store.cc:276] 0709 13:04:27.782025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.301389 (+1519364us) service_pool.cc:224] Handling call
0709 13:04:29.301812 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.302070 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.302109 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:29.302174 (+    65us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:29.302223 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.302295 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.302427 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.315232 14447 raft_consensus.cc:3055] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:29.316521 14534 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.284600 14511 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.319077  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1054) took 3308 ms. Trace:
I20260709 13:04:29.366793  8747 rpcz_store.cc:276] 0709 13:04:26.011052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.714731 (+1703679us) service_pool.cc:224] Handling call
0709 13:04:27.833198 (+118467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.906671 (+ 73473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.318786 (+1412115us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54001040d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:29.318994 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271160192}
I20260709 13:04:29.259719 14556 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:04:29.296763 14478 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.295323 14590 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:29.305361 14560 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:29.382254 14465 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.389349 14465 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.278298 14589 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.098s
I20260709 13:04:29.273073 14601 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.358174 14447 raft_consensus.cc:740] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:29.412747  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 515512us
I20260709 13:04:29.414327  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:29.401605 14605 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.125s	user 0.002s	sys 0.000s
W20260709 13:04:29.423152  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 380753us
I20260709 13:04:29.392433 14599 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:29.425166  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:29.299268  8679 raft_consensus.cc:2955] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 4 to 11, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:29.391552 14590 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:29.380993 14548 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:04:29.357735 14581 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:04:29.407827 14423 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:29.428620  9352 leader_election.cc:336] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.601s (SENT)
I20260709 13:04:29.439934  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:29.448251  9352 leader_election.cc:336] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.611s (SENT)
I20260709 13:04:29.391156 14478 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.415050 14556 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:29.444617  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:29.455139  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:29.440196  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 424691us
I20260709 13:04:29.457718 14447 consensus_queue.cc:260] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.447557 14603 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:29.458482  8719 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:29.428192 14599 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.460894 14599 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:29.456533 14489 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.457054  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:29.457099 14572 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:29.441536  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 111743us
I20260709 13:04:29.460012 14603 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:29.450112  9352 leader_election.cc:336] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.689s (SENT)
I20260709 13:04:29.455061 14556 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:29.430423 14540 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:29.480372  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.476418  9352 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:29.479063 14423 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:29.478394 14566 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:29.463045 14489 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.503439 14511 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:29.478739 14572 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.558486  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1046) took 1780 ms. Trace:
I20260709 13:04:29.502421 14573 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
W20260709 13:04:29.563196  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.484709 14560 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.562587  8679 rpcz_store.cc:276] 0709 13:04:27.778437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.983705 (+1205268us) service_pool.cc:224] Handling call
0709 13:04:28.984390 (+   685us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:28.984792 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:28.984823 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:28.985726 (+   903us) log.cc:844] Serialized 494 byte log entry
0709 13:04:29.298869 (+313143us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:29.470429 (+171560us) log.cc:844] Serialized 14 byte log entry
0709 13:04:29.557967 (+ 87538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.558104 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.558196 (+    92us) raft_consensus.cc:1685] finished
0709 13:04:29.558215 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.558418 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":399694,"threads_started":2,"wal-append.queue_time_us":406463}
I20260709 13:04:29.542069 14575 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:29.482547 14587 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.269s
I20260709 13:04:29.549753 14394 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:29.538260 14519 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:04:29.572389  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1061) took 1780 ms. Trace:
I20260709 13:04:29.537246 14609 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:04:29.575287  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:29.574043  8677 rpcz_store.cc:276] 0709 13:04:27.792080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.329120 (+1537040us) service_pool.cc:224] Handling call
0709 13:04:29.410899 (+ 81779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.411195 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:29.465881 (+ 54686us) log.cc:844] Serialized 14 byte log entry
0709 13:04:29.571994 (+106113us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:29.572052 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:29.572102 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.572178 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.572325 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":134704,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":3604,"thread_start_us":141568,"threads_started":3,"wal-append.queue_time_us":95159}
I20260709 13:04:29.590332  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:29.591910 14423 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:29.592479 14622 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:29.592921 14622 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:29.593185 14423 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.593955 14622 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:29.576674 14592 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.273s	user 0.002s	sys 0.000s
I20260709 13:04:29.600667  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.553457 14556 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.603209 14556 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:29.557724 14603 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.607448 14519 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:29.610404  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.611666  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:29.611974 14602 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.612360 14478 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:29.612838  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.612532 14602 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.613484  8679 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:29.613574 14534 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:29.614338 14598 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:04:29.614889 14622 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:29.577193 14558 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:29.616230  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.617436  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.614379  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:29.623553 14541 log.cc:927] Time spent T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 2.168s	user 0.008s	sys 0.009s
I20260709 13:04:29.626827  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.631996  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:29.603225  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.635362  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.636080  8677 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:29.636152  8679 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:29.637097  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.637820  8677 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
W20260709 13:04:29.625052  8680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 1059) took 1833 ms (client timeout 1831 ms). Trace:
W20260709 13:04:29.639276  8680 rpcz_store.cc:269] 0709 13:04:27.791489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.185201 (+393712us) service_pool.cc:224] Handling call
0709 13:04:29.624848 (+1439647us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b5400298954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:29.624972 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1092750080}
I20260709 13:04:29.640516  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:29.632548 14542 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 2.177s	user 0.006s	sys 0.011s
I20260709 13:04:29.649472  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:29.664498  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:29.665932  8680 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:29.667609  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.663494 14489 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:29.677356  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:29.679078  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:29.679860  8679 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:29.681851 14467 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:29.684576 14576 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:29.685214 14576 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.687814 14576 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:29.679852  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.696632  8679 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
W20260709 13:04:29.697875  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 160655us
I20260709 13:04:29.703632  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.708962 14572 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:29.716056 14565 log.cc:927] Time spent T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 1.511s	user 0.000s	sys 0.028s
I20260709 13:04:29.720034 14439 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:29.720530  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:29.720276 14599 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.722162 14517 raft_consensus.cc:3055] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:29.722594 14517 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:29.723206 14602 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:29.723811 14602 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.723567 14517 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.724287  8756 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:29.724344 14602 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.726423  8752 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:29.727340  8753 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:04:29.727998  8752 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:29.725004  8751 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:29.720726 14439 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.729780 14489 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.730235 14517 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:29.730365 14489 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.734515  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.737810  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.741622  8749 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:29.743295  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.744055  8679 raft_consensus.cc:2376] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14. Re-sending same reply.
I20260709 13:04:29.745038 14467 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.745715 14467 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.741171 14608 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:04:29.749280  8680 raft_consensus.cc:2410] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:04:29.750845 14544 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:29.751606 14544 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:29.753665 14567 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.864s	user 0.026s	sys 0.000s
I20260709 13:04:29.756567 14626 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.757084 14602 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:29.757378 14626 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.759009 13702 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:29.702093 14581 consensus_queue.cc:1243] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:29.759440 14608 consensus_queue.cc:1059] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:04:29.766350  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:29.766810 14628 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:29.767115 14626 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:29.735725 14599 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:29.754395 14544 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:29.769237 14583 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.527s	user 0.006s	sys 0.019s
I20260709 13:04:29.732803 14489 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:29.766451 14467 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:29.754216 14567 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.777760  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.765213 14556 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.807209 14556 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.807840  8575 catalog_manager.cc:5697] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 8 to 9, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 9 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:29.808825  8572 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43478 (request call id 257) took 23785 ms (client timeout 15000 ms). Trace:
I20260709 13:04:29.729782 14439 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.789253  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:29.772724  8677 raft_consensus.cc:2376] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14. Re-sending same reply.
W20260709 13:04:29.803764  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.805039  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:29.806437  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.816637  8679 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:29.817806  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.823065 14581 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.818568  8679 raft_consensus.cc:2368] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16.
I20260709 13:04:29.827841  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.827719  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.828958  8677 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:29.830356  8679 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 14: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:29.831738  8674 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:29.812527 14583 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:29.846196  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.847962  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.852393  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.823884 14581 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.861153 14393 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:29.865582  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:29.867210  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:29.867956  8753 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:29.868250 14567 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:29.868007 14590 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.870718  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:29.869102  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:29.873479  8678 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:29.873811  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:29.875244  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:29.913106  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:29.919862  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2409) took 2281 ms. Trace:
I20260709 13:04:29.920608 14608 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:29.920564  8675 rpcz_store.cc:276] 0709 13:04:27.661912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.662492 (+   580us) service_pool.cc:224] Handling call
0709 13:04:27.662984 (+   492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.776677 (+113693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.776713 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:27.776758 (+    45us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:27.776802 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.919458 (+2142656us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54001f9d54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:29.919514 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.919726 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260917504}
I20260709 13:04:29.922533 14608 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:29.923738 14307 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:29.924401 14307 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:29.907644 14380 raft_consensus.cc:3055] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:29.925745 14380 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:29.926137  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:29.900449 14564 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.868s	user 0.011s	sys 0.021s
I20260709 13:04:29.928150 14608 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:29.897727  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2377) took 2341 ms. Trace:
I20260709 13:04:29.901147 14439 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:29.889851 14581 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:29.918710  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:29.930384  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.929687 14380 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.925581 14307 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:29.932046 14307 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:29.929381  8822 rpcz_store.cc:276] 0709 13:04:27.555880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.824687 (+268807us) service_pool.cc:224] Handling call
0709 13:04:27.832385 (+  7698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.922892 (+ 90507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.897364 (+1974472us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b540016bf54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:29.897637 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112226560}
W20260709 13:04:29.933152  8754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2458) took 2710 ms (client timeout 1856 ms). Trace:
I20260709 13:04:29.935392  8753 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:29.936621 14505 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:29.938156  8754 rpcz_store.cc:269] 0709 13:04:27.223101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.245015 (+ 21914us) service_pool.cc:224] Handling call
0709 13:04:29.932956 (+2687941us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b540031d654. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:29.933077 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3804409216}
I20260709 13:04:29.939409  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:29.908979  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:29.941747  8681 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:29.876494 14566 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:29.942091 14382 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:29.922768  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.945381  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.945680  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:29.946236  8675 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:29.946309  8674 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:29.947364  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.911926  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:53352 (request call id 2408) took 3021 ms. Trace:
I20260709 13:04:29.951071  8719 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:29.950930  8678 rpcz_store.cc:276] 0709 13:04:26.904340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.642785 (+738445us) service_pool.cc:224] Handling call
0709 13:04:29.858466 (+2215681us) spinlock_profiling.cc:243] Waited 70.2 ms on lock 0x7b54001f9d54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f0a3d 00007f32e74c0cd0 00007f32e0a45254 00007f32e0a451e2 00007f32e0a45111 00007f32e0a4508c 00007f32e0a44342 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:29.911258 (+ 52792us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161429248}
I20260709 13:04:29.942780 14566 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:29.943281 14580 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:29.955794  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1161) took 2235 ms. Trace:
I20260709 13:04:29.955528 14566 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.956259  8825 rpcz_store.cc:276] 0709 13:04:27.720363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.864923 (+144560us) service_pool.cc:224] Handling call
0709 13:04:27.865345 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:28.976371 (+1111026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.955512 (+979141us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400126454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:29.955724 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2514956160}
I20260709 13:04:29.957219 14566 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:29.957938  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1068) took 3404 ms. Trace:
I20260709 13:04:29.958370  8748 rpcz_store.cc:276] 0709 13:04:26.553423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.949747 (+1396324us) service_pool.cc:224] Handling call
0709 13:04:27.950077 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:28.952290 (+1002213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.957651 (+1005361us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b5400139554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:29.957866 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2249172736}
I20260709 13:04:29.959756  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.960139  8748 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:29.960424  8675 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:29.962857  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2380) took 2399 ms. Trace:
I20260709 13:04:29.963266  8828 rpcz_store.cc:276] 0709 13:04:27.563442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.827777 (+264335us) service_pool.cc:224] Handling call
0709 13:04:27.877851 (+ 50074us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:28.157143 (+279292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.962599 (+1805456us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b5400267ad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:29.962793 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640832768}
I20260709 13:04:29.964747 14507 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.964943  9352 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:29.965335 14507 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:29.965904 14608 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:29.912021  8679 raft_consensus.cc:2376] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12. Re-sending same reply.
W20260709 13:04:29.809329  8572 rpcz_store.cc:269] 0709 13:04:06.023576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.072761 (+ 49185us) service_pool.cc:224] Handling call
0709 13:04:29.808554 (+23735793us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:19.116525 (+     0us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32df3ef0ac 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df409455 00007f32e5f2e16c
0709 13:04:19.231005 (+114480us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:19.251726 (+ 20721us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:19.251778 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:04:19.251800 (+    22us) tablet.cc:662] Decoding operations
0709 13:04:19.499326 (+247526us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:19.515184 (+ 15858us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:19.520684 (+  5500us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:04:19.594818 (+ 74134us) tablet.cc:701] Row locks acquired
0709 13:04:19.594853 (+    35us) write_op.cc:260] PREPARE: finished
0709 13:04:19.595068 (+   215us) write_op.cc:270] Start()
0709 13:04:19.595311 (+   243us) write_op.cc:276] Timestamp: P: 1783602259594974 usec, L: 0
0709 13:04:19.595340 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:04:19.596248 (+   908us) log.cc:844] Serialized 8642 byte log entry
0709 13:04:20.232071 (+635823us) op_driver.cc:464] REPLICATION: finished
0709 13:04:20.469340 (+237269us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32df3ef0ac 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e5f2fef0
0709 13:04:21.258970 (+789630us) write_op.cc:301] APPLY: starting
0709 13:04:21.287856 (+ 28886us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:21.709502 (+421646us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b54002d7dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f632a 00007f32e247bf14 00007f32e247bec3 00007f32e247be38 00007f32e247bdf3 00007f32e247b0dc 00007f32e24a4e16 00007f32e248ccbb 00007f32e247739d 00007f32e247ad75
0709 13:04:21.808581 (+ 99079us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b180048cd60. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e22380ec 00007f32e2284c0a 00007f32e22844f7 00007f32e2284490 00007f32e223bf48 00007f32e5f501bb 00007f32e5f50532 00007f32e5f97b93 00007f32e5e44de3
0709 13:04:22.098205 (+289624us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:22.098279 (+    74us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:22.486187 (+387908us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:22.594368 (+108181us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=0,mrs_lookups=0
0709 13:04:22.603983 (+  9615us) write_op.cc:312] APPLY: finished
0709 13:04:22.795090 (+191107us) log.cc:844] Serialized 196 byte log entry
0709 13:04:23.026683 (+231593us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:23.115439 (+ 88756us) write_op.cc:454] Released schema lock
0709 13:04:23.115541 (+   102us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:23.116317 (+   776us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:23.116351 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":392508,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":926587,"apply.run_cpu_time_us":77987,"apply.run_wall_time_us":1940318,"cfile_cache_hit":90,"cfile_cache_hit_bytes":95999,"num_ops":22,"prepare.queue_time_us":384344,"prepare.run_cpu_time_us":31152,"prepare.run_wall_time_us":685103,"raft.queue_time_us":87832,"raft.run_cpu_time_us":91408,"raft.run_wall_time_us":1502633,"replication_time_us":636465,"spinlock_wait_cycles":65845632,"thread_start_us":1845107,"threads_started":5,"wal-append.queue_time_us":877505,"wal-append.run_cpu_time_us":31074,"wal-append.run_wall_time_us":2132583}]]}
I20260709 13:04:29.952294 14382 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:29.967875  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.968289  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:29.968612  8679 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:29.961416  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:29.969713  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.942255 14601 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:29.940138  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1060) took 2148 ms. Trace:
W20260709 13:04:29.954377  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.953884  8681 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:29.937062  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:29.970569  8679 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 9: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:30.023443  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:30.024235  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:30.025691 14601 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:29.971868  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.972290 14382 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:29.973642  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:29.971128  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:29.977998  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:29.976025  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.977667  8676 rpcz_store.cc:276] 0709 13:04:27.791804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.317369 (+1525565us) service_pool.cc:224] Handling call
0709 13:04:29.939387 (+622018us) spinlock_profiling.cc:243] Waited 9.13 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:04:29.939593 (+   206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.939794 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.939821 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:29.939867 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:29.939902 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.939959 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.940078 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20991872}
I20260709 13:04:29.992029  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:30.026603  8675 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:04:30.040663  8673 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:29.992904  8822 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:04:30.018603  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:29.993424  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:29.972240  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:30.043097  8672 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:29.993645  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:29.994769  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:30.012156  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:30.045264  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:30.045394  8681 raft_consensus.cc:2410] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:04:30.045900  8830 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:29.995843  8829 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:30.050952  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:30.051610  8680 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:30.051776  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:30.052446  8829 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:30.052613  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:30.053571  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:29.996343  8826 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:30.055986  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:30.057367  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:29.997191  8823 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:30.059020  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:30.059696 14394 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.001583  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:29.978767 14456 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.006665  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:30.061568  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 60163.1us
I20260709 13:04:30.061996 14627 raft_consensus.cc:3055] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:30.062376 14627 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:30.063499 14437 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:30.015352  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:30.063413 14627 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:30.064237 14437 raft_consensus.cc:697] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:30.065090 14627 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:30.065496 14556 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:30.065375 14437 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:30.066828 14462 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:30.016299  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:30.000110 14603 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:30.067885 14444 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:30.068368 14444 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:30.068835  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:30.069520  8753 raft_consensus.cc:1813] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Handling vote request from an unknown peer e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:30.069379 14444 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:30.069877 14447 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:29.972970  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:30.025736  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:30.017100  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:29.975654  9382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 759) took 2508 ms. Trace:
W20260709 13:04:30.067688  8750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2437) took 4354 ms (client timeout 1687 ms). Trace:
I20260709 13:04:30.043496  9388 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:30.073055  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:29.991410 14540 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:29.972873 14546 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 2.520s	user 0.000s	sys 0.024s
I20260709 13:04:29.987916 14548 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:04:29.984958 14579 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.095s	user 0.001s	sys 0.002s
I20260709 13:04:29.976714 14465 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:29.987762 14581 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.047164 14580 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:29.973227 14405 consensus_queue.cc:1059] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 13:04:30.049551  8747 raft_consensus.cc:2393] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 14: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:30.026736 14601 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:04:29.991312 14422 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:30.050275  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:30.059923  8823 raft_consensus.cc:2410] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:30.027210 14382 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.070035  8753 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 14: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:30.073968  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:30.054383  8829 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:30.075203  8756 raft_consensus.cc:2410] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:04:30.038311  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 65635us
I20260709 13:04:30.122884  8822 raft_consensus.cc:2376] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14. Re-sending same reply.
I20260709 13:04:30.131047  8719 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:30.074003  8750 rpcz_store.cc:269] 0709 13:04:25.712957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.715132 (+  2175us) service_pool.cc:224] Handling call
0709 13:04:30.067407 (+4352275us) spinlock_profiling.cc:243] Waited 2.94 s on lock 0x7b540018d8d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:30.067569 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6765267840}
I20260709 13:04:30.132488  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:30.133165  8750 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
W20260709 13:04:30.039259  8793 connection.cc:256] RPC call timeout handler was delayed by -1.7158108869999751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30410000000000004s for a response
I20260709 13:04:30.073704  9382 rpcz_store.cc:276] 0709 13:04:27.467673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.467855 (+   182us) service_pool.cc:224] Handling call
0709 13:04:27.529526 (+ 61671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.922855 (+393329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.975371 (+2052516us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b54005ea1d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:29.975558 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562061056}
I20260709 13:04:30.076333  9352 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:30.080703  8676 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:04:29.983253  8825 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:30.137740  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1064) took 1161 ms. Trace:
I20260709 13:04:30.138306  8676 rpcz_store.cc:276] 0709 13:04:28.976067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.075968 (+1099901us) service_pool.cc:224] Handling call
0709 13:04:30.080298 (+  4330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.137287 (+ 56989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.137654 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.992583  8828 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:30.140019  8676 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:29.987589  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:30.141837  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:29.994236 14561 raft_consensus.cc:3055] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:04:29.884858  8645 leader_election.cc:336] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.687s (SENT)
I20260709 13:04:29.977448 14507 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:30.060329 14394 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:30.125175  8823 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:30.126350 14422 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:30.118191 14444 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:30.143322  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 155836us
I20260709 13:04:30.185825  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:30.144326  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1065) took 1159 ms. Trace:
I20260709 13:04:30.085662 14548 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:30.146319  8756 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:30.147711  8753 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 13:04:30.189846  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 48126.6us
I20260709 13:04:30.191257  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:30.192232  8719 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:30.194556  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 118334us
I20260709 13:04:30.195065  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:30.154990  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:30.155825  8673 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:30.196215  8673 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:30.159832  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2457) took 4397 ms. Trace:
I20260709 13:04:30.198366  8755 rpcz_store.cc:276] 0709 13:04:25.762579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.409089 (+1646510us) service_pool.cc:224] Handling call
0709 13:04:27.422928 (+ 13839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.986483 (+563555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.159496 (+2173013us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b540038aed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:30.159720 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1248527744}
I20260709 13:04:30.162488  8747 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:30.163162 14432 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:30.200966  8829 raft_consensus.cc:2410] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:04:30.201838  8825 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:30.085312 14540 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:30.154012 14580 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:30.248710  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:30.251005  9384 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:30.147583  8645 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:29.897919 14590 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:30.140777 14382 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:30.176047  8674 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 9.
I20260709 13:04:30.204720  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:30.260306  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:30.260855  8677 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:30.187263  8676 rpcz_store.cc:276] 0709 13:04:28.985090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.139287 (+1154197us) service_pool.cc:224] Handling call
0709 13:04:30.139623 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.143911 (+  4288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.144242 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.061493 14456 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:30.214331  8756 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:30.152280 14561 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:30.214592  8719 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:30.146903 14566 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:30.112823 14462 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:04:30.172339  8681 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:04:30.112151 14581 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:30.224303  9381 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:30.197124  8673 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:30.287118  8673 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:30.114212 14168 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:04:30.265316  8675 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:30.263952  8680 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:30.295640 14580 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:30.300033 14580 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:30.304359  8826 raft_consensus.cc:2410] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:04:30.254285 14432 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:04:30.305833  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 164461us
W20260709 13:04:30.273887  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 423913us
I20260709 13:04:30.188684 14505 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.188951 14465 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.265134 14307 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:30.210873 14556 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:30.282707  8748 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:30.220007 14353 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 13:04:30.210712 14511 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:30.211046 14489 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:30.210295 14439 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:30.340487  8793 connection.cc:256] RPC call timeout handler was delayed by -1.689561880000042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
I20260709 13:04:30.310176  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:30.276564  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 165764us
I20260709 13:04:30.205785 14589 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:04:30.315973 14583 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:30.345738  9387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 1052) took 2448 ms. Trace:
I20260709 13:04:30.367970  8675 pending_rounds.cc:85] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 6
I20260709 13:04:30.368396  8675 pending_rounds.cc:107] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:04:30.322415  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:30.368777  8675 raft_consensus.cc:2889] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:30.277985 14537 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:30.249495 14539 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.094s
W20260709 13:04:30.292480  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 78149.8us
W20260709 13:04:30.376469  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.377691  8719 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:30.312297 14408 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:30.342634  8793 connection.cc:256] RPC call timeout handler was delayed by -1.5959618800000044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17920000000000003s for a response
I20260709 13:04:30.321206  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:30.319449 14507 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.280004  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1067) took 1287 ms. Trace:
I20260709 13:04:30.370239  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:30.385617  8681 rpcz_store.cc:276] 0709 13:04:28.992779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.168838 (+1176059us) service_pool.cc:224] Handling call
0709 13:04:30.170654 (+  1816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.279651 (+108997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.279934 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.392103  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:30.311916 14519 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:04:30.372910  8674 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:30.324877 14168 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.399183 14168 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:30.401888 14168 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:30.324684 14307 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:30.402841 14307 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:30.367244  9387 rpcz_store.cc:276] 0709 13:04:27.897804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.898047 (+   243us) service_pool.cc:224] Handling call
0709 13:04:27.902713 (+  4666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:28.437276 (+534563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.339863 (+1902587us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b54006e1754. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:30.345654 (+  5791us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":714751360}
I20260709 13:04:30.368472  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:30.408440  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:30.412259  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:30.413283  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 200829us
I20260709 13:04:30.414587 14459 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:30.418865  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 104930us
I20260709 13:04:30.420303  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1062) took 2627 ms. Trace:
I20260709 13:04:30.420771  8680 rpcz_store.cc:276] 0709 13:04:27.792384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.053819 (+2261435us) service_pool.cc:224] Handling call
0709 13:04:30.263749 (+209930us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.298562 (+ 34813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.420096 (+121534us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400174b50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:30.420237 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481541376}
W20260709 13:04:30.413451  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 116261us
I20260709 13:04:30.422555 14628 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:04:30.424696  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.424609 14628 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:30.432070 14481 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:30.432540 14481 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:30.433359 14481 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:30.442817 14566 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:30.443490 14566 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:30.449054  8676 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:30.497362  8676 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:30.343967 14439 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.340006 14432 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:30.341210 14489 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:30.323897 14505 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:30.366823 14511 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:30.369443 14476 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.913s	user 0.013s	sys 0.041s
I20260709 13:04:30.294538 14422 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:30.324007 14465 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:30.367127 14583 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:30.597644  8747 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:04:30.597918  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.574779  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:30.558300 14636 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:04:30.565618 14462 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:30.601558  8681 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:30.559818 14432 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:30.566776 14489 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:30.560544 14439 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:30.579123 14476 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.123s	user 0.013s	sys 0.042s
I20260709 13:04:30.592379  8680 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:30.563138 14168 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:30.335100 14548 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:30.585994 14624 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
W20260709 13:04:30.590946 14546 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:04:30.542706 14507 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:30.644806  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:30.568871 14611 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.357s
W20260709 13:04:30.288309 14435 consensus_queue.cc:1175] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7aff04f53d4845a6a4fa491fd2a06145" responder_term: 18 status { last_received { term: 18 index: 14 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:30.543749 14408 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:30.590950 14537 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.604070 14583 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:30.652089  8681 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: [15.8-16.9]
I20260709 13:04:30.706554 14540 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:30.590261 14656 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:30.379534 14561 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:30.612486 14422 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:30.614532 14556 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:30.629566 14581 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:30.679745 14638 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.353s	user 0.001s	sys 0.000s
I20260709 13:04:30.681370 14456 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:30.683588 14571 raft_consensus.cc:3055] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:30.686407 14566 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:30.692818  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2436) took 2720 ms. Trace:
I20260709 13:04:30.696123 14394 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:30.594229 14405 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.077s
W20260709 13:04:30.736253  9388 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.662s	user 0.010s	sys 0.007s
I20260709 13:04:30.843731  9388 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:30.704800 14465 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:30.739544 13702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.013s	sys 0.024s
W20260709 13:04:30.792279  8753 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.014s	sys 0.014s
I20260709 13:04:30.794353  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2797) took 1024 ms. Trace:
W20260709 13:04:30.739756 14573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.000s	sys 0.038s
I20260709 13:04:30.761356 14548 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:30.825021  8824 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.053s	sys 0.088s
W20260709 13:04:30.838195  8722 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.013s	sys 0.017s
I20260709 13:04:30.837713  8679 rpcz_store.cc:276] 0709 13:04:27.972151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.071423 (+2099272us) service_pool.cc:224] Handling call
0709 13:04:30.071881 (+   458us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:30.072305 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.072344 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:30.072990 (+   646us) log.cc:844] Serialized 52 byte log entry
0709 13:04:30.322428 (+249438us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:30.322478 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.322551 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.692435 (+369884us) raft_consensus.cc:1685] finished
0709 13:04:30.692469 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.692725 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2948,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":8005,"thread_start_us":202642,"threads_started":1,"wal-append.queue_time_us":457459}
W20260709 13:04:30.859383  8824 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.054s	sys 0.089s
W20260709 13:04:30.859575  8722 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.014s	sys 0.017s
I20260709 13:04:30.763418 14611 consensus_queue.cc:1243] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:04:30.861590  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:30.841080  8750 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.034s	sys 0.064s
W20260709 13:04:30.807891 14603 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.731s	user 0.014s	sys 0.000s
I20260709 13:04:30.837423 14556 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:30.845283  9382 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:30.703362 14511 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:30.913980 14447 consensus_meta.cc:287] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.843s	user 0.013s	sys 0.000s
I20260709 13:04:30.753266 14168 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:30.852347  8753 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.014s	sys 0.014s
I20260709 13:04:30.852425  8681 rpcz_store.cc:276] 0709 13:04:29.769668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.615968 (+846300us) service_pool.cc:224] Handling call
0709 13:04:30.651690 (+ 35722us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:30.793909 (+142219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.794258 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.918555  8752 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:30.919955  8681 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:04:30.920387  8681 pending_rounds.cc:85] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 8
I20260709 13:04:30.920629  8681 pending_rounds.cc:107] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:04:30.920892  8681 raft_consensus.cc:2889] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:30.921694  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2805) took 1087 ms. Trace:
I20260709 13:04:30.860864  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2800) took 1090 ms. Trace:
I20260709 13:04:30.922794  8679 rpcz_store.cc:276] 0709 13:04:29.770532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.859912 (+1089380us) service_pool.cc:224] Handling call
0709 13:04:30.860291 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.860508 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.860534 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:30.860584 (+    50us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:30.860639 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.860697 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.860797 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.837165 14571 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:30.855054 14573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.000s	sys 0.038s
W20260709 13:04:30.852236 13702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.014s	sys 0.025s
W20260709 13:04:30.927852  8796 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.011s	sys 0.023s
W20260709 13:04:30.868093 14658 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
W20260709 13:04:30.869544 14654 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:04:30.878656  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.879657  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:30.880383  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 502723us
I20260709 13:04:30.880964 14612 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:30.810250 14459 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:04:30.883181  8674 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:30.933558  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:30.938010  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2804) took 1110 ms. Trace:
W20260709 13:04:30.884043  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.886909 14505 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:30.888909 14507 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:30.890040  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1091) took 1085 ms. Trace:
I20260709 13:04:30.890172  8754 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:30.796056 14537 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:30.892834 14307 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:30.896799  8750 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.035s	sys 0.064s
I20260709 13:04:30.968469  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2471) took 1115 ms. Trace:
I20260709 13:04:30.968883  8750 rpcz_store.cc:276] 0709 13:04:29.852582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.133917 (+281335us) service_pool.cc:224] Handling call
0709 13:04:30.134234 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.134475 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:30.173644 (+ 39169us) log.cc:844] Serialized 14 byte log entry
0709 13:04:30.331990 (+158346us) spinlock_profiling.cc:243] Waited 4.37 ms on lock 0x7b18004c50a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:04:30.968047 (+636057us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:30.968109 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:30.968164 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.968237 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.968400 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38988,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":8171,"spinlock_wait_cycles":10053504,"thread_start_us":635761,"threads_started":1}
W20260709 13:04:30.843029  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.014s	sys 0.017s
W20260709 13:04:30.970893  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.014s	sys 0.017s
I20260709 13:04:30.917945 14168 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:30.922046  8681 rpcz_store.cc:276] 0709 13:04:29.834434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.919061 (+1084627us) service_pool.cc:224] Handling call
0709 13:04:30.919525 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.921393 (+  1868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.921625 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.943293  8751 rpcz_store.cc:276] 0709 13:04:29.804924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.805111 (+   187us) service_pool.cc:224] Handling call
0709 13:04:29.805505 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.805924 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.805956 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:29.806506 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:04:30.123011 (+316505us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:30.123063 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.123181 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.889784 (+766603us) raft_consensus.cc:1685] finished
0709 13:04:30.889808 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.889968 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10,"raft.queue_time_us":6851,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":13056,"thread_start_us":316042,"threads_started":1,"wal-append.queue_time_us":456082}
W20260709 13:04:30.947497  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:30.985091  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:30.991761  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.936266 14447 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:30.928071  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.003s	sys 0.031s
W20260709 13:04:30.995391  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.004s	sys 0.031s
I20260709 13:04:30.936987 14612 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.933395  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:30.998570  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 65331.9us
I20260709 13:04:30.998948  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1063) took 3206 ms. Trace:
I20260709 13:04:30.938530  8674 rpcz_store.cc:276] 0709 13:04:29.827331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.882426 (+1055095us) service_pool.cc:224] Handling call
0709 13:04:30.882927 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.937565 (+ 54638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.937923 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:30.966632 14563 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.009s	sys 0.019s
W20260709 13:04:30.995857  8793 connection.cc:256] RPC call timeout handler was delayed by -1.5284618800000089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260709 13:04:31.002431  8793 connection.cc:256] RPC call timeout handler was delayed by -1.5149618799999871s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
W20260709 13:04:30.930222  9352 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.004s	sys 0.016s
W20260709 13:04:31.003592  9352 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.004s	sys 0.016s
W20260709 13:04:30.997051  8677 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.024s	sys 0.020s
W20260709 13:04:31.004536  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.004884  8677 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.024s	sys 0.021s
I20260709 13:04:30.940371 14603 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:30.928660  8796 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.011s	sys 0.023s
W20260709 13:04:31.000237  9353 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.025s	sys 0.009s
W20260709 13:04:31.012014  9353 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.025s	sys 0.009s
W20260709 13:04:31.000690  8794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.016s	sys 0.017s
I20260709 13:04:30.999408  8678 rpcz_store.cc:276] 0709 13:04:27.792832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.064752 (+2271920us) service_pool.cc:224] Handling call
0709 13:04:30.065166 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.065604 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.065637 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:30.066195 (+   558us) log.cc:844] Serialized 26 byte log entry
0709 13:04:30.216722 (+150527us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:30.216803 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.216914 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.998649 (+781735us) raft_consensus.cc:1685] finished
0709 13:04:30.998675 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.998878 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":682,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":150096,"threads_started":1,"wal-append.queue_time_us":652152}
W20260709 13:04:31.012310  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.012550  8794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.016s	sys 0.018s
I20260709 13:04:31.013515  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:31.013958  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 605573us
I20260709 13:04:31.014726  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:31.003098  8793 connection.cc:256] RPC call timeout handler was delayed by -1.4897618799999464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.191s for a response
W20260709 13:04:31.015376  8793 connection.cc:256] RPC call timeout handler was delayed by -1.4825618800000484s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
W20260709 13:04:31.015810  8793 connection.cc:256] RPC call timeout handler was delayed by -1.4024618800000326s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20070000000000002s for a response
W20260709 13:04:31.016176  8793 connection.cc:256] RPC call timeout handler was delayed by -1.3763618799999904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2036s for a response
W20260709 13:04:31.016544  8793 connection.cc:256] RPC call timeout handler was delayed by -1.3556955790000984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1915s for a response
W20260709 13:04:31.016886  8793 connection.cc:256] RPC call timeout handler was delayed by -1.3304618800000299s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20870000000000002s for a response
W20260709 13:04:31.017220  8793 connection.cc:256] RPC call timeout handler was delayed by -1.2700421339999366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
W20260709 13:04:31.017585  8793 connection.cc:256] RPC call timeout handler was delayed by -1.1818421340000214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
W20260709 13:04:31.017943  8793 connection.cc:256] RPC call timeout handler was delayed by -1.1657626999999593s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:04:31.018294  8793 connection.cc:256] RPC call timeout handler was delayed by -1.1116421340000215s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16870000000000002s for a response
W20260709 13:04:31.012346  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 597887us
I20260709 13:04:31.028250  9387 raft_consensus.cc:2393] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:30.990347 14611 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:30.999533 14612 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:31.008396 14563 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.936s	user 0.009s	sys 0.019s
I20260709 13:04:31.010437 14603 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.009428 14507 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.026714 14626 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:04:31.078181  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.998991 14447 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:31.082525  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:31.102962  8756 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
W20260709 13:04:31.091181  8795 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.016s	sys 0.053s
I20260709 13:04:31.081545  8747 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:31.104514  8795 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.017s	sys 0.054s
W20260709 13:04:31.098877  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 83524.8us
W20260709 13:04:31.099485  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 84873.5us
W20260709 13:04:31.105265  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.098933  8793 connection.cc:256] RPC call timeout handler was delayed by -1.0325618800000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24180000000000001s for a response
I20260709 13:04:31.105885  8794 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:31.116369  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2463) took 1278 ms. Trace:
I20260709 13:04:31.120203  8754 rpcz_store.cc:276] 0709 13:04:29.838007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.939314 (+101307us) service_pool.cc:224] Handling call
0709 13:04:30.889969 (+950655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.978065 (+ 88096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.116027 (+137962us) spinlock_profiling.cc:243] Waited 950 ms on lock 0x7b54002fd5d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:31.116269 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2185527680}
W20260709 13:04:31.119046  8676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.028s	sys 0.044s
W20260709 13:04:31.123214  8676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.028s	sys 0.044s
I20260709 13:04:31.116458  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:31.124220 14652 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:31.124747 14652 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:31.124890  8793 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:31.125200 14652 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:31.126304  8793 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:31.131570  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:31.133875  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:31.137022  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:31.140190 14652 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:31.142887  8748 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:31.073230 14603 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:31.143217  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:31.064400 14534 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:04:31.148059  8673 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:04:31.270484  8673 pending_rounds.cc:85] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 13
I20260709 13:04:31.067247 14611 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:31.157553  8821 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.016s	user 0.004s	sys 0.012s
W20260709 13:04:31.278012  8821 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.136s	user 0.004s	sys 0.012s
W20260709 13:04:31.109370 14661 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:04:31.270932  8673 pending_rounds.cc:107] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:04:31.287890  8673 raft_consensus.cc:2889] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:31.289139  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1066) took 2296 ms. Trace:
I20260709 13:04:31.289633  8673 rpcz_store.cc:276] 0709 13:04:28.992331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.155324 (+1162993us) service_pool.cc:224] Handling call
0709 13:04:30.155675 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.147651 (+991976us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b540040d1a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2470dc0 00007f32e24fbd87 00007f32e24fc470 00007f32e24f8d8f 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c
0709 13:04:31.288720 (+141069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.289052 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":907793792}
I20260709 13:04:31.053596 14505 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:04:31.105712 14626 raft_consensus.cc:697] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:31.142119 14456 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.141317 14168 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:31.102576 14656 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:31.149065 14540 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:31.093928 14432 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:31.062325 14571 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:31.301362  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1095) took 1409 ms. Trace:
I20260709 13:04:31.093132 14507 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:31.286671 14437 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.005s	sys 0.024s
I20260709 13:04:31.330003  8753 rpcz_store.cc:276] 0709 13:04:29.892128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.211227 (+319099us) service_pool.cc:224] Handling call
0709 13:04:30.211609 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.211848 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:30.918523 (+706675us) log.cc:844] Serialized 14 byte log entry
0709 13:04:31.300850 (+382327us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:31.300941 (+    91us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:31.301014 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.301090 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.301276 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6306,"prepare.queue_time_us":54357,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":2748,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":8975,"thread_start_us":1032710,"threads_started":2}
W20260709 13:04:31.327494 14476 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.002s	sys 0.030s
I20260709 13:04:31.311100 14607 raft_consensus.cc:3055] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:31.323302 14540 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:31.426030  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 414831us
W20260709 13:04:31.369580 14437 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.006s	sys 0.024s
I20260709 13:04:31.362795 14659 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:31.399076 14571 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:31.422806 14476 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.007s	sys 0.031s
I20260709 13:04:31.329651 14608 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.103s
I20260709 13:04:31.457039 14560 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:04:31.459062 14674 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:04:31.459931 14663 log.cc:927] Time spent T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
W20260709 13:04:31.460747 14667 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:04:31.298324 14626 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:31.462792  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.501940  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.398108 14394 consensus_queue.cc:1175] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" responder_term: 13 status { last_received { term: 13 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 13 index: 11 } }
I20260709 13:04:31.423846 14607 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:31.464946  9355 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.982s	user 0.020s	sys 0.040s
I20260709 13:04:31.427896 14540 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:31.410288 14307 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:31.329346 14656 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:31.408722 14668 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:31.317981 14669 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:31.325256 14456 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:31.333868 14556 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:31.469875 14560 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:31.474067 14673 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:31.465193 14437 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:31.402307 14353 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:04:31.508397  8679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.002s	sys 0.015s
W20260709 13:04:31.502912 14511 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.004s	sys 0.019s
W20260709 13:04:31.506333  9355 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.050s	user 0.021s	sys 0.041s
W20260709 13:04:31.508620 14439 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.855s	user 0.009s	sys 0.005s
W20260709 13:04:31.517661 14641 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.041s	sys 0.112s
W20260709 13:04:31.537587  8679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.002s	sys 0.015s
I20260709 13:04:31.538391  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2860) took 1548 ms. Trace:
I20260709 13:04:31.490239 14534 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:31.545960  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:31.548815  8722 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.013s	sys 0.032s
I20260709 13:04:31.580111  8829 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 11. Current term is 12. Ops: []
W20260709 13:04:31.583271  8722 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.014s	sys 0.033s
I20260709 13:04:31.553191  8752 rpcz_store.cc:276] 0709 13:04:29.989953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.120185 (+130232us) service_pool.cc:224] Handling call
0709 13:04:31.538155 (+1417970us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b540034a654. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:31.538313 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1661391232}
W20260709 13:04:31.540591 14511 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.004s	sys 0.019s
W20260709 13:04:31.591356  8645 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.005s	sys 0.016s
W20260709 13:04:31.583781  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 539080us
I20260709 13:04:31.588344  8756 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:04:31.550591  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.593910  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:31.594633  8752 raft_consensus.cc:3055] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:31.589821  8822 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:31.590509  8750 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:04:31.597066  8750 pending_rounds.cc:85] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 6
I20260709 13:04:31.597434  8750 pending_rounds.cc:107] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 12.7
W20260709 13:04:31.591758  8645 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.005s	sys 0.016s
I20260709 13:04:31.597746  8750 raft_consensus.cc:2889] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:31.598014  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 620377us
W20260709 13:04:31.598816  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.540066 14462 raft_consensus.cc:3055] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:31.587767  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:31.549463 14641 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.041s	sys 0.112s
I20260709 13:04:31.499099 14670 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.109s
I20260709 13:04:31.595011  8752 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:31.647224  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 54874.9us
I20260709 13:04:31.545979 14676 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.654995  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:31.656101  8754 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:04:31.590216 14677 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.090s	user 0.000s	sys 0.002s
W20260709 13:04:31.698915  8676 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.014s	sys 0.037s
W20260709 13:04:31.723408  8676 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.014s	sys 0.037s
I20260709 13:04:31.661135  8752 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:31.725484  8752 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:31.581568 13702 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:31.726874  8680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.013s	sys 0.026s
W20260709 13:04:31.727682  8680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.013s	sys 0.026s
I20260709 13:04:31.622495  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:31.737453  8681 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.010s	sys 0.031s
W20260709 13:04:31.739459  8681 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.010s	sys 0.031s
W20260709 13:04:31.740538  8674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.012s	sys 0.016s
W20260709 13:04:31.742384  8674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.012s	sys 0.016s
W20260709 13:04:31.741400  8794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.015s	sys 0.013s
W20260709 13:04:31.743461  8794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.015s	sys 0.013s
W20260709 13:04:31.744477  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 601277us
I20260709 13:04:31.617849 14462 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:31.552718 14439 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:31.655026  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.019s	sys 0.024s
W20260709 13:04:31.761946  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.019s	sys 0.024s
W20260709 13:04:31.762966  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 625948us
I20260709 13:04:31.764091  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:31.644233 14666 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:31.699273 14548 consensus_meta.cc:287] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.834s	user 0.011s	sys 0.002s
I20260709 13:04:31.770789  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:31.692911 14447 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:31.583505 14537 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:31.613564 14534 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:31.745939 14627 consensus_meta.cc:287] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.680s	user 0.019s	sys 0.001s
I20260709 13:04:31.781016  8793 leader_election.cc:400] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16.
I20260709 13:04:31.583731 14652 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:31.592248 14505 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.781849  8672 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:04:31.782401  8796 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.026s	sys 0.036s
I20260709 13:04:31.727649 14611 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:31.790025  8752 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
W20260709 13:04:31.822263  8796 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.032s	sys 0.036s
W20260709 13:04:31.822836  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 676831us
I20260709 13:04:31.806740  8829 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:31.771858 14668 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:31.828647  9353 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.008s	sys 0.027s
W20260709 13:04:31.852769  9353 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.008s	sys 0.027s
W20260709 13:04:31.853201  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 751855us
I20260709 13:04:31.829167  9385 raft_consensus.cc:2410] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:31.800314  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2385) took 1806 ms. Trace:
I20260709 13:04:31.834537  9387 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:31.836757  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:31.855016  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.858354  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.859586  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:31.809331 14489 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.146s	user 0.024s	sys 0.000s
I20260709 13:04:31.760578 14353 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 13:04:31.752102 13702 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:31.797299 14678 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:04:31.856237  8821 rpcz_store.cc:276] 0709 13:04:29.993539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.993694 (+   155us) service_pool.cc:224] Handling call
0709 13:04:29.994067 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.994323 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:30.141283 (+146960us) log.cc:844] Serialized 14 byte log entry
0709 13:04:31.799806 (+1658523us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:31.799894 (+    88us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:31.799967 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.800064 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.800229 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22822,"prepare.queue_time_us":313593,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":198204,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":5355,"thread_start_us":1781782,"threads_started":3,"wal-append.queue_time_us":1376605}
I20260709 13:04:31.619424 14607 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:31.872300  8678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.014s	sys 0.046s
I20260709 13:04:31.904796  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2446) took 2342 ms. Trace:
W20260709 13:04:31.905222  8678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.014s	sys 0.046s
I20260709 13:04:31.801759 14507 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:31.823454 14476 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:31.645485 14560 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:31.918334  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.896687  8751 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.035s	sys 0.014s
W20260709 13:04:31.856663 14444 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.662s	user 0.015s	sys 0.008s
I20260709 13:04:31.828040 14556 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.729709 14676 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:31.776084 14437 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 13:04:31.851220 14534 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:31.867041 14675 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.364s	user 0.001s	sys 0.001s
I20260709 13:04:31.770226 14669 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.103s
I20260709 13:04:31.863303 14672 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:31.923786 14679 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
W20260709 13:04:31.925999 14685 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:04:31.950503 14560 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:31.928117 14456 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:31.934352  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:31.938807 14701 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:31.976272  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.940047 14307 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:31.781870 14666 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:31.853089 14505 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:31.934170  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 344786us
I20260709 13:04:31.948385  9387 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:04:31.892885 14489 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:31.921906 14607 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:31.867620 14462 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:31.927238 14686 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
W20260709 13:04:31.977010  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.960819 14696 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.003074 14691 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.087s
W20260709 13:04:31.986718 14422 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.162s	user 0.015s	sys 0.000s
W20260709 13:04:32.004562 14690 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 13:04:32.008613 14659 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.560s	user 0.011s	sys 0.000s
I20260709 13:04:32.009197 14659 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:31.983870 14682 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.253s	user 0.001s	sys 0.001s
I20260709 13:04:31.957490 14556 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:32.013355 14382 consensus_meta.cc:287] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.729s	user 0.015s	sys 0.003s
I20260709 13:04:31.967926 14534 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:31.961846 14656 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:32.029191 14687 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:04:32.035220 14405 consensus_queue.cc:1059] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:31.970726 14672 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:32.060166 14680 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.428s	user 0.002s	sys 0.000s
I20260709 13:04:32.052790 14534 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.061895 14534 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:32.068928 14666 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:32.074434 14603 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.790s	user 0.015s	sys 0.000s
I20260709 13:04:32.021194 14462 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:32.049136 14676 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:32.078408 14505 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:32.042122 14432 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:32.076088 14439 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:32.075426 14684 log.cc:927] Time spent T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Append to log took a long time: real 0.158s	user 0.000s	sys 0.002s
W20260709 13:04:32.083859 14583 consensus_meta.cc:287] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.276s	user 0.016s	sys 0.007s
I20260709 13:04:32.077283 14489 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:32.090102 14612 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.018s	user 0.012s	sys 0.003s
W20260709 13:04:32.091905 14571 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.627s	user 0.001s	sys 0.013s
I20260709 13:04:32.091353 14612 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.096438 14612 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:31.901151  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.944146  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2799) took 2173 ms. Trace:
I20260709 13:04:31.905308  8675 rpcz_store.cc:276] 0709 13:04:29.562306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.329025 (+766719us) service_pool.cc:224] Handling call
0709 13:04:30.367726 (+ 38701us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:30.391941 (+ 24215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.391981 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:30.392680 (+   699us) log.cc:844] Serialized 52 byte log entry
0709 13:04:31.078633 (+685953us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:31.078697 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.078803 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:31.904495 (+825692us) raft_consensus.cc:1685] finished
0709 13:04:31.904524 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.904716 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":166235,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":16272,"thread_start_us":488632,"threads_started":1,"wal-append.queue_time_us":911378}
I20260709 13:04:31.924428  8748 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:04:31.945792  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.949965  8749 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:04:31.934468  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:31.953348  8751 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.035s	sys 0.014s
W20260709 13:04:31.942020  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 157576us
W20260709 13:04:31.985023  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 125576us
I20260709 13:04:31.985767 14307 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.084617 14583 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.181687  8755 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:32.182893  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1098) took 1103 ms. Trace:
I20260709 13:04:32.183657  8756 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: []
W20260709 13:04:32.255030  8679 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.010s	sys 0.010s
I20260709 13:04:32.197609  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:32.169497  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2442) took 2637 ms. Trace:
I20260709 13:04:32.172170  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1074) took 2213 ms. Trace:
I20260709 13:04:32.187053  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1075) took 2227 ms. Trace:
I20260709 13:04:32.173816  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2452) took 2408 ms. Trace:
W20260709 13:04:32.104580 14673 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.564s	user 0.014s	sys 0.000s
I20260709 13:04:32.265031 14673 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:32.262605  8677 rpcz_store.cc:276] 0709 13:04:29.532095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.316699 (+784604us) service_pool.cc:224] Handling call
0709 13:04:30.335288 (+ 18589us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:31.005495 (+670207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.005549 (+    54us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:31.119695 (+114146us) log.cc:844] Serialized 78 byte log entry
0709 13:04:31.763897 (+644202us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:32.166070 (+402173us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.167678 (+  1608us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.168121 (+   443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.168388 (+   267us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.168561 (+   173us) raft_consensus.cc:1685] finished
0709 13:04:32.168611 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.169092 (+   481us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12663,"prepare.queue_time_us":163213,"raft.queue_time_us":219195,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":156221,"spinlock_wait_cycles":11081600,"thread_start_us":1491032,"threads_started":3,"wal-append.queue_time_us":754128}
W20260709 13:04:32.182417  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.181865  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1077) took 1750 ms. Trace:
I20260709 13:04:32.266242 14691 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:04:32.234400 14422 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.176112  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:32.270516 14422 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.179275  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:32.175757  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.274282 14707 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.277283 14710 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.277827 14707 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.279072 14708 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.280012 14708 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.014226 14696 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.201453  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:32.246811 14382 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.284034  8675 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 18: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:32.286656  8750 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:32.287209 14719 raft_consensus.cc:3055] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:32.287976 14719 raft_consensus.cc:740] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:32.291239 14432 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.292318 14432 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.170302  8680 rpcz_store.cc:276] 0709 13:04:29.770261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.638488 (+868227us) service_pool.cc:224] Handling call
0709 13:04:30.708639 (+ 70151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.708993 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:31.099703 (+390710us) log.cc:844] Serialized 14 byte log entry
0709 13:04:31.943693 (+843990us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:31.943765 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:31.943816 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.943901 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.944068 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25939,"prepare.queue_time_us":807136,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":2131,"raft.queue_time_us":94926,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":13064,"thread_start_us":1182411,"threads_started":3,"wal-append.queue_time_us":838644}
I20260709 13:04:31.986810 14701 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.239346  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:32.297976  8749 raft_consensus.cc:2410] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:04:32.299540  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.290194  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:32.246081 14307 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:32.207546  8794 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:32.239598  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:32.309466  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.199916  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:32.262955  8676 rpcz_store.cc:276] 0709 13:04:29.764832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.499163 (+734331us) service_pool.cc:224] Handling call
0709 13:04:30.499635 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:31.127402 (+627767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.127435 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:31.127993 (+   558us) log.cc:844] Serialized 26 byte log entry
0709 13:04:31.724048 (+596055us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:31.750476 (+ 26428us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.046295 (+295819us) spinlock_profiling.cc:243] Waited 8.65 ms on lock 0x7b1800288a00. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:32.172827 (+126532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.173017 (+   190us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.173138 (+   121us) raft_consensus.cc:1685] finished
0709 13:04:32.173301 (+   163us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.173572 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22988,"prepare.queue_time_us":127576,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":5650,"raft.queue_time_us":233025,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":14278,"spinlock_wait_cycles":19888000,"thread_start_us":1643336,"threads_started":3,"wal-append.queue_time_us":812363}
I20260709 13:04:32.313665  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:32.314735 14581 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:32.315289  8748 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:04:32.315788 14581 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.278362 14710 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.320758 14505 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.321642 14505 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.098830 14603 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.324731  8747 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:32.237110 13702 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:32.325752  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:32.317162  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:32.334554  8748 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
W20260709 13:04:32.238669  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 253993us
I20260709 13:04:32.334707  8676 raft_consensus.cc:2376] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13. Re-sending same reply.
I20260709 13:04:32.335407  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:32.343556 14708 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:32.345409 14583 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:32.346380 14583 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:32.347151 14583 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.348563 14505 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:32.271668 14539 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:32.349985 14583 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:32.351004  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:32.267570  8678 rpcz_store.cc:276] 0709 13:04:30.424429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.053843 (+629414us) service_pool.cc:224] Handling call
0709 13:04:31.054381 (+   538us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:31.064918 (+ 10537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.064969 (+    51us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:31.072975 (+  8006us) log.cc:844] Serialized 78 byte log entry
0709 13:04:31.905783 (+832808us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:31.923682 (+ 17899us) log.cc:844] Serialized 14 byte log entry
0709 13:04:31.924136 (+   454us) log.cc:844] Serialized 14 byte log entry
0709 13:04:31.924451 (+   315us) log.cc:844] Serialized 14 byte log entry
0709 13:04:31.924598 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.924687 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.174058 (+249371us) raft_consensus.cc:1685] finished
0709 13:04:32.174106 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.174530 (+   424us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12110,"prepare.queue_time_us":63427,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":94,"raft.queue_time_us":32442,"raft.run_cpu_time_us":519,"raft.run_wall_time_us":33064,"thread_start_us":754263,"threads_started":1,"wal-append.queue_time_us":924076}
I20260709 13:04:32.352509 14408 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:32.353106  8756 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:32.353215  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:32.353657 14701 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:32.354519  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 53620.3us
I20260709 13:04:32.355464 14539 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.356371 14710 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:32.355476 14603 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:32.358157 14422 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:32.358958  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:32.362835  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:32.262203  8751 rpcz_store.cc:276] 0709 13:04:31.079109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.079297 (+   188us) service_pool.cc:224] Handling call
0709 13:04:31.079782 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.080078 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:31.080461 (+   383us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.181823 (+1101362us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:32.181980 (+   157us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:32.182135 (+   155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.182304 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.182598 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12121,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":83765,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":300,"thread_start_us":1079576,"threads_started":1,"wal-append.queue_time_us":882209}
I20260709 13:04:32.290002 14719 consensus_queue.cc:260] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.372380  8822 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:04:32.372851  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 175796us
I20260709 13:04:32.373091  8822 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:32.374534  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:32.392820 14380 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:32.393935 14476 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:32.373278  9384 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:32.374570 14437 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:32.397852 14437 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:32.398458 14437 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:32.387372  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:32.384567  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:32.383512  9383 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
W20260709 13:04:32.382948  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 175818us
I20260709 13:04:32.369174  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 1083) took 1044 ms. Trace:
I20260709 13:04:32.408213  8678 rpcz_store.cc:276] 0709 13:04:31.324869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.352673 (+1027804us) service_pool.cc:224] Handling call
0709 13:04:32.369120 (+ 16447us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:32.400050  8677 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:32.375931  9382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 850) took 2086 ms. Trace:
W20260709 13:04:32.388574  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 194515us
I20260709 13:04:32.395228  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:32.388962 14540 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.812s	user 0.009s	sys 0.009s
W20260709 13:04:32.378772  9385 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 854) took 2038 ms (client timeout 2042 ms). Trace:
I20260709 13:04:32.383510 14659 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:32.434579  9385 rpcz_store.cc:269] 0709 13:04:30.340658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.340829 (+   171us) service_pool.cc:224] Handling call
0709 13:04:32.378085 (+2037256us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b540037da54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:32.378481 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3312003328}
I20260709 13:04:32.450093  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:32.451013 14174 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:32.453287  8680 raft_consensus.cc:2410] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:04:32.454468  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:32.458120 14556 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:32.418152 14539 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:32.415880  9382 rpcz_store.cc:276] 0709 13:04:30.289778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.290113 (+   335us) service_pool.cc:224] Handling call
0709 13:04:30.844984 (+554871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.912443 (+ 67459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.375302 (+1462859us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b54001ccd50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:32.375625 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1207208320}
I20260709 13:04:32.451113  8678 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:32.389894 14353 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:32.377965 14432 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:32.446702 14659 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.404770 14476 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 13:04:32.473591  8676 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:32.482668  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2468) took 1008 ms. Trace:
I20260709 13:04:32.483049  8676 rpcz_store.cc:276] 0709 13:04:31.473881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.367332 (+893451us) service_pool.cc:224] Handling call
0709 13:04:32.482569 (+115237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:32.475544  8719 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:32.468060 14353 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:32.482087 14659 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:32.480211  8822 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.487233 14719 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:32.499066 14561 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:32.506795 14548 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.508904  8822 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:32.513413 14607 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.512s	user 0.017s	sys 0.001s
W20260709 13:04:32.513716  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 51925.7us
W20260709 13:04:32.516458  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 121500us
W20260709 13:04:32.517354  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 63315.1us
I20260709 13:04:32.484828 14701 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:32.461310 14540 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.530959 14673 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:32.532269  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2437) took 3063 ms. Trace:
I20260709 13:04:32.533108 14673 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:32.534005 14673 consensus_queue.cc:1243] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:32.534824 14707 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:32.535056 14476 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:32.535967 14476 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:32.538655 14701 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:32.539126 14168 consensus_meta.cc:287] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.219s	user 0.012s	sys 0.016s
I20260709 13:04:32.542420  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:32.550951  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:32.547905  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1078) took 1361 ms. Trace:
I20260709 13:04:32.530230  8794 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:32.549510  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:32.554509 14691 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:32.555392 14672 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:32.532816  8672 rpcz_store.cc:276] 0709 13:04:29.468773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.173845 (+705072us) service_pool.cc:224] Handling call
0709 13:04:30.202570 (+ 28725us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.823508 (+1620938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.531566 (+708058us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b5400046d54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:32.531955 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3619349760}
I20260709 13:04:32.540102 14652 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:32.262825  8674 rpcz_store.cc:276] 0709 13:04:29.959041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.000117 (+1041076us) service_pool.cc:224] Handling call
0709 13:04:31.063210 (+ 63093us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:31.074911 (+ 11701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.074962 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:31.075605 (+   643us) log.cc:844] Serialized 52 byte log entry
0709 13:04:31.742926 (+667321us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:32.168895 (+425969us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.169867 (+   972us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.170161 (+   294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.170301 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.170407 (+   106us) raft_consensus.cc:1685] finished
0709 13:04:32.170423 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.186755 (+ 16332us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14560,"prepare.queue_time_us":247001,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":173267,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":24605,"thread_start_us":969198,"threads_started":2,"wal-append.queue_time_us":854648}
I20260709 13:04:32.541543  9352 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:32.562731  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 96461.9us
I20260709 13:04:32.551874  8673 rpcz_store.cc:276] 0709 13:04:31.007419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.293540 (+286121us) service_pool.cc:224] Handling call
0709 13:04:31.293986 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.320074 (+ 26088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.320169 (+    95us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:31.320238 (+    69us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:31.320305 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.358389 (+1038084us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b54003a25d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:32.358468 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.368976 (+ 10508us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59244928}
I20260709 13:04:32.262719  8681 rpcz_store.cc:276] 0709 13:04:29.958808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.000034 (+1041226us) service_pool.cc:224] Handling call
0709 13:04:31.061963 (+ 61929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.062215 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:31.741897 (+679682us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.171139 (+429242us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:32.171287 (+   148us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:32.171450 (+   163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.171605 (+   155us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.171911 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1300,"prepare.queue_time_us":487141,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":73,"raft.queue_time_us":100036,"raft.run_cpu_time_us":417,"raft.run_wall_time_us":31443,"thread_start_us":1098272,"threads_started":2,"wal-append.queue_time_us":258817}
I20260709 13:04:32.577927  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:32.578699  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:32.581723  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:32.582468  8681 raft_consensus.cc:1813] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Handling vote request from an unknown peer e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:32.582765 14168 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:32.583153  8681 raft_consensus.cc:2410] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:32.584362  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:32.585541  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:32.561306 14548 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.593943 14691 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:32.593897 14548 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:32.595604 14548 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.598127 14548 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.597290 14696 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:32.599817 14353 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.624465 14353 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:32.625703  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1096) took 2733 ms. Trace:
I20260709 13:04:32.601488  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:32.537925 14476 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:32.610412  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 80369.4us
I20260709 13:04:32.601751 14481 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:32.602561  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:32.602984  8681 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:32.604215  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2466) took 1174 ms. Trace:
I20260709 13:04:32.605400 14701 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:32.628386  8677 rpcz_store.cc:276] 0709 13:04:31.429566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.333154 (+903588us) service_pool.cc:224] Handling call
0709 13:04:32.603965 (+270811us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b5400299354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:32.604116 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73405184}
I20260709 13:04:32.606575  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1079) took 1598 ms. Trace:
I20260709 13:04:32.596153  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:32.525490  8645 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:32.622942 14652 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:32.626148  8755 rpcz_store.cc:276] 0709 13:04:29.892425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.211884 (+319459us) service_pool.cc:224] Handling call
0709 13:04:30.212204 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.286196 (+2073992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.625294 (+339098us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b5400102a54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:32.625583 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4527294720}
I20260709 13:04:32.627080  8672 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:32.630888  8675 rpcz_store.cc:276] 0709 13:04:31.007790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.605081 (+1597291us) service_pool.cc:224] Handling call
0709 13:04:32.605662 (+   581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.606085 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:32.606124 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:32.606209 (+    85us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:32.606264 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.606336 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.606478 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:32.652204  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 110819us
I20260709 13:04:32.534578 14673 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.112s
I20260709 13:04:32.658277  8676 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:32.632210 14652 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.642565 14556 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:32.661659  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 174447us
I20260709 13:04:32.663995  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:32.684549  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.680274 14652 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:32.706328  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.706419  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2463) took 2748 ms. Trace:
I20260709 13:04:32.706354  8822 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:04:32.716483  8822 pending_rounds.cc:85] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 8
I20260709 13:04:32.727839  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1080) took 1713 ms. Trace:
I20260709 13:04:32.728564  8681 rpcz_store.cc:276] 0709 13:04:31.008088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.648234 (+1640146us) service_pool.cc:224] Handling call
0709 13:04:32.707267 (+ 59033us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.721133 (+ 13866us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:32.721173 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:32.721229 (+    56us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:32.721275 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.721354 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.721529 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:32.709065 14673 consensus_queue.cc:1243] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:32.739984 14629 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.722071  8822 pending_rounds.cc:107] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:04:32.741735  8822 raft_consensus.cc:2889] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:32.729296  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:32.742950  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:32.748036  8678 raft_consensus.cc:2410] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:32.756379  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2472) took 1227 ms. Trace:
I20260709 13:04:32.711338  8679 rpcz_store.cc:276] 0709 13:04:29.958401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.923831 (+965430us) service_pool.cc:224] Handling call
0709 13:04:30.924292 (+   461us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:30.924877 (+   585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.924925 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:30.925739 (+   814us) log.cc:844] Serialized 52 byte log entry
0709 13:04:31.580692 (+654953us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:31.633406 (+ 52714us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.484480 (+851074us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.634294 (+149814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.652493 (+ 18199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.668456 (+ 15963us) raft_consensus.cc:1685] finished
0709 13:04:32.668475 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.706341 (+ 37866us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54114,"prepare.queue_time_us":168037,"prepare.run_cpu_time_us":142,"prepare.run_wall_time_us":5085,"raft.queue_time_us":227438,"raft.run_cpu_time_us":595,"raft.run_wall_time_us":38177,"thread_start_us":1251234,"threads_started":2,"wal-append.queue_time_us":856862}
I20260709 13:04:32.756865  8678 rpcz_store.cc:276] 0709 13:04:31.528679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.621741 (+1093062us) service_pool.cc:224] Handling call
0709 13:04:32.756278 (+134537us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:32.719120 14707 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:32.736016 14539 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:32.745965 14607 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:32.763882  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.764606  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1081) took 1753 ms. Trace:
I20260709 13:04:32.766481  8673 rpcz_store.cc:276] 0709 13:04:31.011664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.656057 (+1644393us) service_pool.cc:224] Handling call
0709 13:04:32.699998 (+ 43941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.756493 (+ 56495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:32.756531 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:32.760726 (+  4195us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:32.764241 (+  3515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.764374 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.764537 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:32.765625  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55022.6us
W20260709 13:04:32.769352  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.759136 14707 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.772763 14174 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:32.773416 14444 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 13:04:32.776024 14534 consensus_meta.cc:287] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.637s	user 0.017s	sys 0.006s
I20260709 13:04:32.777065 14534 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.780972  9355 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:32.741047 14629 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.784345 14566 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:32.785193  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.785917  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.785741 14629 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:32.790820 14539 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:32.792994  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.773043 14707 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:32.800449  8681 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:04:32.801757  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.800426  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:32.797027  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:32.809898  9383 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:32.799124  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:32.809399  9384 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
W20260709 13:04:32.811755  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.795146  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:32.815305  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2470) took 1287 ms. Trace:
I20260709 13:04:32.815734  8681 rpcz_store.cc:276] 0709 13:04:31.528011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.751250 (+1223239us) service_pool.cc:224] Handling call
0709 13:04:32.799969 (+ 48719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.807980 (+  8011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.815232 (+  7252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:32.817710  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:32.819247  8677 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 13:04:32.794756 14462 consensus_meta.cc:287] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.677s	user 0.015s	sys 0.009s
I20260709 13:04:32.799479  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:32.804070 14476 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:32.792397 14405 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:32.794579 14571 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:32.793524 14539 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:32.818521  8673 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:32.799685 14548 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:32.827366  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1082) took 1536 ms. Trace:
I20260709 13:04:32.811848 14683 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.838487  8677 rpcz_store.cc:276] 0709 13:04:31.287582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.692985 (+1405403us) service_pool.cc:224] Handling call
0709 13:04:32.770884 (+ 77899us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:32.823144 (+ 52260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.823402 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:32.820912 14540 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:32.842216  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.855647  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.859457  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:32.860585 14571 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:32.860769  9352 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:32.861124 14571 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.861608 14571 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:32.862068 13702 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:32.862330  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 426697us
I20260709 13:04:32.861153 14465 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:32.863370  8673 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:32.863464 14465 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.863073 14688 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:32.865084  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:32.865407 14688 raft_consensus.cc:697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:32.865963  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 185145us
I20260709 13:04:32.862669 13702 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:32.867460 13702 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:32.866796 14688 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.866911 14439 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:32.866390 14465 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:32.827280 14405 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.871106  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:32.874465  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.875084  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.870524  8722 leader_election.cc:336] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.632s (SENT)
I20260709 13:04:32.873102  8679 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 9.
I20260709 13:04:32.792457 14556 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.885775  8749 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:32.870820 14439 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.891383 14439 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:32.893448  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.895043  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:32.896624  8681 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:32.897477  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2471) took 1369 ms. Trace:
I20260709 13:04:32.897801  8681 rpcz_store.cc:276] 0709 13:04:31.528383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.837972 (+1309589us) service_pool.cc:224] Handling call
0709 13:04:32.896309 (+ 58337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.897163 (+   854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.897409 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:32.820587 14607 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:32.910804  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 48817us
I20260709 13:04:32.912688  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:32.918987 14670 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:32.919770 14670 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:32.920372 14670 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:32.920259  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:32.923019  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 201379us
I20260709 13:04:32.924161  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:32.925457 14664 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:32.926052  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:32.928547 14696 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:32.929139 14696 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:32.880602 14405 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:32.905632  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:32.914636  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:32.911263  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 333463us
I20260709 13:04:32.860841 14626 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.938719  9385 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:32.944336  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:32.950456  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:32.926273 14571 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.965657  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:32.930224 14696 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.877130 14683 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:32.941260 14581 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:32.952745 14664 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:32.974577  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:32.979951  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54110.2us
I20260709 13:04:32.982052  9352 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:32.986990 14537 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.987931 14537 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:32.988744 14672 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:32.989195 14672 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.989580 14672 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:32.993492 14380 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.994133 14380 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:32.996166  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 121812us
I20260709 13:04:32.997556  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:32.997550 14380 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:33.002372 14607 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:33.004173  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 204889us
I20260709 13:04:33.005211  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:32.952975 14465 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:33.010105  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:33.022930  8822 raft_consensus.cc:2393] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:33.017546 14382 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:33.041810  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:32.960223 14626 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:33.040448  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:33.048781  8749 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:33.022221 14537 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:33.080467  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:33.081362  8681 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
W20260709 13:04:33.086107 14719 consensus_meta.cc:287] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.562s	user 0.007s	sys 0.003s
I20260709 13:04:33.078157  9388 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
W20260709 13:04:33.079064 14437 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.680s	user 0.012s	sys 0.006s
I20260709 13:04:33.078599 14394 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:33.088496 14626 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:33.088864  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:33.100121  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54103.1us
I20260709 13:04:33.024845 14671 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.095636 14556 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:33.026918 14432 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:04:33.044641 14696 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:33.046087 14583 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:33.131784 14583 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:33.132349 14583 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:33.133291 14707 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:33.138851 14444 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:33.139559  8753 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:33.139678 14444 raft_consensus.cc:697] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:33.140511 14707 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:33.103295  8719 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:33.146122  8749 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:33.108668  9353 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:33.090075 14652 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:33.147405  8822 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:33.148149 14607 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:33.148677 14607 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:33.148695 14382 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:33.149125 14652 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:33.149343 14382 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:33.149849 14382 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:33.150235  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:33.151051  8751 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:33.151196  8753 raft_consensus.cc:1217] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Deduplicated request from leader. Original: 13.9->[16.10-16.10]   Dedup: 16.10->[]
I20260709 13:04:33.151835  8753 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:04:33.152980  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:33.116554 14611 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.153671  8753 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:33.153772 14611 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:33.073056 14534 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:33.099925 14659 consensus_meta.cc:287] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.533s	user 0.014s	sys 0.000s
I20260709 13:04:33.100201 14481 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:04:33.155328 14659 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:33.157845 14659 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:33.124009  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:33.102998 14589 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:04:33.127841 14671 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:33.130390 14719 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:33.162842 14719 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.163305 14719 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:33.164168  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.165194  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.165771  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:33.139559 14560 consensus_meta.cc:287] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.166s	user 0.013s	sys 0.006s
I20260709 13:04:33.125156 14672 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:33.167343  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:33.168269  8829 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:33.140910 14444 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:33.142500  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 102236us
I20260709 13:04:33.143250  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:33.110257 14437 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:33.155699 14481 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:33.177882 14481 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:33.177971  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:33.177996  9381 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:04:33.178812 14571 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:33.179262 14481 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:33.180656  8821 raft_consensus.cc:2410] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:04:33.180821 14481 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:33.102551 14732 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:33.122239 14432 consensus_queue.cc:1243] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:04:33.174147  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 43928.8us
I20260709 13:04:33.082413  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:33.169898 14672 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:33.171212  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 49843.1us
I20260709 13:04:33.176700 14673 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.177064  8747 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:33.170558  8680 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:33.186244  8681 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:33.149173 14607 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:33.199061  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:33.204681  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.113806 14462 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:04:33.209276  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:33.172915 14670 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:33.213485 14673 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:33.215423 14670 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:33.215821 14673 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:33.212302  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2811) took 1017 ms. Trace:
I20260709 13:04:33.217558  8826 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:33.221151  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:33.223749  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:33.224421  8821 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:33.225200 14589 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:33.226039  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:33.226917  9381 raft_consensus.cc:2393] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:33.098534 13702 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:33.230973  8829 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:33.225845  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:33.239423  9352 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:33.247781 14561 consensus_meta.cc:287] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.700s	user 0.012s	sys 0.002s
I20260709 13:04:33.248855  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:33.249370  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:33.249718  9387 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:33.250015  8681 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:33.259542  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:33.260334  8679 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:33.250955 14701 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:33.267491  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 330042us
I20260709 13:04:33.255666  8796 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:33.208848 14659 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.053s
W20260709 13:04:33.271154  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 69509.6us
I20260709 13:04:33.247885 14437 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:33.276396 14519 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:33.276984 14519 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.277364 14519 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:33.279053  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:33.280577  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:33.281790  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:33.283347  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:33.284904  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:33.287098  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:33.223206 14626 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:04:33.291167  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:33.292301  8829 raft_consensus.cc:2410] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:33.294493 13702 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:33.273353 14701 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:33.282428  8828 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:33.298287  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:33.299775  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:33.300663  9386 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:33.301192  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:33.294643  8722 leader_election.cc:336] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.837s (SENT)
I20260709 13:04:33.302304  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:33.286262  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:33.299325 14583 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:33.305094  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 51839.6us
I20260709 13:04:33.306336  8827 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:04:33.310223  8681 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
W20260709 13:04:33.311527  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 374523us
I20260709 13:04:33.316350  8749 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:33.314366  8679 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 10.
W20260709 13:04:33.321596  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 58969.6us
I20260709 13:04:33.331131  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:33.333446  8751 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:33.329774  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:33.336838  8675 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:33.339439  8823 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:33.341163  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:33.341957  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 63072.1us
I20260709 13:04:33.319010 14437 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:33.349829  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:33.353118  8751 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:33.347565  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.319904 14439 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:33.356621  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 62020.8us
I20260709 13:04:33.414006 14439 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:33.327265 14405 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:33.359642 14611 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:33.364234  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:33.365855  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:33.361861 14566 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:33.341225 14432 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:33.352782 14437 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.375248  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:33.218447  8673 rpcz_store.cc:276] 0709 13:04:32.186588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.864645 (+678057us) service_pool.cc:224] Handling call
0709 13:04:32.864996 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:32.865257 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:32.871317 (+  6060us) log.cc:844] Serialized 14 byte log entry
0709 13:04:33.203475 (+332158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:33.203555 (+    80us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:33.203629 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.203731 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.203928 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2195,"prepare.queue_time_us":2360,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":2355,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":7577,"thread_start_us":265307,"threads_started":1}
I20260709 13:04:33.381294  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:33.386108 14707 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:33.387990  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.440294  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:33.440620  8678 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
W20260709 13:04:33.394131  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.441692  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.442073  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:33.396411  8748 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:33.327064 14382 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:33.398124 14671 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:33.405311 14735 log.cc:927] Time spent T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.302s	user 0.001s	sys 0.003s
I20260709 13:04:33.447230  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:33.358325  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:33.415201  8756 raft_consensus.cc:2393] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 17: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:33.420671  8675 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:33.447201  8672 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:33.448729 14312 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:33.449364  8754 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:33.449934  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:33.452279 14744 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:04:33.456223  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:33.461475  8821 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:33.462499  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:33.463574  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:33.471515  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:33.443413 14437 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:33.462983  8647 connection.cc:256] RPC call timeout handler was delayed by -1.5197655030000305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
I20260709 13:04:33.472841  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:33.468359  8749 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:33.472659  9352 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:33.408238 14589 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:04:33.464042 14566 consensus_queue.cc:1243] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:33.461350 14312 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:33.494072  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:33.491822  8647 connection.cc:256] RPC call timeout handler was delayed by -1.2172105729999885s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
I20260709 13:04:33.499024  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:33.479775 14394 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:33.510670  8678 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:33.481019 14560 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:33.537513  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 80489.8us
I20260709 13:04:33.539466  8719 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:33.540409 14439 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:33.540532  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:33.535576  8680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 1085) took 1345 ms (client timeout 1682 ms). Trace:
W20260709 13:04:33.542268  8680 rpcz_store.cc:269] 0709 13:04:32.166576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.454132 (+287556us) service_pool.cc:224] Handling call
0709 13:04:33.511300 (+1057168us) spinlock_profiling.cc:243] Waited 715 ms on lock 0x7b54002562d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:33.511438 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1645054848}
I20260709 13:04:33.543092  8751 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
W20260709 13:04:33.543898  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 45289us
I20260709 13:04:33.491739 13702 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:33.492581 14659 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:33.497218 14556 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.507323 14432 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:33.560767  8675 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:33.541080 14439 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.565296  8681 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:33.565295 13702 raft_consensus.cc:697] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:33.552362  9388 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 858) took 2666 ms (client timeout 1536 ms). Trace:
W20260709 13:04:33.568660  9388 rpcz_store.cc:269] 0709 13:04:30.878934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.879296 (+   362us) service_pool.cc:224] Handling call
0709 13:04:33.535596 (+2656300us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b5400018bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:33.545475 (+  9879us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4805352192}
W20260709 13:04:33.578564  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.579741  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:33.579815  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.581570  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.586410  9380 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:33.587836  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:33.575909 14312 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:33.589556 14405 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:33.589699 14312 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:33.592075 14382 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:33.597708  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2473) took 2068 ms. Trace:
I20260709 13:04:33.602331  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:33.598357  8677 rpcz_store.cc:276] 0709 13:04:31.529037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.840613 (+1311576us) service_pool.cc:224] Handling call
0709 13:04:32.916270 (+ 75657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:32.955898 (+ 39628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:32.955954 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:32.978925 (+ 22971us) log.cc:844] Serialized 26 byte log entry
0709 13:04:33.171444 (+192519us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:33.171504 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.171593 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:33.451120 (+279527us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:33.580214 (+129094us) raft_consensus.cc:1685] finished
0709 13:04:33.580253 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.597587 (+ 17334us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4181,"raft.queue_time_us":8943,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":4703,"spinlock_wait_cycles":24263040,"thread_start_us":132412,"threads_started":1,"wal-append.queue_time_us":255538}
I20260709 13:04:33.560767 14394 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:33.570488 14439 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:33.577909 14560 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.610267  8754 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
W20260709 13:04:33.613483  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 186196us
W20260709 13:04:33.610152  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 251951us
I20260709 13:04:33.611297  8672 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:33.607605  8680 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:33.580530 14583 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:33.636561  8675 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:33.626766 14382 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:33.648362 14560 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:33.560453 14607 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:33.567800 14556 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:33.578135 14719 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:33.625905 14626 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:33.660418  9388 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:04:33.661062  8754 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:33.661090  8677 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:33.645455 13702 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:33.664615  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.654701  8679 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:33.675897  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1084) took 2223 ms. Trace:
I20260709 13:04:33.676496  8676 rpcz_store.cc:276] 0709 13:04:31.452271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.702022 (+1249751us) service_pool.cc:224] Handling call
0709 13:04:32.753926 (+ 51904us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:32.836455 (+ 82529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:32.836509 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:32.926436 (+ 89927us) log.cc:844] Serialized 26 byte log entry
0709 13:04:33.267380 (+340944us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:33.335801 (+ 68421us) log.cc:844] Serialized 14 byte log entry
0709 13:04:33.336177 (+   376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.662639 (+326462us) spinlock_profiling.cc:243] Waited 47.9 ms on lock 0x7b5400298954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:33.662701 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:33.662750 (+    49us) raft_consensus.cc:1685] finished
0709 13:04:33.675599 (+ 12849us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.675817 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2128,"prepare.queue_time_us":98236,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":66,"raft.queue_time_us":264,"raft.run_cpu_time_us":407,"raft.run_wall_time_us":14147,"spinlock_wait_cycles":110117120,"thread_start_us":337014,"threads_started":1,"wal-append.queue_time_us":615419}
I20260709 13:04:33.640290 14566 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.713855  8680 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:33.735546  9382 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:33.736593  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:33.737114  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.737967  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.739003  8679 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:33.742921  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:33.772217  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 113866us
I20260709 13:04:33.742913  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:33.776567  9388 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:33.780408 14743 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 13:04:33.780633 14741 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.074s	user 0.000s	sys 0.003s
I20260709 13:04:33.781380  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:33.788342  8681 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
W20260709 13:04:33.794100  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.796932 14560 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:33.754524 14476 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:33.777531 14740 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.070s	user 0.003s	sys 0.000s
I20260709 13:04:33.792748  8679 pending_rounds.cc:85] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 8
I20260709 13:04:33.808485  8679 pending_rounds.cc:107] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:04:33.808773  8679 raft_consensus.cc:2889] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:33.807905  9380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 1065) took 1487 ms. Trace:
I20260709 13:04:33.811617  9380 rpcz_store.cc:276] 0709 13:04:32.320199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.321522 (+  1323us) service_pool.cc:224] Handling call
0709 13:04:33.552211 (+1230689us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.587069 (+ 34858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.799620 (+212551us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400018bd0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:33.807813 (+  8193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2814012800}
W20260709 13:04:33.806257  8822 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.031s	sys 0.027s
W20260709 13:04:33.814281  8822 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.031s	sys 0.027s
I20260709 13:04:33.816471  8672 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:33.663170 14394 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:33.778901 14626 raft_consensus.cc:697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:33.812552 14748 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.150s	user 0.002s	sys 0.002s
I20260709 13:04:33.739259 14566 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:33.855343  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:33.857685  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:33.860200  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 47840.7us
W20260709 13:04:33.857412  8722 leader_election.cc:336] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.914s (SENT)
I20260709 13:04:33.867216  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:33.864087  9386 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 851) took 3568 ms (client timeout 1764 ms). Trace:
I20260709 13:04:33.801288 14540 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:04:33.860275 14659 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:33.875224  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 219762us
W20260709 13:04:33.888620  8722 leader_election.cc:336] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.942s (SENT)
I20260709 13:04:33.889402  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:33.870971  9386 rpcz_store.cc:269] 0709 13:04:30.296172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.296422 (+   250us) service_pool.cc:224] Handling call
0709 13:04:33.851400 (+3554978us) spinlock_profiling.cc:243] Waited 2.93 s on lock 0x7b5400520b54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:33.863969 (+ 12569us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6741061504}
I20260709 13:04:33.890859 14465 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:04:33.892575 14749 log.cc:927] Time spent T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:04:33.868603 14476 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:33.873061 14659 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.903316 14659 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:33.898245 14519 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.620s	user 0.012s	sys 0.000s
I20260709 13:04:33.913427 14607 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:33.870347 14589 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:33.921869  8678 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:33.873633 14672 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:04:33.929452  8722 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.443s (SENT) [suppressed 40 similar messages]
I20260709 13:04:33.935401 14691 raft_consensus.cc:3055] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:33.935943 14691 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:33.936961 14691 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:33.937682 14566 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:33.938278 14707 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:33.939800 14168 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:04:33.972508  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.940267 14168 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:33.924525 14540 consensus_queue.cc:1059] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:04:33.989713  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:33.992528  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.999537  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 70141.3us
I20260709 13:04:34.015877  9388 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:33.925982 14589 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:33.917793 14519 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:34.031116  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 162512us
I20260709 13:04:34.034826 14465 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.037314 14168 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.038633  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:34.109642  8752 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:34.040617 14701 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:34.043301 14746 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:04:34.043623  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:34.044234 14757 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 13:04:34.049055  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.050596 14589 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:34.052120  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 126565us
I20260709 13:04:34.052151 14519 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:34.056540  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:34.058552  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:34.059547  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:34.123277  9355 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:34.125304  8821 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:34.060374 14439 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.127558  8793 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:34.068066 14556 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:34.069293  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.129436 14439 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:34.129969  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.130290 14462 raft_consensus.cc:3055] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:34.130753 14462 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:34.131325  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.131685 14462 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:34.132678  8754 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:34.070062  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.133098 14462 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.133417  8749 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:34.134372  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.121603 14540 consensus_queue.cc:1059] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:04:34.134977  8749 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:34.135915  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.136673 14394 raft_consensus.cc:2764] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:04:34.136937  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.137140  8747 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:34.137598  8749 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:34.138103  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.070837  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:34.139019 14394 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:34.139681  8756 raft_consensus.cc:2368] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15.
I20260709 13:04:34.139585 14394 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.072043  8750 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:34.072654  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.077167 14683 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:34.083268  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 62369.8us
W20260709 13:04:34.087078 14740 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:34.030866 14626 consensus_queue.cc:237] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:34.031666 14444 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:34.138456  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:34.089226  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 54663us
W20260709 13:04:34.090587  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.091246 14760 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.091300 14611 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:04:34.094424 14747 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.444s	user 0.000s	sys 0.004s
I20260709 13:04:34.156996  8749 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:34.149660  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:34.098405 14437 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.588s	user 0.012s	sys 0.000s
I20260709 13:04:34.100844 14719 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:34.106442 14465 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:34.115835  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.108477 14652 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:04:34.122179 14607 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.127128  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.132543  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.069967  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.155673  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:34.161697  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.173210  9355 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:34.200076 13702 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:34.200753  8751 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:34.200796 13702 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.201525  9355 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:34.201892 13702 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:34.179979  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:34.204077  8752 raft_consensus.cc:2393] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:34.176576  8722 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:34.169510  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 123141us
I20260709 13:04:34.179498  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.179064  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.213624  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:34.157009 14611 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:34.252496  9382 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:34.229815  8750 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 14: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:34.248543  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:34.248522  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:34.261619  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:34.262140  8753 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:34.262238  8826 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:34.177137 14168 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:34.207437 14683 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:34.212327 14701 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:04:34.273236  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 56154us
W20260709 13:04:34.214660 14757 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 13:04:34.267978  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 157123us
I20260709 13:04:34.268922  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.285075  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:34.287477  8756 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:34.250823 14439 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:04:34.288731  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 81523.4us
I20260709 13:04:34.299259  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.214478 14382 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:34.302433  8645 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:34.177457 14437 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.214818 14465 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:34.261044 14719 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.318591  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 53846.9us
I20260709 13:04:34.319259  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:34.314824  8749 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:34.321038  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:34.324090  9382 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:34.211380 14607 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.272055 14683 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.325137 14683 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:34.327014 14607 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:34.282158 14394 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:34.319727  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:34.322494  8722 leader_election.cc:336] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.541s (SENT)
I20260709 13:04:34.333113  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.333379  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.335404  8751 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:34.335999  8750 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 21: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:04:34.336447  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.214744 14760 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:34.339066  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:34.329161 14540 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:04:34.345170  8823 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 17: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:04:34.345901  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.348568  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:34.343376 14688 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:34.339892  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:34.330024 14556 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:34.359871  8645 leader_election.cc:400] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15.
I20260709 13:04:34.360451  8645 leader_election.cc:403] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:04:34.356842  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.329456 14652 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:34.337544 14439 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:34.366087  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 81114.3us
I20260709 13:04:34.372347  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:34.340508 14755 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:34.377305  8821 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:04:34.353783 14540 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.374197  8794 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:34.362879 14556 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.341149 14659 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:34.387629 14707 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:34.387890 14767 raft_consensus.cc:3055] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:34.407464 14767 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:34.388863 14768 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:04:34.394996  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 81869.8us
W20260709 13:04:34.397357  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 66074.7us
I20260709 13:04:34.371336 14611 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:34.412760 14611 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:34.415757 14439 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.420739 14760 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:34.420398  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:34.383247 13702 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.346453 14719 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.425100 14707 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:34.370132 14537 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15.
I20260709 13:04:34.426119  9386 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:34.426826 14719 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:34.428606 13702 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:34.428741  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:34.429481  8827 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:34.385321 14766 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.431497 14766 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.434062 14766 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:34.379339 14765 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:34.435297 14765 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.435734 14765 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:34.436673  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:34.437635  9380 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
W20260709 13:04:34.438347 14763 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:04:34.403015  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:34.408726 14767 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.424403  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:34.424727 14659 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:34.401597 14437 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:34.398283 14540 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:34.380067 14755 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.399832 14556 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.441716 14476 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:34.439620  8796 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:34.491168  8830 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:34.434110  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:34.491964  8827 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
W20260709 13:04:34.492185  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 92863.2us
I20260709 13:04:34.492663  9352 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:34.493492  8753 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
W20260709 13:04:34.493997  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 318112us
I20260709 13:04:34.495357  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:34.496666 14683 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.499642  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:34.500854  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:34.503412  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.615255 14312 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.628533  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 189028us
I20260709 13:04:34.629136 14312 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.629895  8674 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:34.630128  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.628641  9385 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:34.629302 14168 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:34.630908 14673 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.631278 14168 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.631767 14168 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:34.631861 14312 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:34.631538 14673 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:34.634132 14673 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:34.559904  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:34.637269  8672 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:04:34.507982 14652 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:34.640132  8680 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:34.642604 14589 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:34.600652 14691 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:34.643865 14589 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:34.644341 14589 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:34.645363 14589 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:34.646901 14589 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:34.648818  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.652689 14540 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:34.510232  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.504120  8826 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
W20260709 13:04:34.516207 14762 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.220s	user 0.000s	sys 0.006s
W20260709 13:04:34.655993  8564 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0e9b0850145a4c66bfe983ffdfa93a57 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:34.657935  9386 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:34.667788  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 233750us
I20260709 13:04:34.669188  8719 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:34.669282  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:34.669423 14771 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:34.670002  8675 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:34.670554 14771 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:34.671454 14771 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:34.672230  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:34.673194 14583 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:34.673416  8719 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:34.673803 14583 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:34.658054  8722 leader_election.cc:336] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.818s (SENT)
I20260709 13:04:34.658355  9385 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:34.674876  8722 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:34.675987  8796 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:34.676448  8722 leader_election.cc:336] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.863s (SENT)
I20260709 13:04:34.662858 14168 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.677042  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:34.675918 14583 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:34.678244 14439 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:34.679495  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:34.679641 14537 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:34.680230  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:34.680883  9385 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:34.680958  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 256676us
I20260709 13:04:34.682045  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:34.683027  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:34.684242  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2494) took 1274 ms. Trace:
I20260709 13:04:34.684758  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:34.685729 14683 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:34.684684  8679 rpcz_store.cc:276] 0709 13:04:33.409725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.767087 (+357362us) service_pool.cc:224] Handling call
0709 13:04:33.792471 (+ 25384us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:33.853773 (+ 61302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:33.853812 (+    39us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:33.862061 (+  8249us) log.cc:844] Serialized 78 byte log entry
0709 13:04:34.121331 (+259270us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:34.121379 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.121482 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:34.515361 (+393879us) spinlock_profiling.cc:243] Waited 8.36 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:34.683902 (+168541us) raft_consensus.cc:1685] finished
0709 13:04:34.683928 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.684177 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31059,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":6676,"spinlock_wait_cycles":19237760,"thread_start_us":233554,"threads_started":1,"wal-append.queue_time_us":302209}
I20260709 13:04:34.686710 14770 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:34.583554  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:34.687465 14770 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:34.687929  8678 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:34.688673 14770 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.662060 14755 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:34.665519  9388 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:04:34.539726  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 165563us
I20260709 13:04:34.692414  9382 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:04:34.692473 14168 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:34.573081  8575 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43482 (request call id 229) took 59513 ms (client timeout 15000 ms). Trace:
I20260709 13:04:34.675007 14382 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:34.693583 14382 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.694005 14382 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.694090  8794 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:34.666779  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.050s	sys 0.045s
W20260709 13:04:34.694823  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.051s	sys 0.045s
I20260709 13:04:34.695253  8794 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:34.695521  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 568083us
I20260709 13:04:34.506529 14765 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:34.672523 14771 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:34.664536  8673 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:34.699810 13702 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:34.699901  9355 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:34.692440 14476 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:34.701771 13702 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:34.702840  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:34.703410 13702 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.712713 14688 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:34.712986  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:34.689373  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 241425us
I20260709 13:04:34.713811  8748 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:34.714577  8752 raft_consensus.cc:3055] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:34.715010  8752 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:34.716001  8752 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.717401  8752 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:34.713729 14688 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:04:34.722506  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.724351  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.726130  8747 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:34.729725  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:34.640358 14652 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:34.642829  8645 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:34.698328 14765 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:34.746151  8674 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:34.734570  9385 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:04:34.713761  8675 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:34.738675  8681 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:34.739547  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.749682  8754 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
W20260709 13:04:34.742472  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.750861  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:34.751127  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 47944.5us
I20260709 13:04:34.751602  8755 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:34.743739  9355 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:34.752357  8796 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:34.753259  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:34.753319  8672 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:34.754983  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:34.755940  8678 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:34.760109  8679 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:34.761493  9355 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:34.748725  9353 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:34.765506  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 122798us
I20260709 13:04:34.766531  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:34.745385 14659 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:34.745626 14683 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:34.745808 14312 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:34.785809  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 111412us
W20260709 13:04:34.797235  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.754365 14537 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:34.799724  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:34.800529  8675 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:34.801576 14783 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:34.802075 14783 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:34.802284  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 53329us
I20260709 13:04:34.802405 14783 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:34.803789  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:34.804090  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50282.7us
I20260709 13:04:34.804544  8749 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:34.805889  9352 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:34.806434  8749 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:04:34.765615 14673 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:34.807101  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50478.3us
I20260709 13:04:34.772962 14519 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:34.779809 14611 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:34.808907  8719 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:34.845753  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:34.771924 14671 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:04:34.848385  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:34.849128  8749 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:04:34.766074 14444 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:34.780900 14607 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:34.767309 14462 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:04:34.791487 14589 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:04:34.861277  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.790982 14767 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:34.799196 14312 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.863348  8748 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
W20260709 13:04:34.855190  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 49367.7us
I20260709 13:04:34.765430 14168 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:04:34.765091 14437 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:34.859447  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.859953  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:34.864233  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 182307us
I20260709 13:04:34.865620  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:34.865797  9352 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:34.858798  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.843313 14611 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.833184 14659 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:34.894068  8748 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:34.852644 14394 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:34.851754 14673 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:34.912513  8680 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:34.914250  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:34.863385 14589 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.861915 14519 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:34.897123  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:34.864161 14766 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:34.918059  8748 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:34.868367 14312 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:34.919479 14481 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:04:34.896145 14607 raft_consensus.cc:697] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:34.912159 14394 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:34.961162 14394 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:34.896560 14626 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:04:34.924554 14589 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:34.963660 14382 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:34.920323 14766 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.924448 14701 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:04:34.966956 14766 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:34.926563 14623 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:34.968830 14623 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.969285 14623 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:34.969785 14781 log.cc:927] Time spent T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:04:34.729544  8575 rpcz_store.cc:269] 0709 13:03:35.059813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.060186 (+   373us) service_pool.cc:224] Handling call
0709 13:04:34.572792 (+59512606us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:30.611677 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:30.637155 (+ 25478us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:30.637210 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:04:30.637231 (+    21us) tablet.cc:662] Decoding operations
0709 13:04:30.637975 (+   744us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:30.638079 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:30.638105 (+    26us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:04:30.638547 (+   442us) tablet.cc:701] Row locks acquired
0709 13:04:30.638598 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:04:30.694345 (+ 55747us) write_op.cc:270] Start()
0709 13:04:30.754538 (+ 60193us) write_op.cc:276] Timestamp: P: 1783602270657636 usec, L: 0
0709 13:04:30.758105 (+  3567us) op_driver.cc:348] REPLICATION: starting
0709 13:04:30.887415 (+129310us) log.cc:844] Serialized 1303 byte log entry
0709 13:04:32.107719 (+1220304us) op_driver.cc:464] REPLICATION: finished
0709 13:04:32.112555 (+  4836us) write_op.cc:301] APPLY: starting
0709 13:04:32.112650 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:32.114328 (+  1678us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:32.114413 (+    85us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:32.481548 (+367135us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:32.685834 (+204286us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 13:04:32.703118 (+ 17284us) write_op.cc:312] APPLY: finished
0709 13:04:32.773732 (+ 70614us) log.cc:844] Serialized 41 byte log entry
0709 13:04:32.774095 (+   363us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:32.774359 (+   264us) write_op.cc:454] Released schema lock
0709 13:04:32.774471 (+   112us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:32.775386 (+   915us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:32.775433 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6208417,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":4534,"apply.run_cpu_time_us":12111,"apply.run_wall_time_us":662692,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16655,"num_ops":3,"prepare.queue_time_us":787543,"prepare.run_cpu_time_us":160208,"prepare.run_wall_time_us":1134259,"raft.queue_time_us":22999,"raft.run_cpu_time_us":4309,"raft.run_wall_time_us":5087,"replication_time_us":1349344,"thread_start_us":1242031,"threads_started":4,"wal-append.queue_time_us":937560,"wal-append.run_cpu_time_us":15145,"wal-append.run_wall_time_us":1966323}]]}
I20260709 13:04:34.971607  8747 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:34.962163 14607 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.974992 14768 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.966326 14382 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:34.975630 14768 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.979099 14519 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.980561 14540 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:34.981035 14540 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:34.913967 14719 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:34.982595 14719 raft_consensus.cc:697] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:34.983767 14719 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.930553 14765 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:04:34.898000 14611 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:34.854987 14777 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:04:34.982064 14659 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:34.989156 14673 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:34.992530  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.926316 14760 raft_consensus.cc:3055] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:34.994213 14760 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:34.995257 14760 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:34.996718 14760 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:34.989151  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 60956.4us
I20260709 13:04:35.000800  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:35.010880  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 334446us
I20260709 13:04:34.909898 14688 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:35.024597  8673 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:35.027904  8677 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:35.033890  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 172971us
W20260709 13:04:35.034896  8722 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.954s (SENT)
I20260709 13:04:34.896493 14444 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:34.870905 14652 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:35.045226  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:35.046941  8719 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:35.068477 14312 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:35.044938 14437 raft_consensus.cc:697] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:34.978500 14556 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:35.046085 14462 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:35.022728 14768 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:35.050733 14688 raft_consensus.cc:697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:35.058513 14583 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:35.062374 14778 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:35.085590  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:35.103440  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 91617us
W20260709 13:04:35.109091  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 45085.2us
I20260709 13:04:35.114516  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:35.088878  9352 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:35.089177  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 88507.5us
W20260709 13:04:35.082485  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 279109us
I20260709 13:04:35.110239  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:35.126003  8824 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:35.134954 14465 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:35.135751 14465 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:35.136837 14465 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:35.091045 14707 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:35.149823  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 61043.5us
I20260709 13:04:35.087224 14589 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:35.139160 14771 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:35.158736  8752 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:04:35.161257  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 95955.8us
I20260709 13:04:35.162417  8722 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:35.163004  8749 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:35.165115  8755 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:35.166213  8753 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:04:35.175103  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:35.184520  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 66406.4us
W20260709 13:04:35.190178  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50608.2us
I20260709 13:04:35.192647  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:35.164134 14607 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:35.156536 14790 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:35.150949 14707 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:35.211606 14760 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:35.186786 14444 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:35.201573  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:35.185958 14777 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:04:35.254004  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.280215  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:35.281270  8672 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:35.220818 14607 raft_consensus.cc:697] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:35.283326  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 90785.8us
I20260709 13:04:35.284263  8681 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:35.255692 14777 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.256201 14789 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.172186 14688 consensus_queue.cc:237] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:35.196966 14437 consensus_queue.cc:237] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:35.312536  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:35.315222  8827 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:35.316567 14659 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:35.318495  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.319351  8826 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:35.296061 14777 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:35.325672  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.297591 14312 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:35.329839  8828 raft_consensus.cc:2410] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:04:35.332168  8827 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:35.304996 14556 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:35.333971  9355 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:35.335330 14519 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:35.336097 14519 raft_consensus.cc:697] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:35.336221 14803 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:35.331923  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:35.341455  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.342696  8821 raft_consensus.cc:2410] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:04:35.367859  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:35.376170  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 85021.4us
I20260709 13:04:35.389479  8823 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:35.305339 14789 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:35.399196 14707 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:35.327452 14795 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:04:35.404682  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.408036  8678 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:35.410488  8828 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
W20260709 13:04:35.408720 14793 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 13:04:35.465360  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.400020 14589 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:35.451267 14783 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.648s	user 0.014s	sys 0.000s
I20260709 13:04:35.473114  8828 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:35.483908  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:35.485687  8673 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:04:35.457994 14794 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:04:35.499085  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:35.500233  8827 raft_consensus.cc:2410] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:35.500586 14540 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:35.502152 14540 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:35.503872 14798 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 13:04:35.505676 14797 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
W20260709 13:04:35.507628 14791 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
W20260709 13:04:35.509261 14799 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:04:35.516995  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 1205) took 1213 ms. Trace:
I20260709 13:04:35.517671  8824 rpcz_store.cc:276] 0709 13:04:34.278346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.290989 (+ 12643us) service_pool.cc:224] Handling call
0709 13:04:35.491845 (+1200856us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b5400126454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:35.491968 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1853571328}
W20260709 13:04:35.522928  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.564268  8674 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:35.403476 14519 consensus_queue.cc:237] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:35.396039 14607 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:35.565948  8673 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
W20260709 13:04:35.565688 14792 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 13:04:35.491732 14783 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:35.592476  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.593847  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.595127  8824 raft_consensus.cc:2410] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:04:35.593811  8828 raft_consensus.cc:2393] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:35.569212 14583 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
W20260709 13:04:35.598685 14801 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
W20260709 13:04:35.597973  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 422983us
W20260709 13:04:35.613715  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 451345us
W20260709 13:04:35.600040 14623 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.630s	user 0.012s	sys 0.002s
I20260709 13:04:35.623023  9385 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:35.625073  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:35.649315  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 167140us
W20260709 13:04:35.626739 14611 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.640s	user 0.015s	sys 0.001s
W20260709 13:04:35.625448 14806 log.cc:927] Time spent T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:04:35.657541  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.658906  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.663007  8828 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:35.659094  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:35.668462  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 471206us
I20260709 13:04:35.669858  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.670715  8828 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:35.669512  8645 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:35.673931  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:35.682885  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.635357 14808 consensus_queue.cc:1059] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:04:35.629992 14623 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:35.687393  8647 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:35.688607  8828 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:35.689258  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:35.689924  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:35.707379  8754 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:04:35.691180 14813 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
W20260709 13:04:35.692211 14816 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:04:35.693369 14815 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:04:35.673507 14825 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.685838  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.683965 14589 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:04:35.714468  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 56557.7us
I20260709 13:04:35.718364  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:35.721357  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:35.694793 14804 log.cc:927] Time spent T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 13:04:35.667661 14790 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:04:35.665540 14611 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:35.711378  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:35.736126  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.702116 14789 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:35.737663  8823 raft_consensus.cc:2410] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 13:04:35.731189 14394 consensus_meta.cc:287] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.769s	user 0.017s	sys 0.000s
W20260709 13:04:35.745639  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 58366.5us
I20260709 13:04:35.746135  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.695483 14707 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:04:35.749054  8830 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:35.768791  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.772459  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:35.772794  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.773121  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.773757  8825 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:35.741457 14812 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:35.739351 14589 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:35.736630 14825 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:35.809216  8824 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 14. Ops: []
I20260709 13:04:35.808436  8829 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
W20260709 13:04:35.813061  8722 leader_election.cc:336] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.536s (SENT)
I20260709 13:04:35.813568  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:35.814543  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:35.818531  8722 leader_election.cc:336] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.678s (SENT)
W20260709 13:04:35.805965 14766 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.819s	user 0.007s	sys 0.014s
W20260709 13:04:35.810753 14755 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.107s	user 0.010s	sys 0.006s
I20260709 13:04:35.827010  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:35.749459 14394 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:35.812171 14783 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:35.844251 14805 log.cc:927] Time spent T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.266s	user 0.001s	sys 0.001s
I20260709 13:04:35.885485  8680 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:35.851568 14394 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:35.851814  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.889096  8822 raft_consensus.cc:2410] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 13:04:35.853236 14827 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:04:35.857692  8722 leader_election.cc:336] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.734s (SENT)
I20260709 13:04:35.891047 14623 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:35.892151  8722 leader_election.cc:336] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.764s (SENT)
I20260709 13:04:35.892668  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:35.819833 14771 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:35.859238 14611 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:35.860430 14825 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:35.832232 14755 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:35.895339 14825 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:35.896103 14755 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:35.833251  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:35.862092 14439 raft_consensus.cc:3055] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:35.897672 14439 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:35.898983  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.898679 14439 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:35.899920 14439 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:35.827843 14766 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:35.872125  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.815910 14808 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.887848  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:35.891119  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.903465  8821 raft_consensus.cc:2368] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17.
I20260709 13:04:35.915102 14808 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:35.900436  8722 leader_election.cc:336] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.882s (SENT)
W20260709 13:04:35.916627 14767 consensus_meta.cc:287] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.020s	user 0.010s	sys 0.007s
I20260709 13:04:35.905218 14766 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:35.914850 14589 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:35.929639  8680 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:35.930886  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:35.931102  8823 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:35.931581  8680 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:04:35.933092  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 160709us
I20260709 13:04:35.934433 14819 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:35.914897  8822 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:35.923174  8828 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:35.926307  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:35.944437  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:35.949020  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:35.951074  8680 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:35.956321  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:35.942740 14777 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.601s	user 0.013s	sys 0.001s
W20260709 13:04:35.961632  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 61694.8us
I20260709 13:04:35.962925  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:35.964737  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:35.966447  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:35.967644  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:35.967665  8828 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:35.968957  8722 leader_election.cc:336] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.893s (SENT)
I20260709 13:04:35.943213 14656 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.978565  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:35.966467 14777 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:35.979827 14790 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:35.980244 14836 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:35.955421 14611 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:35.956310 14755 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:35.983085  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:35.989125  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:35.960193 14583 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:36.001055  8822 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
W20260709 13:04:35.998639  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 278029us
I20260709 13:04:35.967670 14808 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:36.005826  8722 leader_election.cc:336] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.896s (SENT)
I20260709 13:04:36.012187  8828 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:36.014065  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.015142  8822 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:36.016142  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.016754  8822 raft_consensus.cc:2376] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17. Re-sending same reply.
I20260709 13:04:36.017858  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.018738  8822 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:36.019857  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.021740  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.022495  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:36.026894  8753 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:35.997145 14766 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.026782 14556 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:36.046284  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:36.047034  8751 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:04:36.047457  9352 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:36.047610  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.047998  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:36.042943  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.042826 14789 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:36.044097 14788 raft_consensus.cc:2764] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:04:36.056197  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.044977  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:36.057178  8828 raft_consensus.cc:2410] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:04:36.049678  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:36.057410  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:36.057972  8756 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 17: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:36.058061  8748 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:36.058401 14628 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:36.058962 14784 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:36.059103 14628 consensus_queue.cc:1243] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:36.059573 14683 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:04:36.050405  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 67460.5us
I20260709 13:04:36.065315  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:36.066027  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.048724  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:36.052184  8822 raft_consensus.cc:2393] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:36.068846  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:36.069496  8748 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:36.076442  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:36.080334 14830 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:04:36.083811 14847 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:36.087285  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.088115  8674 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:35.992193 14656 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:36.072204 14784 raft_consensus.cc:697] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:36.051440 14766 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:36.090973 14784 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:36.092828 14766 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:36.089745  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 83954.1us
I20260709 13:04:36.094105  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:36.096691  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:36.097364  9382 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
W20260709 13:04:36.097856  8564 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0e9b0850145a4c66bfe983ffdfa93a57 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:36.100206 14626 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:36.055085 14826 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:04:36.110751  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.112197  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.113030  8827 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:36.078135 14444 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:36.110841  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:36.120219  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 55014.2us
W20260709 13:04:36.116475  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 67832.5us
I20260709 13:04:36.117498  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:36.118247  8722 leader_election.cc:336] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.921s (SENT)
I20260709 13:04:36.121865  8722 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:36.122951  8828 raft_consensus.cc:2410] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 13:04:36.128336 14540 consensus_meta.cc:287] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.626s	user 0.010s	sys 0.010s
I20260709 13:04:36.130470  8826 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:36.130501 14623 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:36.131541 14444 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:36.131106  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.132845  8829 raft_consensus.cc:2410] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:04:36.132695 14444 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:36.133589  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.133829  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.134505  8825 raft_consensus.cc:2410] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:36.135700 14623 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:36.136909 14623 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:36.135540  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.141247  8829 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:36.135568  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.151738  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:36.152547  8750 raft_consensus.cc:1813] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Handling vote request from an unknown peer e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:36.153074  8750 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:36.152887  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.153710  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2500) took 1467 ms. Trace:
I20260709 13:04:36.155561 14828 raft_consensus.cc:3055] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:36.155757  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.156481  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.146673  8825 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:36.157122  8677 rpcz_store.cc:276] 0709 13:04:34.686042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.686169 (+   127us) service_pool.cc:224] Handling call
0709 13:04:35.027686 (+341517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.032143 (+  4457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.153444 (+1121301us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b54001f9850. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:36.153635 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":784554368}
I20260709 13:04:36.159019  8822 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:36.160212  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.160851  8827 raft_consensus.cc:2376] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15. Re-sending same reply.
I20260709 13:04:36.160283  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.162988  8825 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:36.165943  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.166666  8829 raft_consensus.cc:2376] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15. Re-sending same reply.
I20260709 13:04:36.167870  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.168507  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.169214  8672 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:04:36.169232  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.170176  8677 raft_consensus.cc:2368] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16.
I20260709 13:04:36.171140  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.164769 14382 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:36.173271 14828 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:36.173748  8672 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:36.156651 14766 consensus_queue.cc:1059] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
W20260709 13:04:36.172267  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 54055.3us
I20260709 13:04:36.181631  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:36.182595  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.281932  8673 raft_consensus.cc:2368] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13.
I20260709 13:04:36.185941  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.186549  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.188817  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:36.288982  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 100261us
I20260709 13:04:36.192883  8748 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:36.292603  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:36.157859  8751 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:36.240242  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:36.243221  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.189743  8829 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:36.298566  8747 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:36.302949  8827 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:36.304857 14683 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:36.305843  8722 leader_election.cc:336] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.945s (SENT)
I20260709 13:04:36.297227  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:36.309063  8822 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:36.309847  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.312887  8749 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:36.317108  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.318836  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:36.319912  8672 raft_consensus.cc:2410] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:04:36.330816  8750 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:36.280381 14771 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.337662  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.296317 14803 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:36.341686  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:36.347888 14774 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.516s	user 0.005s	sys 0.005s
I20260709 13:04:36.305979 14847 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:36.363770  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:36.364676  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.332872 14437 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:36.367596  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 85932.7us
I20260709 13:04:36.369107  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.370293  8672 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:04:36.369673  8722 leader_election.cc:336] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.027s (SENT)
I20260709 13:04:36.376748  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.378281  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.357681 14755 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:36.383256  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:36.388638  8674 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:36.389925  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.390991  8822 raft_consensus.cc:2410] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:04:36.391577  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:36.385301  8673 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:36.396222  8756 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:36.400549  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.406690  8793 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:36.409475  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.415139  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:36.359180 14771 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:36.421454  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:36.423100  8722 leader_election.cc:336] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.200s (SENT)
I20260709 13:04:36.407598 14790 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:36.434415  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:36.433982  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.419143 14767 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.441010  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.441282  9385 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:04:36.360952 14828 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:36.442673  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.420215 14766 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:36.444223  8674 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:04:36.443018 14828 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:36.445751 14767 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:36.412441 14755 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:36.448144 14767 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:36.449275  8672 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
W20260709 13:04:36.449796  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:36.451545  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 44981.1us
I20260709 13:04:36.452299  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:36.454362  8680 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:36.459087 14767 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:36.455334 14439 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:36.461730 14771 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:36.473225  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:36.427191 14683 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:36.472321  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.474538 14439 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.475206 14439 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:36.475518  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:36.475831 14755 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:36.476334  8673 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:36.477339  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:36.477666 14439 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:36.478653  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.478542  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:36.487169  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.479281  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:36.488029  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.486442  8825 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
W20260709 13:04:36.489123  8722 leader_election.cc:336] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.599s (SENT)
I20260709 13:04:36.494009  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.494149  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.495152  8825 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:36.495404  8822 raft_consensus.cc:2410] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:04:36.496377  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.496733  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.497329  8825 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:36.497532  8822 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:36.498484  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.499680  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:36.500659  8674 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:36.499610  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.503363  8673 raft_consensus.cc:1813] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Handling vote request from an unknown peer e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:36.507349  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.508157  8823 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:36.509253  8673 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:36.524425  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.526064  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.527470  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.528268  8822 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:36.529251  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.530145  8825 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:36.530405  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.533247  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.533444  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.534114  8823 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:36.535825  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.536700  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.537596  8823 raft_consensus.cc:2410] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:04:36.537957  8793 leader_election.cc:400] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16.
I20260709 13:04:36.538319  8793 leader_election.cc:403] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:36.538647  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.540882  8645 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:36.543363 14537 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:36.543946 14537 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:36.544898 14537 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:36.552747  8829 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:36.555608  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.536770 14755 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.556833  8829 raft_consensus.cc:2410] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:36.558277  8822 raft_consensus.cc:2410] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:04:36.558175  8793 leader_election.cc:400] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13.
I20260709 13:04:36.559163  8793 leader_election.cc:403] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:04:36.559563  8678 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:04:36.559737  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.562299  8823 raft_consensus.cc:2410] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:04:36.567739  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.568061  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.560876 14766 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13.
I20260709 13:04:36.570227  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.572079  8822 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:36.536691 14465 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:04:36.557083 14444 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:36.577973  8722 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:36.577850  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:36.587839  8750 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:04:36.585012  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.594329 14437 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:36.598950 14437 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:36.600255  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:36.601349 14437 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:36.601887  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:36.605125 14671 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.605762 14671 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:36.608273 14671 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:36.617933 14707 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:36.568113 14755 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:36.626387  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:36.621853 14707 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:36.628005 14707 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:36.628851  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.621981 14465 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:36.630689 14465 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.628896 14766 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:36.631170 14465 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:36.633055  8829 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:04:36.633483  8829 pending_rounds.cc:85] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 10
I20260709 13:04:36.633739  8829 pending_rounds.cc:107] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:04:36.633981  8829 raft_consensus.cc:2889] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:36.635839  8829 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:36.636271  8825 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:36.638199  8822 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:36.642340  8822 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:36.643929  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2504) took 1367 ms. Trace:
I20260709 13:04:36.644474  8675 rpcz_store.cc:276] 0709 13:04:35.276208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.276359 (+   151us) service_pool.cc:224] Handling call
0709 13:04:35.276799 (+   440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:35.362278 (+ 85479us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:35.377220 (+ 14942us) log.cc:844] Serialized 14 byte log entry
0709 13:04:35.618215 (+240995us) log.cc:844] Serialized 14 byte log entry
0709 13:04:35.618591 (+   376us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:35.649273 (+ 30682us) log.cc:844] Serialized 52 byte log entry
0709 13:04:35.649535 (+   262us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:35.649573 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.649654 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:36.135129 (+485475us) spinlock_profiling.cc:243] Waited 5.95 ms on lock 0x7b18002b33c0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:36.602778 (+467649us) spinlock_profiling.cc:243] Waited 6.01 ms on lock 0x7b18002b33c0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:36.608829 (+  6051us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b18002b33c0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:36.643684 (+ 34855us) raft_consensus.cc:1685] finished
0709 13:04:36.643711 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:36.643867 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68442,"prepare.queue_time_us":1689,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":87,"raft.queue_time_us":114048,"raft.run_cpu_time_us":483,"raft.run_wall_time_us":22291,"spinlock_wait_cycles":34129408,"thread_start_us":212608,"threads_started":3,"wal-append.queue_time_us":221240}
I20260709 13:04:36.644059  8822 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:36.654832  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2334) took 1043 ms. Trace:
I20260709 13:04:36.655368  8824 rpcz_store.cc:276] 0709 13:04:35.611174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.611372 (+   198us) service_pool.cc:224] Handling call
0709 13:04:35.611847 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.811165 (+199318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.654484 (+843319us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54002283d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:36.654750 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":453340928}
I20260709 13:04:36.658934 14766 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:36.665208  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:36.671715  8793 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:36.692306  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:36.695317  8793 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:36.673286 14673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.463s	user 0.025s	sys 0.032s
I20260709 13:04:36.704850  8793 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:36.690362 14825 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:36.701287 14673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.503s	user 0.025s	sys 0.032s
I20260709 13:04:36.693151 14437 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:36.694381 14766 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:36.710389  8722 leader_election.cc:336] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.443s (SENT)
I20260709 13:04:36.748136  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:36.738224 14673 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:36.738996 14437 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:36.699986 14767 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:36.668855 14707 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:36.794164  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2511) took 1107 ms. Trace:
I20260709 13:04:36.835897  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:36.836921  9382 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
W20260709 13:04:36.797206  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 86848.1us
I20260709 13:04:36.801267 14444 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16.
I20260709 13:04:36.778395 14864 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:36.761771 14825 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:36.806342 14465 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:36.807145  8647 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:36.840624  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:36.845994  8675 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:36.853264  8722 leader_election.cc:336] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.604s (SENT)
I20260709 13:04:36.835582  8753 rpcz_store.cc:276] 0709 13:04:35.686867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.686991 (+   124us) service_pool.cc:224] Handling call
0709 13:04:35.690839 (+  3848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:36.027381 (+336542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.793844 (+766463us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b540038bdd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:36.794084 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756614912}
I20260709 13:04:36.880491  8722 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:36.881765  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.882922  8752 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:04:36.883658  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 76686.3us
I20260709 13:04:36.885447  8675 raft_consensus.cc:2410] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 22 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:04:36.884581  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:36.884676  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:36.888824  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.893464  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:36.831961 14707 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:36.853080 14673 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.879743 14537 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:36.855121 14766 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:36.914844  8676 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:36.910502  8823 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
W20260709 13:04:36.912017  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 207233us
I20260709 13:04:36.919407  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:36.892040 14771 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:36.895299 14611 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:36.853502 14808 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:36.928835  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 104785us
I20260709 13:04:36.897989 14444 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:36.900067 14807 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:36.853852 14656 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:36.935928 14808 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:36.938676  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 85455.5us
W20260709 13:04:36.939836  8722 leader_election.cc:336] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.646s (SENT)
I20260709 13:04:36.915839 14673 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:36.940268  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:36.941130 14864 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:36.944203 14808 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:36.951629  8679 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:36.952241  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:36.952960  9382 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:36.954129  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:36.954928  9388 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:04:36.955183  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2326) took 1609 ms. Trace:
I20260709 13:04:36.955648  8821 rpcz_store.cc:276] 0709 13:04:35.345432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.346077 (+   645us) service_pool.cc:224] Handling call
0709 13:04:36.954940 (+1608863us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b54003ac0d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:36.955116 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1212607232}
I20260709 13:04:36.957019  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:36.957799  8821 raft_consensus.cc:2393] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:36.960340  8796 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:36.954314  8680 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:36.963754  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2349) took 1190 ms. Trace:
I20260709 13:04:36.964319  8826 rpcz_store.cc:276] 0709 13:04:35.773581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.814211 (+ 40630us) service_pool.cc:224] Handling call
0709 13:04:36.963536 (+1149325us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b540016a154. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:36.963682 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724726656}
W20260709 13:04:36.964769  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.963945  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:36.966686  8821 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:36.925004 14766 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:36.968909 14766 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:36.969367  8821 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:36.970521 14825 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:36.973508  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:36.976058  9355 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:36.942368 14673 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:36.987231  9380 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:36.899461  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:36.989411 14766 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:36.997615  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 98244.8us
I20260709 13:04:36.999910  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:37.002132  9382 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:37.002211  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:36.997463  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:37.004454  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:36.991493  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:36.972029 14767 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.976125 14671 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.003700  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:36.914980 14537 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:36.990027 14683 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.004123 14850 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:36.971762 14444 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:37.030447 14611 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:37.000806 14760 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:04:37.033263 14673 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:37.055758  8672 raft_consensus.cc:2410] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:04:37.033830 14869 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:37.000988 14825 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:04:37.004627 14623 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:04:37.070775  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2506) took 1782 ms. Trace:
I20260709 13:04:37.051342 14850 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:37.079493  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:37.087517  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 85919.5us
I20260709 13:04:37.040386 14767 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.095225  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:37.040508 14671 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.048457 14683 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:37.106807  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.071903 14607 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:37.051589 14870 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
W20260709 13:04:37.109156  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 112777us
I20260709 13:04:37.071473 14819 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:37.069136 14869 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.083906 14465 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:37.085158 14850 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:37.076340 14803 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:37.120676  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 118528us
I20260709 13:04:37.121754 14869 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:37.138981  9381 raft_consensus.cc:2368] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20.
I20260709 13:04:37.142688  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:37.111167 14766 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:37.143524  8679 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:37.144717  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:37.145352  8679 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:04:37.143571 14766 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.149014  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1142) took 1092 ms. Trace:
I20260709 13:04:37.149766 14766 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:37.149973  9385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 876) took 1054 ms. Trace:
I20260709 13:04:37.119792 14656 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.150377  9385 rpcz_store.cc:276] 0709 13:04:36.095941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.096828 (+   887us) service_pool.cc:224] Handling call
0709 13:04:36.097146 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:36.441986 (+344840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.149714 (+707728us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b540046a5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:37.149913 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":790692224}
I20260709 13:04:37.150985 14656 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.153296 14656 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:37.154170  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2905) took 1086 ms. Trace:
I20260709 13:04:37.154834 14537 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:37.154632  8756 rpcz_store.cc:276] 0709 13:04:36.068211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.181571 (+113360us) service_pool.cc:224] Handling call
0709 13:04:36.181925 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:36.396748 (+214823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.153905 (+757157us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b5400156dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:37.154107 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":492250880}
W20260709 13:04:37.122677 14871 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:04:37.157043 14671 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:37.160604 14537 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:37.113679 14719 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 13:04:37.161458  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:37.116452 14808 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:37.162477  8676 raft_consensus.cc:2410] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:04:37.162925 14808 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.163724  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:37.164052  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:37.164434  8676 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:04:37.162495 14803 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:04:37.126686 14771 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.127921 14870 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.125265 14444 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:37.166301 14771 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.166559 14870 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.167025 14755 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:37.169087 14870 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:37.169186  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:37.154336 14168 raft_consensus.cc:3055] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:37.170573 14168 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:37.171440  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:37.171597 14168 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:37.172385  8679 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:37.172847 14168 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:37.175139 14537 consensus_queue.cc:1243] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:37.129299 14767 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:37.133111  8722 leader_election.cc:336] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.841s (SENT)
I20260709 13:04:37.133563 14683 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:37.117448 14607 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.177801  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:37.160977  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:37.085198 14875 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
W20260709 13:04:37.179270  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46218.5us
I20260709 13:04:37.131523 14394 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:37.072695 14673 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.180666 14837 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:37.165429 14808 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:37.180035  8722 leader_election.cc:336] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.842s (SENT)
I20260709 13:04:37.178177 14607 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:37.182427 14673 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:37.184546  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:37.205722  9381 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:37.169127 14771 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:37.182783  8647 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:37.179970 14869 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.193624  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:37.208340  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:37.209146  8752 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:37.209865 14869 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:37.198009  8676 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:37.149439  8749 rpcz_store.cc:276] 0709 13:04:36.056319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.056445 (+   126us) service_pool.cc:224] Handling call
0709 13:04:36.056715 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:36.313285 (+256570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.148760 (+835475us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54001572d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:37.148947 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588443904}
I20260709 13:04:37.201102 14777 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:37.211009  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:37.213451 14873 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:04:37.203385  8750 pending_rounds.cc:85] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 7
I20260709 13:04:37.214859  8750 pending_rounds.cc:107] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:04:37.215205  8750 raft_consensus.cc:2889] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:37.190249 14803 raft_consensus.cc:3055] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:37.215726 14803 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:37.201423  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:37.217721  8753 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:04:37.090750  8681 rpcz_store.cc:276] 0709 13:04:35.288843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.293300 (+  4457us) service_pool.cc:224] Handling call
0709 13:04:35.311693 (+ 18393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.311952 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:35.597868 (+285916us) log.cc:844] Serialized 14 byte log entry
0709 13:04:35.737837 (+139969us) log.cc:844] Serialized 14 byte log entry
0709 13:04:35.930218 (+192381us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b180046b220. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32e63cd58f 00007f32df40c571 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd
0709 13:04:36.971497 (+1041279us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b180046b220. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:04:37.070273 (+ 98776us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:37.070353 (+    80us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:37.070408 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.070512 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.070704 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73938,"prepare.queue_time_us":58148,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":2973,"raft.queue_time_us":255922,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":4238,"spinlock_wait_cycles":16215040,"thread_start_us":442713,"threads_started":3,"wal-append.queue_time_us":145136,"wal-append.run_cpu_time_us":2242,"wal-append.run_wall_time_us":1111228}
I20260709 13:04:37.226945  8752 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:37.230450  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:37.232441  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:37.229506  9352 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:37.240731  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:37.241250  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:37.241587  8748 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
W20260709 13:04:37.242141  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 59577.1us
I20260709 13:04:37.243975  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:37.244648  8748 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:37.216966 14803 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.246348 14782 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:37.249106 14782 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.249300 14803 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:37.249511 14782 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:37.250905  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:37.252244  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:37.245862 14312 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:04:37.253978  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:37.255769 14811 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:37.256582 14811 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:37.256963 14811 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:37.256860 14382 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:37.257503 14811 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:37.258798 14589 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:37.259038  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:37.260735  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:37.262706  8796 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:37.263846  8647 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:37.262697  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:37.317801  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:37.321264  8748 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
W20260709 13:04:37.327520  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 80452.7us
I20260709 13:04:37.330077  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:37.331060  9383 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 17: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:37.333976 14850 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.335184  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:37.336028  9380 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:37.336593 14850 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:37.337111 14607 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.338718 14790 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.339258 14790 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:37.339834 14782 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.341720 14790 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:37.342060 14782 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:37.342325 14807 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.342857 14807 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:37.343804  9387 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 18: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:37.268896 14589 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:37.345495  9385 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:04:37.345662 14807 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:37.352200 14802 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.352993 14673 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.352905 14802 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:37.353691 14673 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.321751 14869 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:37.358273 14869 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.358829 14869 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:37.363147  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:37.363976  8828 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:37.366297  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:37.367022  8749 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 19
W20260709 13:04:37.367928  8564 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0e9b0850145a4c66bfe983ffdfa93a57 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:37.378132  8676 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:04:37.382822 14607 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:37.383091  8677 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:04:37.389627  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:37.383975  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:37.390990  8672 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 21
I20260709 13:04:37.393774  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:37.395495  8756 pending_rounds.cc:85] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 7
I20260709 13:04:37.395884  8756 pending_rounds.cc:107] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:04:37.396195  8756 raft_consensus.cc:2889] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:37.398839  8748 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:37.399592  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:37.400188  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.400274  8747 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:37.400701  8748 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:37.401613  8747 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:04:37.409017  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:37.410094  8827 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:37.411749  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:37.427135  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:37.460032  8827 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:37.402370  8748 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:04:37.460927  8748 pending_rounds.cc:85] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 10
I20260709 13:04:37.461241  8828 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
W20260709 13:04:37.461450  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 98737.6us
W20260709 13:04:37.457230  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.462268  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:37.464804  8747 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:04:37.465884  8722 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.805s (SENT)
I20260709 13:04:37.466519  8722 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:37.458906  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:37.468231  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 48464.7us
I20260709 13:04:37.469820  8825 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:37.458865  8681 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:04:37.427029 14540 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:37.469585  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2429) took 1301 ms. Trace:
I20260709 13:04:37.473978  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:37.474967  8827 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:37.457643  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:37.461275  8748 pending_rounds.cc:107] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:04:37.476717  8748 raft_consensus.cc:2889] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:37.408918  9380 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:04:37.479118  8796 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:37.473281  8823 rpcz_store.cc:276] 0709 13:04:36.168212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.567834 (+399622us) service_pool.cc:224] Handling call
0709 13:04:37.469350 (+901516us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54003ddbd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:37.469516 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748634368}
W20260709 13:04:37.481931  8824 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.012s	sys 0.038s
I20260709 13:04:37.484740  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:37.482875  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:37.488687  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:37.489183  8823 raft_consensus.cc:1275] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:37.489558  8678 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:04:37.487288  8824 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.012s	sys 0.038s
W20260709 13:04:37.493862  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.035s	sys 0.081s
I20260709 13:04:37.456712 14875 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.474804 14465 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:37.455752 14673 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:37.497766  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 506330us
W20260709 13:04:37.500669  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.035s	sys 0.081s
I20260709 13:04:37.464497 14688 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.504374  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2426) took 1345 ms. Trace:
W20260709 13:04:37.517620  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 596774us
I20260709 13:04:37.402968  8749 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
W20260709 13:04:37.509177  8822 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.034s	sys 0.070s
I20260709 13:04:37.488588 14870 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:37.514673  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:37.537421  8647 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:37.537986  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.538968  8647 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:37.527081  8822 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.036s	sys 0.072s
I20260709 13:04:37.487746 14444 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:37.518671  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2433) took 1322 ms. Trace:
I20260709 13:04:37.473018 14837 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:04:37.496112 14696 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:37.464689 14767 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:04:37.481652 14881 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:37.526929 14683 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 13:04:37.594190 14656 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:37.594779 14656 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.595214 14656 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.580195 14777 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.597203 14777 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:37.599088 14872 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:04:37.600016 14777 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:37.533409  9353 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:37.506146 14439 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:37.517832  8829 rpcz_store.cc:276] 0709 13:04:36.158725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.647416 (+488691us) service_pool.cc:224] Handling call
0709 13:04:36.647945 (+   529us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:36.664888 (+ 16943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.664966 (+    78us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:36.669648 (+  4682us) log.cc:844] Serialized 52 byte log entry
0709 13:04:36.900852 (+231204us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:37.498328 (+597476us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.503861 (+  5533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.503996 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.504059 (+    63us) raft_consensus.cc:1685] finished
0709 13:04:37.504080 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.504286 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119252,"raft.queue_time_us":26149,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":20300,"thread_start_us":636765,"threads_started":2,"wal-append.queue_time_us":334767}
W20260709 13:04:37.606736  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 73426.6us
I20260709 13:04:37.607715  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:37.497110 14394 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:04:37.532524 14802 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:37.501890 14785 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
I20260709 13:04:37.568341  8824 rpcz_store.cc:276] 0709 13:04:36.196146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.656516 (+460370us) service_pool.cc:224] Handling call
0709 13:04:36.656933 (+   417us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:36.667427 (+ 10494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.667479 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:36.691332 (+ 23853us) log.cc:844] Serialized 52 byte log entry
0709 13:04:36.873452 (+182120us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:37.498389 (+624937us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.518088 (+ 19699us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.518308 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.518392 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.518436 (+    44us) raft_consensus.cc:1685] finished
0709 13:04:37.518450 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.518578 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28996,"prepare.queue_time_us":644208,"raft.queue_time_us":29978,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":5320,"thread_start_us":699049,"threads_started":2,"wal-append.queue_time_us":230880}
I20260709 13:04:37.617151  8829 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:04:37.590443 14871 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:04:37.578923 14444 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.591782 14696 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.612319  8672 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 21.
I20260709 13:04:37.586872 14611 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:37.528517 14766 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:04:37.590425 14818 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:37.511734 14882 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.124s
I20260709 13:04:37.593331 14881 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.629297 14672 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:04:37.531757 14688 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:37.626214 14777 raft_consensus.cc:3055] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:37.630924 14790 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:37.631914 14444 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.637113 14696 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.623867 14782 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:37.659931  8824 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:04:37.663406 14671 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:37.664963 14786 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:37.666581 14719 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:37.667953 14623 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:37.671756  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:37.655567 14881 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:37.658052 14790 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:37.681671  8673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.016s	sys 0.001s
I20260709 13:04:37.657631 14777 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:37.636234 14611 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:37.716845  8673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.016s	sys 0.001s
W20260709 13:04:37.741708  8750 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.051s	sys 0.055s
I20260709 13:04:37.721702 14790 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:37.742872  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:37.745468  8750 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.051s	sys 0.056s
I20260709 13:04:37.732566 14611 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:37.753939  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:37.759476  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:37.760627  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2431) took 1591 ms. Trace:
I20260709 13:04:37.761083  8822 rpcz_store.cc:276] 0709 13:04:36.168783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.649115 (+480332us) service_pool.cc:224] Handling call
0709 13:04:36.689693 (+ 40578us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:36.853426 (+163733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.853460 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:36.853983 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:04:37.539925 (+685942us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:37.539970 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.540048 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.760330 (+220282us) raft_consensus.cc:1685] finished
0709 13:04:37.760355 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.760522 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8305,"raft.queue_time_us":13476,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":23495,"thread_start_us":775203,"threads_started":2,"wal-append.queue_time_us":854590}
I20260709 13:04:37.764101  9353 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:37.718490 14782 consensus_queue.cc:1048] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:04:37.735864 14683 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
W20260709 13:04:37.777675  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 258701us
I20260709 13:04:37.654631 14818 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.516731 14875 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:37.786309  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:37.787158  8822 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:37.778780 14683 consensus_queue.cc:1243] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:04:37.786507 14810 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.006s	sys 0.026s
I20260709 13:04:37.805670  8824 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:04:37.806020  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 339795us
I20260709 13:04:37.792750 14786 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:37.806027 14810 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.007s	sys 0.026s
I20260709 13:04:37.768611 14777 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:37.806869  8719 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:37.838194 14688 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:37.849422 14777 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:37.852725  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2397) took 1748 ms. Trace:
I20260709 13:04:37.853345  8825 rpcz_store.cc:276] 0709 13:04:36.104223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.637379 (+533156us) service_pool.cc:224] Handling call
0709 13:04:37.469522 (+832143us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:37.470784 (+  1262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.852378 (+381594us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54003ddbd0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:37.852624 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912619264}
I20260709 13:04:37.871170  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:37.834429 14837 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:37.889554  8822 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:04:37.898350  8719 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:37.899645  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 433765us
W20260709 13:04:37.901916  8722 leader_election.cc:336] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 3.001s (SENT)
I20260709 13:04:37.902438  8722 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:37.896569  9355 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:37.872567 14850 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.013s	sys 0.025s
I20260709 13:04:37.913522  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:37.916095  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:37.918768  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 379882us
I20260709 13:04:37.920492  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:37.921710  8822 raft_consensus.cc:2368] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18.
I20260709 13:04:37.922886  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:37.924484  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:37.925242  8822 raft_consensus.cc:2376] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13. Re-sending same reply.
I20260709 13:04:37.910351 14688 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:04:37.915220 14850 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.014s	sys 0.026s
I20260709 13:04:37.926342  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:37.930250  8822 raft_consensus.cc:2368] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16.
I20260709 13:04:37.922358  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:37.930780  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:37.935731  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:37.910869 14782 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:37.904299 14837 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:37.927660 14439 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:37.927588 14688 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.968864 14782 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:37.973011  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 71154.1us
I20260709 13:04:37.978499  8822 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:37.979745  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:37.980584 14465 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:37.980587 14382 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:37.983247  8822 raft_consensus.cc:2410] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 13:04:37.972582 14803 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.723s	user 0.011s	sys 0.003s
I20260709 13:04:37.985635  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:37.989629  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 58314.5us
I20260709 13:04:38.005364  8822 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:38.021821  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:38.023056 14898 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:04:38.027226 14782 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:38.029119 14811 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.771s	user 0.014s	sys 0.001s
I20260709 13:04:37.995800 14818 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:38.029760 14811 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.031236 14875 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:38.039633  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1230) took 1194 ms. Trace:
W20260709 13:04:38.042907 14869 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.684s	user 0.013s	sys 0.002s
I20260709 13:04:38.043663 14869 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:38.045248  8830 raft_consensus.cc:2368] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15.
I20260709 13:04:37.973469 14688 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:38.040324  8821 rpcz_store.cc:276] 0709 13:04:36.845063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.971882 (+126819us) service_pool.cc:224] Handling call
0709 13:04:37.031672 (+ 59790us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:37.056465 (+ 24793us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.056508 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:37.057225 (+   717us) log.cc:844] Serialized 26 byte log entry
0709 13:04:37.482127 (+424902us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:37.482168 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.039004 (+556836us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b54002292d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:38.039082 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:38.039161 (+    79us) raft_consensus.cc:1685] finished
0709 13:04:38.039182 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.039538 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3053,"raft.queue_time_us":25950,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4233,"spinlock_wait_cycles":34528000,"thread_start_us":376109,"threads_started":1,"wal-append.queue_time_us":654775}
I20260709 13:04:38.056190  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:38.057255  9385 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 15
W20260709 13:04:38.058780  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.064741  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:38.072928  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:38.073344  8722 leader_election.cc:336] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.724s (SENT)
I20260709 13:04:38.059844 14869 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:38.078455 14902 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:04:38.078593  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:38.071246 14891 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 13:04:38.026264 14611 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:38.083652 14897 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:04:38.084581 14444 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:38.085299 14892 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:04:38.085513 14611 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:38.086496  8825 raft_consensus.cc:2368] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16.
I20260709 13:04:38.087265 14818 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:38.088538 14818 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:38.088943 14818 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:38.089964  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2435) took 1810 ms. Trace:
I20260709 13:04:38.096046  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:38.095963  8826 rpcz_store.cc:276] 0709 13:04:36.279365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.965312 (+685947us) service_pool.cc:224] Handling call
0709 13:04:36.965680 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:37.070832 (+105152us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:37.243178 (+172346us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.087481 (+844303us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b18001db360. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e5ef13c7 00007f32e25039a7 00007f32e25130bf 00007f32e2513038
0709 13:04:38.087702 (+   221us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.087889 (+   187us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:38.087935 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:38.088009 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.089705 (+  1696us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b540016bf54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:38.089755 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.089898 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29569,"prepare.queue_time_us":918125,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":26276,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":7292,"spinlock_wait_cycles":243675392,"thread_start_us":611586,"threads_started":2,"wal-append.queue_time_us":596460}
I20260709 13:04:38.089898 14818 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:38.093380 14790 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.094988 14837 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:38.068465 14811 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:38.101786  8821 raft_consensus.cc:2410] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:04:38.060753 14900 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:38.103689  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:38.091344 14444 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:38.109483  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:38.076604 14899 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 13:04:38.112242  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:38.123198  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 124393us
I20260709 13:04:38.123770  9383 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:38.123235  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:38.111299  9385 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:38.125828  8796 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:38.136451  8722 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:04:38.127857 14870 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.006s	sys 0.004s
I20260709 13:04:38.127938 14696 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:38.154779  8821 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:38.134426 14869 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:38.130254 14900 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:38.090036 14881 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.143496 14818 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:38.161743 14870 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.006s	sys 0.004s
I20260709 13:04:38.152210 14790 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:38.191823  8827 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:38.192974  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 56656.5us
W20260709 13:04:38.174003 14656 consensus_meta.cc:287] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.578s	user 0.011s	sys 0.004s
I20260709 13:04:38.204174 14696 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:38.206936  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:38.208595 14912 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:38.209196 14912 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:38.215875 14656 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:38.173250 14915 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:38.222687  8829 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:04:38.190054 14870 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:38.209604 14912 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:38.243600  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 94844.5us
I20260709 13:04:38.216280 14777 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:38.219211 14869 raft_consensus.cc:697] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:38.219609 14795 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:38.224658 14696 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:04:38.246975  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 52902.4us
I20260709 13:04:38.253630  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:38.255287 14901 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:38.257019 14656 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:38.257335 14924 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:38.255234  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:38.261380  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:38.257922 14924 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.257750 14656 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.273207  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:38.257934 14881 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:38.249019 14439 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.292387  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:38.294699  9387 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:38.301021  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:38.282153 14915 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:38.251417 14777 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:38.259519 14696 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.290730 14911 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
W20260709 13:04:38.325347 14919 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:04:38.325600 14777 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:38.326831 14925 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:04:38.296406 14439 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:38.331956 14696 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:38.333204  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 1237) took 1196 ms. Trace:
I20260709 13:04:38.333678  8830 rpcz_store.cc:276] 0709 13:04:37.137262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.162431 (+ 25169us) service_pool.cc:224] Handling call
0709 13:04:37.389451 (+227020us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:04:38.332958 (+943507us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b5400126454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:38.333131 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1562724992}
I20260709 13:04:38.336591  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:38.338176  8823 pending_rounds.cc:85] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 10
I20260709 13:04:38.250406 14900 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:38.338925 14760 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.339514 14760 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.341796 14760 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:38.286758 14656 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.283321 14901 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:04:38.344317 14924 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:38.282583 14837 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:04:38.345413 14656 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:38.253456 14795 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:38.349581  8822 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:38.336840  9355 leader_election.cc:400] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15.
I20260709 13:04:38.351086  9355 leader_election.cc:403] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:38.342787 14869 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:38.354523  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:38.355334  8749 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:38.338506  8823 pending_rounds.cc:107] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:04:38.355943  8823 raft_consensus.cc:2889] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:38.356297  8753 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:04:38.357594  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:38.373096  9352 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:38.378233  9383 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:38.296694 14808 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:04:38.370905 14540 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:38.398473  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 124464us
I20260709 13:04:38.306888 14870 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:38.388406 14673 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:38.400892 14671 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15.
I20260709 13:04:38.422348 14671 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:38.423053  8722 leader_election.cc:400] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18.
I20260709 13:04:38.424407  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:38.429870  8722 leader_election.cc:400] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16.
I20260709 13:04:38.423022 14671 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:38.431900 14671 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:38.402695 14837 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:04:38.415717 14901 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:04:38.445071 14465 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:38.445714 14465 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:38.448140  8756 raft_consensus.cc:2410] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:04:38.445931 14912 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.399757 14540 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.479627  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:38.428898 14915 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:38.484272  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 147569us
W20260709 13:04:38.489686  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 116717us
I20260709 13:04:38.426599 14673 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.494292 14901 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:38.532963 14870 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:38.424692 14688 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:38.437958  8571 catalog_manager.cc:5697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:38.534296 14936 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.005s
W20260709 13:04:38.543622  8564 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0e9b0850145a4c66bfe983ffdfa93a57 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:38.561843  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1150) took 1574 ms. Trace:
W20260709 13:04:38.567457  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 647494us
W20260709 13:04:38.566891  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 211229us
I20260709 13:04:38.568394  8647 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:38.565806  8680 rpcz_store.cc:276] 0709 13:04:36.987601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.989706 (+  2105us) service_pool.cc:224] Handling call
0709 13:04:37.187417 (+197711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:37.187738 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:37.188336 (+   598us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.678554 (+490218us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.556924 (+878370us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b180030b8c0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:38.559070 (+  2146us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b180030b8c0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:04:38.559316 (+   246us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:38.559385 (+    69us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:38.559446 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.559541 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.561568 (+  2027us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400174b50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:38.561747 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":871365,"prepare.queue_time_us":8988,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":7885,"raft.queue_time_us":4623,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":13609,"spinlock_wait_cycles":487401984,"thread_start_us":489384,"threads_started":1,"wal-append.queue_time_us":597841}
I20260709 13:04:38.496445 14782 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:04:38.585300 14905 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:38.586808 14688 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:38.591655  9383 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:38.592660  9383 raft_consensus.cc:606] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:38.592866  8571 catalog_manager.cc:5697] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.596841 14904 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:38.597633 14904 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.602113 14912 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:38.545217 14808 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:04:38.621788  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:38.600091 14904 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:38.622197  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 53896.5us
I20260709 13:04:38.622591  8751 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:38.585992 14905 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:38.553412 14870 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.637243 14755 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:38.640201  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1238) took 1502 ms. Trace:
I20260709 13:04:38.619372 14810 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:38.657914 14810 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.658349 14905 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.652884 14537 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:38.659710 14537 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.660759 14810 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:38.652513  8829 rpcz_store.cc:276] 0709 13:04:37.137567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.636761 (+499194us) service_pool.cc:224] Handling call
0709 13:04:37.637175 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:38.223324 (+586149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.639843 (+416519us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b54001946d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:38.640119 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1345648896}
I20260709 13:04:38.661978 14537 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:38.662153 14945 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:04:38.619768 14790 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:38.663105 14790 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.665475 14790 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:38.667919  8571 catalog_manager.cc:5697] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 5 to 12, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 12 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:38.672079 14673 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:38.677335  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:38.683660 14870 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:38.684541  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 50617.1us
I20260709 13:04:38.688153  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1160) took 1318 ms. Trace:
W20260709 13:04:38.691572  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55968.6us
I20260709 13:04:38.694602  9383 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:38.695669  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:38.696775  9385 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:38.689780  8676 rpcz_store.cc:276] 0709 13:04:37.369280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.384988 (+ 15708us) service_pool.cc:224] Handling call
0709 13:04:37.385378 (+   390us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:37.385755 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.385783 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:37.386464 (+   681us) log.cc:844] Serialized 52 byte log entry
0709 13:04:37.718836 (+332372us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:37.850994 (+132158us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b48001002d8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:04:37.987485 (+136491us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.643244 (+655759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.643336 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:38.643380 (+    44us) raft_consensus.cc:1685] finished
0709 13:04:38.687815 (+ 44435us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.688062 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1294346,"prepare.queue_time_us":312965,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":59,"raft.queue_time_us":601583,"raft.run_cpu_time_us":3691,"raft.run_wall_time_us":464487,"spinlock_wait_cycles":39745792,"thread_start_us":565890,"threads_started":3,"wal-append.queue_time_us":382274}
I20260709 13:04:38.701203  9383 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:38.702132  9383 raft_consensus.cc:606] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:38.704166  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:38.707569  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:38.718066  8751 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
W20260709 13:04:38.725514  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:38.727975  8647 connection.cc:256] RPC call timeout handler was delayed by -1.1620827520000603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1603s for a response
W20260709 13:04:38.728489  8647 connection.cc:256] RPC call timeout handler was delayed by -1.0459827519999862s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
I20260709 13:04:38.731730  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:38.733346  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:38.743185  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:38.745334  8824 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:38.732741  8828 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:38.746206  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:38.746730  8822 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:38.693866 14771 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:38.724444  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:38.752738  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:38.758411  8645 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:38.759923  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1140) took 1907 ms. Trace:
I20260709 13:04:38.763816  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:38.765166  8751 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:38.764776  8830 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:38.710963 14810 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.760423  8675 rpcz_store.cc:276] 0709 13:04:36.852043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.896154 (+ 44111us) service_pool.cc:224] Handling call
0709 13:04:36.896462 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:36.896677 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:04:37.037145 (+140468us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.405412 (+368267us) spinlock_profiling.cc:243] Waited 2.33 ms on lock 0x7b18004cd140. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:04:38.759554 (+1354142us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:38.759622 (+    68us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:38.759674 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.759751 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.759866 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":945812,"prepare.queue_time_us":140197,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":4498,"raft.queue_time_us":869174,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":9952,"spinlock_wait_cycles":5363968,"thread_start_us":502994,"threads_started":2,"wal-append.queue_time_us":767178}
I20260709 13:04:38.768654  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:38.729413 14937 log.cc:927] Time spent T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:04:38.711503 14785 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:38.688884 14465 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.869825  8821 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:38.804776  8822 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:38.779474  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:38.871304  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:38.774967  8751 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:38.748525 14556 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:38.723403 14755 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:38.754191 14818 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:38.703890 14673 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
W20260709 13:04:39.012894  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:39.015535  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 257190us
I20260709 13:04:39.069955  8675 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:39.070837  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.071214  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.072253  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:38.777345 14825 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:38.760881 14881 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:38.760164 14912 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:38.698423 14870 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 13:04:39.073148  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.104122  8681 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:39.104990  8676 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:39.076429  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.079627  8677 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:04:39.079487  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.106807  8677 pending_rounds.cc:85] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 8
I20260709 13:04:39.107152  8677 pending_rounds.cc:107] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:04:39.050482 14785 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.059988 14688 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:39.064674 14444 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:39.082216 14943 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.312s	user 0.000s	sys 0.000s
I20260709 13:04:39.107471  8677 raft_consensus.cc:2889] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:39.060734 14901 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:38.698050 14611 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:04:39.089202  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.054284 14811 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:39.008647 14790 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:39.107604  8675 raft_consensus.cc:2410] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:04:38.866128 14771 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:38.876190 14810 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.154225  8678 raft_consensus.cc:2410] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:04:39.148640  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.156740  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:39.083412 14951 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:39.174934 14905 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:39.075129 14818 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:39.271663  8647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.094s	sys 0.171s
I20260709 13:04:39.112473 14934 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:39.154670 14688 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.100466 14825 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.323352  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1233) took 2441 ms. Trace:
I20260709 13:04:39.303013  8751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
W20260709 13:04:39.147470 14908 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.086s	user 0.001s	sys 0.001s
I20260709 13:04:39.311091  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1249) took 1345 ms. Trace:
I20260709 13:04:39.270385  8674 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:39.334580  8827 rpcz_store.cc:276] 0709 13:04:36.881875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.494081 (+612206us) service_pool.cc:224] Handling call
0709 13:04:37.496454 (+  2373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:38.194970 (+698516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.323030 (+1128060us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b5400125f54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:39.323273 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1598669696}
I20260709 13:04:39.299655  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.348824  8676 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:39.332199  8821 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:39.297746  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.336223  8751 raft_consensus.cc:606] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:39.350994  8751 raft_consensus.cc:614] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:39.324709  8647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.094s	sys 0.171s
W20260709 13:04:39.311125 14950 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
W20260709 13:04:39.302996 14947 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:04:39.342594  8823 rpcz_store.cc:276] 0709 13:04:37.965888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.337683 (+371795us) service_pool.cc:224] Handling call
0709 13:04:38.337967 (+   284us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:38.385919 (+ 47952us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:38.385959 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:38.386781 (+   822us) log.cc:844] Serialized 52 byte log entry
0709 13:04:38.670941 (+284160us) spinlock_profiling.cc:243] Waited 880 us on lock 0x7b18004634e0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:04:38.802194 (+131253us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:38.974549 (+172355us) log.cc:844] Serialized 14 byte log entry
0709 13:04:39.056024 (+ 81475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.056151 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:39.310456 (+254305us) raft_consensus.cc:1685] finished
0709 13:04:39.310655 (+   199us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.310961 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95806,"prepare.queue_time_us":211814,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":11593,"raft.queue_time_us":88512,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":12665,"spinlock_wait_cycles":2023168,"thread_start_us":130945,"threads_started":1,"wal-append.queue_time_us":906347}
I20260709 13:04:39.126197 14912 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.274331 14951 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:39.373688  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.392297  8675 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 9.
I20260709 13:04:39.111177 14881 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:39.293591 14904 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:39.318506 14465 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:39.149120 14444 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.317054 14771 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:39.179368 14954 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:39.383258  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:38.976825  8571 catalog_manager.cc:5697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.402343  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.409444 14785 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:39.175922 14901 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.192430 14811 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:39.448933 14811 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:39.410941 14611 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:39.451155 14611 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.453835 14611 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:39.411580 14810 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:39.413956  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:39.455700 14810 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.456321 14810 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.432649  8675 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:39.434566  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.366804 14755 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.437302  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.329391 14934 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.391517 14807 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:39.463502 14807 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.465655  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1164) took 1364 ms. Trace:
I20260709 13:04:39.466465  8748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:04:39.467283  8748 raft_consensus.cc:606] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:39.466116  8756 rpcz_store.cc:276] 0709 13:04:38.100984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.450118 (+349134us) service_pool.cc:224] Handling call
0709 13:04:38.455632 (+  5514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:38.456212 (+   580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:38.456254 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:38.456896 (+   642us) log.cc:844] Serialized 26 byte log entry
0709 13:04:38.618300 (+161404us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:38.618368 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.618479 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:39.442303 (+823824us) raft_consensus.cc:1685] finished
0709 13:04:39.442343 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.465574 (+ 23231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":156,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":6733,"thread_start_us":160949,"threads_started":1,"wal-append.queue_time_us":407804}
I20260709 13:04:39.468415 14688 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:39.441766 14941 raft_consensus.cc:993] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: : Instructing follower 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:04:39.486096  8679 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:04:39.403353 14951 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:39.478207  8824 consensus_meta.cc:287] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.000s
I20260709 13:04:39.523444  8824 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:39.479120  8678 raft_consensus.cc:2410] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:39.443483 14825 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:39.536397  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:39.537729  8751 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:39.539003  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.540740  8675 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:39.406373 14802 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.355738 14790 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.542876  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:39.462740 14908 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:04:39.437640 14912 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:39.548766  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:39.551419  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:39.466287 14807 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:39.412609 14904 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.534829 14444 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:39.571013  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:39.573279  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:39.555820 14810 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:39.559131 14901 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:39.578729  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:39.581259  8756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:04:39.582180  8756 raft_consensus.cc:606] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:39.582653  8756 raft_consensus.cc:614] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:39.486399 14954 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.561399 14963 raft_consensus.cc:1081] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Signalling peer 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:04:39.583549  8719 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:39.547866 14802 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:39.597775  8645 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.991s (SENT) [suppressed 19 similar messages]
I20260709 13:04:39.592463  8719 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:39.603483  8674 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:04:39.586845 14688 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:39.653697  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 61362.2us
I20260709 13:04:39.655531  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:39.656453  9383 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:39.441772  8571 catalog_manager.cc:5697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:39.658178  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:39.653254  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:39.594442 14901 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:39.601256 14913 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:39.588443 14912 consensus_queue.cc:1059] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:04:39.617897 14465 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:39.675760  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:39.655306  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:39.680189 14382 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:39.679582 14786 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.010s	sys 0.014s
I20260709 13:04:39.685773 14913 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:39.658546 14818 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:39.694394  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:39.708391 14954 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:39.698427 14913 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:39.692062 14786 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.010s	sys 0.014s
W20260709 13:04:39.686223  8646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.016s	sys 0.032s
W20260709 13:04:39.716197  8646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.017s	sys 0.032s
I20260709 13:04:39.724052  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:39.688603 14771 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:39.689025 14802 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:39.689654 14912 raft_consensus.cc:3055] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:39.690200 14382 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:39.719223 14954 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:39.742870  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 199658us
I20260709 13:04:39.723465 14465 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:39.728159 14785 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:39.733289 14771 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:39.734522 14382 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:39.732765 14807 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:39.734210 14825 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:39.728989 14688 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:04:39.733996 14912 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:39.736815 14818 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:04:39.756275 14771 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:39.748258 14954 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:39.751489 14790 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:39.770844  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 173011us
I20260709 13:04:39.759577 14807 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.687211  8571 catalog_manager.cc:5697] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.753494 14785 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:39.789165  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:39.790048  8756 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:39.767676  8675 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:55676
I20260709 13:04:39.795842  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:39.796139  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1169) took 1039 ms. Trace:
I20260709 13:04:39.796496  8680 rpcz_store.cc:276] 0709 13:04:38.756670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.757810 (+  1140us) service_pool.cc:224] Handling call
0709 13:04:38.758280 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.054255 (+295975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.054296 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:39.099180 (+ 44884us) log.cc:844] Serialized 26 byte log entry
0709 13:04:39.595843 (+496663us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:39.595902 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.596001 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:39.795882 (+199881us) raft_consensus.cc:1685] finished
0709 13:04:39.795906 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.796073 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":223270,"raft.queue_time_us":332238,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":25899,"thread_start_us":300996,"threads_started":1,"wal-append.queue_time_us":577128}
I20260709 13:04:39.797622 14954 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:39.798718 14889 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.890s	user 0.004s	sys 0.002s
I20260709 13:04:39.800592  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2836) took 3326 ms. Trace:
I20260709 13:04:39.799957  8571 catalog_manager.cc:5697] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:39.800978  8673 rpcz_store.cc:276] 0709 13:04:36.473821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.539678 (+ 65857us) service_pool.cc:224] Handling call
0709 13:04:36.555110 (+ 15432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:36.558950 (+  3840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.558989 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:36.569156 (+ 10167us) log.cc:844] Serialized 26 byte log entry
0709 13:04:37.161199 (+592043us) spinlock_profiling.cc:243] Waited 2.75 ms on lock 0x7b18000189a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:04:37.732549 (+571350us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:37.978100 (+245551us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.732806 (+754706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.742803 (+  9997us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:39.798059 (+1055256us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:39.800345 (+  2286us) raft_consensus.cc:1685] finished
0709 13:04:39.800368 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.800526 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":593004,"prepare.queue_time_us":299669,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":58,"raft.queue_time_us":611929,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":1862,"spinlock_wait_cycles":15641856,"thread_start_us":964912,"threads_started":3,"wal-append.queue_time_us":1263207}
I20260709 13:04:39.793535  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:39.773023 14802 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:39.807825  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:39.818064  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1157) took 2073 ms. Trace:
I20260709 13:04:39.818730  8754 rpcz_store.cc:276] 0709 13:04:37.745157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.745354 (+   197us) service_pool.cc:224] Handling call
0709 13:04:37.745808 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:37.796778 (+ 50970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.796826 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:37.796888 (+    62us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:37.796936 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.817652 (+2020716us) spinlock_profiling.cc:243] Waited 50.7 ms on lock 0x7b54002fd5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:39.817750 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.817976 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116631296}
W20260709 13:04:39.829602  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 55678us
I20260709 13:04:39.809846  8647 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:39.802063  8571 catalog_manager.cc:5697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 11 to 13, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 13 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:39.838824 14980 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:39.838229  8571 catalog_manager.cc:5697] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 6 to 13. New cstate: current_term: 13 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:39.841907  8571 catalog_manager.cc:5697] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 4 to 11, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 11 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:04:39.816493 14802 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:39.795626 14912 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:39.860690  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 50933.7us
I20260709 13:04:39.863274 14912 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:39.862993  8571 catalog_manager.cc:5697] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:39.864583 14807 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:39.867075  8647 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:39.867462  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:39.869338  8647 leader_election.cc:400] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20.
I20260709 13:04:39.869831  8647 leader_election.cc:403] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:39.868247  8754 raft_consensus.cc:2376] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18. Re-sending same reply.
I20260709 13:04:39.870415 14952 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:39.872383 14807 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20.
I20260709 13:04:39.877135  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:39.877956  8754 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:39.877419 14954 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:39.881446 14382 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:39.884109 14382 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:39.898941 14785 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:39.903443  8752 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:39.903954 14802 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:39.904000  8752 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:39.908135  8752 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:39.921698  8571 catalog_manager.cc:5697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 7 to 12, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 12 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:04:39.923677  8571 catalog_manager.cc:5697] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 11 to 15, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 15 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:39.925542  8571 catalog_manager.cc:5697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 11 to 15, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 15 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.928488  8571 catalog_manager.cc:5697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 10 to 11, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 11 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:39.951922  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:39.963084  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:39.975833  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.977600  8679 raft_consensus.cc:2410] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:04:40.000099  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.967288  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:39.929261 14771 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:39.934656 14893 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.969s	user 0.005s	sys 0.018s
I20260709 13:04:39.934779 14951 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:39.937799 14825 raft_consensus.cc:993] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: : Instructing follower 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:04:39.941236 14795 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:39.945140  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.947430 14883 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:39.947978 14465 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:39.949285 14934 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:40.038725  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55118.6us
W20260709 13:04:40.101895  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.040475  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:40.040880  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:40.103232  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 62536.2us
I20260709 13:04:40.104318  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:40.030090 14807 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:04:40.045209  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:40.107003  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:40.107600  8680 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:40.046753  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:40.043025  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:40.108767  8681 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:40.029695 14611 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:40.062503 14965 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.363s	user 0.000s	sys 0.001s
I20260709 13:04:40.093297 14444 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:40.105726  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:40.039593 14556 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:04:40.100926 14916 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:40.110162  8675 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:40.073791 14912 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.102062  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:40.107692  8672 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:40.117489  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:40.044184  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:40.100461 14465 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:40.109519 14934 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.128655 14915 raft_consensus.cc:1081] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Signalling peer 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:04:40.049928  8571 catalog_manager.cc:5697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 9 to 12, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 12 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:40.185644  8675 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:40.188004  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 70629.5us
I20260709 13:04:40.201670  8680 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:40.109647  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:40.172561 14771 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:40.107051 14952 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:40.201202  8719 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:40.180079 14611 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:40.184744 14883 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.181437 14972 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:40.327996  8681 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:40.294875  8672 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:04:40.302378  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 101235us
I20260709 13:04:40.194365 14825 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:40.186800 14980 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.125s
I20260709 13:04:40.184504 14444 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:40.324889 14382 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:40.277994 14936 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:40.276314 14875 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:40.275825 14785 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:40.377476 14589 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:40.191057 14916 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:40.340736 14893 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:04:40.207013 14900 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.339169  8675 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:40.192009 14912 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:40.200654 14934 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:40.402109 14589 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.402338 14785 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:40.420480  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 314771us
I20260709 13:04:40.378872 14790 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:40.397117 14382 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:40.386101 14952 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:40.411688  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 309600us
I20260709 13:04:40.386023 14825 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:40.407158  8679 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 12. Ops: []
I20260709 13:04:40.340984 14883 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:40.422210  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 213555us
I20260709 13:04:40.425590  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:40.434069  8793 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:40.436260  8793 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:40.468164  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:40.418731 14589 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:40.476269  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51175us
I20260709 13:04:40.402383 14936 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:40.481115  8647 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:40.407672 14829 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.308s	user 0.046s	sys 0.029s
I20260709 13:04:40.423471 14382 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:40.417943 14875 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:40.495281  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:40.496614 14537 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:04:40.484398 14893 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:04:40.571081  9385 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:04:40.573377  9387 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:40.500851 14903 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:40.505340 14949 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 1.415s	user 0.015s	sys 0.007s
W20260709 13:04:40.507632  8825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2403) took 2722 ms (client timeout 1724 ms). Trace:
I20260709 13:04:40.510246  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2918) took 3286 ms. Trace:
W20260709 13:04:40.591997  8825 rpcz_store.cc:269] 0709 13:04:37.784683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.870720 (+ 86037us) service_pool.cc:224] Handling call
0709 13:04:40.507135 (+2636415us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54002689d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:40.507463 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485909760}
I20260709 13:04:40.592145  8753 rpcz_store.cc:276] 0709 13:04:37.224318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.224500 (+   182us) service_pool.cc:224] Handling call
0709 13:04:38.337604 (+1113104us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b5800083198. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:04:38.337836 (+   232us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:38.356956 (+ 19120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.509794 (+2152838us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b54002ace54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:40.510152 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49635584}
I20260709 13:04:40.430384 14825 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:40.531524  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.427114 14790 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:40.418275 14900 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:40.419566 14993 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.109s
W20260709 13:04:40.578253 14989 log.cc:927] Time spent T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:04:40.483788 14807 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.585950  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:40.614923  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:40.388192 14465 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:40.532608 14829 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.449s	user 0.047s	sys 0.029s
I20260709 13:04:40.497649  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:40.422214 14611 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:40.600540  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.514432 14951 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:40.565776 14537 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:40.623157 14952 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:40.617223  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2530) took 3414 ms. Trace:
W20260709 13:04:40.630774 14986 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:04:40.631846 14944 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.633536 14439 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:40.637080 14881 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:40.639678 14540 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:40.640565 14803 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:40.642282  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2853) took 1466 ms. Trace:
I20260709 13:04:40.718079  8750 rpcz_store.cc:276] 0709 13:04:37.202460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.202694 (+   234us) service_pool.cc:224] Handling call
0709 13:04:37.203113 (+   419us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:37.216546 (+ 13433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.216583 (+    37us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:37.217330 (+   747us) log.cc:844] Serialized 78 byte log entry
0709 13:04:37.747775 (+530445us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:37.755503 (+  7728us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.760073 (+  4570us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.761080 (+  1007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.761164 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:39.933868 (+2172704us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:40.616887 (+683019us) raft_consensus.cc:1685] finished
0709 13:04:40.616924 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.617143 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8368,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":73,"raft.queue_time_us":2952,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":313,"spinlock_wait_cycles":8398720,"thread_start_us":529899,"threads_started":1,"wal-append.queue_time_us":686617}
I20260709 13:04:40.731160  8676 rpcz_store.cc:276] 0709 13:04:39.175642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.220167 (+ 44525us) service_pool.cc:224] Handling call
0709 13:04:39.311787 (+ 91620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:39.349587 (+ 37800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.642013 (+1292426us) spinlock_profiling.cc:243] Waited 36.5 ms on lock 0x7b54002567d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:40.642212 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83891584}
I20260709 13:04:40.646701  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2857) took 1249 ms. Trace:
I20260709 13:04:40.732455  8674 rpcz_store.cc:276] 0709 13:04:39.397177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.397463 (+   286us) service_pool.cc:224] Handling call
0709 13:04:40.646426 (+1248963us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54003395d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:40.646630 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321827584}
I20260709 13:04:40.732942  8672 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:40.647749 14766 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:40.649948  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:40.733903  8673 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:40.672776  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1248) took 2777 ms. Trace:
I20260709 13:04:40.735208  8826 rpcz_store.cc:276] 0709 13:04:37.895444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.124935 (+229491us) service_pool.cc:224] Handling call
0709 13:04:38.125314 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:38.128101 (+  2787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:38.128144 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:38.128907 (+   763us) log.cc:844] Serialized 26 byte log entry
0709 13:04:39.875432 (+1746525us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b18004637e0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:04:40.093917 (+218485us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:40.093967 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.094050 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.672131 (+578081us) raft_consensus.cc:1685] finished
0709 13:04:40.672166 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.672620 (+   454us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39631,"raft.queue_time_us":1188,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":4090,"spinlock_wait_cycles":9959680,"thread_start_us":218266,"threads_started":1,"wal-append.queue_time_us":2069440}
I20260709 13:04:40.737632  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:40.677093 14934 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:40.678714 14785 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:40.745728 14998 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:40.679258 14771 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.680622 14904 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:40.620473 14903 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:40.681969 14795 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:40.687388 14912 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:40.714476  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:40.587200  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 106186us
I20260709 13:04:40.624557 14996 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:40.761659  8647 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:40.722852  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:40.763003  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:40.731320 14465 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:40.763718  8674 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:40.764691  8826 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:40.729328  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:40.696379 14916 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:40.675136 14875 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:40.743561 14803 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:40.622483 14807 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:40.763041 14795 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:40.717211 14537 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:40.750186 14444 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:40.750044 14771 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:40.791781 14875 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.792460 14875 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:40.792666 14382 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:40.793845 14771 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:40.681926  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:40.751538  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 315331us
I20260709 13:04:40.796070  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:40.717872 14952 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:40.755874  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:40.738116 14766 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:40.746475 14998 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:40.791240 14767 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.791049 14803 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:40.799052  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:40.802929  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:40.790035 14795 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:40.760850 14994 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:40.810753  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:40.814808  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:40.769204 14904 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:40.730602 14944 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:40.771667 14540 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:40.771369 14921 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:40.600739  8571 catalog_manager.cc:5697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 11 to 15, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 15 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:04:40.837194  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:40.837585  8828 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:04:40.843150  9388 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:40.845695  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:40.846786  9380 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:40.851191  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:40.811465 14952 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:40.769210 14623 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:04:40.855818  8722 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:40.865859  8674 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:40.854741  8793 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:40.869519  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:40.873656  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:40.835855 14875 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:40.809549 14916 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:04:40.807400 14903 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 13:04:40.815276 14990 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:04:40.843604 14995 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:40.850565 14934 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:40.788130 14996 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:40.847811 14883 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:40.851779 14696 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:40.862196 14382 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:40.899811 14998 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:40.886816  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 77789.4us
I20260709 13:04:40.850476 14540 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:40.878306 14795 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:40.836369 14767 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:40.853129 14994 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:40.894080  8722 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:40.897912  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 253074us
I20260709 13:04:40.865402 14444 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:04:40.887458 14807 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:40.903275 14903 consensus_queue.cc:1243] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 21.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:04:40.855679 14921 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:40.909760 14900 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:40.938747  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:40.944270  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 50313.4us
I20260709 13:04:40.941133 14623 consensus_queue.cc:1243] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:40.962841  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:40.944975 14900 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:40.954823 14795 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:04:40.897332 14995 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:40.946818 15003 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:40.976455 15000 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:04:41.000636 14994 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:41.003518  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51514.9us
W20260709 13:04:41.005847  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 151223us
I20260709 13:04:41.008872  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:41.011749  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:40.965513 15002 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:41.017542  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.026950  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:41.031315  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:41.014089 14916 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:41.037663  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:41.038131  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:41.040818  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:41.042732  9388 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
W20260709 13:04:41.046236  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68903.6us
I20260709 13:04:41.023747 14623 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.048511  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:41.053758  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:41.054693  9386 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:41.033385 14944 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:41.058238 15009 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.023258 14696 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:04:41.061245  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:41.041404 14916 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:41.051733 14905 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:41.071264 14921 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:41.072513  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:41.075901  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.077152  8825 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:41.067737 14916 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:41.079320  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:41.079813  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.067231 14540 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:41.067229 14803 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:04:41.082686 15006 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:04:41.050077  8571 catalog_manager.cc:5697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 13, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 13 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:41.086844 14803 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.087917  8571 catalog_manager.cc:5697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:41.083915  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.092427  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.090116  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.094264  8824 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:41.095953  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.096889  8824 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:41.098088  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:41.099952  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.100746  8824 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:41.101831  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.057159 14623 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:41.103813  8824 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:41.107312  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:41.107844  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.087337 14803 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:41.108788  8824 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:41.110788  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:41.112337  8825 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:41.062076 15014 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:04:41.076900 14905 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:41.080291 14996 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:41.135190  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1184) took 1109 ms. Trace:
I20260709 13:04:41.156016  8679 rpcz_store.cc:276] 0709 13:04:40.026239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.113317 (+ 87078us) service_pool.cc:224] Handling call
0709 13:04:40.113655 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.428132 (+314477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.134774 (+706642us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5400174b54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:41.135084 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674514176}
I20260709 13:04:41.146062  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:41.129001  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.160684  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:41.132507  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:52324 (request call id 7) took 1530 ms. Trace:
I20260709 13:04:41.161614  8674 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:41.161520  8748 rpcz_store.cc:276] 0709 13:04:39.602679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.602863 (+   184us) service_pool.cc:224] Handling call
0709 13:04:41.132125 (+1529262us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b5400139554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24ff735 00007f32e24f79fa 00007f32e74bf4c9 00007f32e0a3bc54 00007f32e0a3bbe2 00007f32e0a3bb11 00007f32e0a3ba8c 00007f32e0a3ad42 00007f32e083448c 00007f32e08338c6
0709 13:04:41.132387 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54778752}
I20260709 13:04:41.163820  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:41.164186  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.139098  8752 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:41.108664 14589 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.591s	user 0.013s	sys 0.001s
I20260709 13:04:41.168960  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.170082  8826 raft_consensus.cc:2393] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:41.122218 14767 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:41.195375  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:41.138816 14905 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:41.179776 14537 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:41.181425  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:41.184103  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:41.114930  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.189081  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:41.189680  8755 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:04:41.136252 14825 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.526s	user 0.012s	sys 0.000s
I20260709 13:04:41.199767 14439 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.203209  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1148) took 3811 ms. Trace:
I20260709 13:04:41.215695  8755 rpcz_store.cc:276] 0709 13:04:37.392170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.402740 (+ 10570us) service_pool.cc:224] Handling call
0709 13:04:38.680513 (+1277773us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b5800083198. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:04:41.164953 (+2484440us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b5800083198. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4e9c 00007f32df3d4a5b 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:04:41.165207 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.202894 (+ 37687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.203139 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67500160}
I20260709 13:04:41.202417  8756 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:41.219319  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:41.220207  8756 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:41.205953  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:41.208524  8749 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:41.209484  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:41.211184  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.211925  8753 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:04:41.226454  8753 pending_rounds.cc:85] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 13
I20260709 13:04:41.226828  8753 pending_rounds.cc:107] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:04:41.227084  8753 raft_consensus.cc:2889] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:41.074157 15009 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:41.230278  8825 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:41.181425 14900 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:41.228240  8752 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:41.090059  8571 catalog_manager.cc:5697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 13 to 15, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 15 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:41.242085  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2555) took 1345 ms. Trace:
I20260709 13:04:41.192155 14795 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:41.244483  8825 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:41.247612  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:41.245528 14994 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:41.245764 14900 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:41.252542 14995 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:41.258410  8753 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:41.182790 14589 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:41.267410  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:41.268458  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1177) took 1511 ms. Trace:
I20260709 13:04:41.269213  8753 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:04:41.270731  8753 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:41.260637  8825 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:04:41.272717 14882 raft_consensus.cc:3055] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:41.241501 14465 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.273336 14882 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:41.274654 14623 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:41.247148  8752 rpcz_store.cc:276] 0709 13:04:39.896869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.902873 (+  6004us) service_pool.cc:224] Handling call
0709 13:04:39.903270 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.138879 (+1235609us) spinlock_profiling.cc:243] Waited 6.78 ms on lock 0x7b58008195d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24550aa 00007f32e24d7447 00007f32e24effe7 00007f32e24f17de 00007f32e24fbb49 00007f32e24fc416 00007f32e24f8d8f 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4
0709 13:04:41.241322 (+102443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.242006 (+   684us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15597312}
I20260709 13:04:41.269050  8677 rpcz_store.cc:276] 0709 13:04:39.757315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.757465 (+   150us) service_pool.cc:224] Handling call
0709 13:04:39.757839 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:39.788843 (+ 31004us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:39.832735 (+ 43892us) log.cc:844] Serialized 14 byte log entry
0709 13:04:40.073339 (+240604us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.073586 (+   247us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:40.073990 (+   404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.268135 (+1194145us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b54001f9354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:41.268208 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.268380 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44078,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":325,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":2108,"spinlock_wait_cycles":70131584,"thread_start_us":283557,"threads_started":2,"wal-append.queue_time_us":474764}
I20260709 13:04:41.281069  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:41.282754  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:41.272459 14916 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:41.283638  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 243528us
I20260709 13:04:41.273473  8825 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:41.287905  8752 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:41.219691 14825 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:41.293262  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:41.298954  8678 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:41.299150  8681 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 18: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:41.230839 14439 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:41.302857  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:41.304281  8755 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:41.305932  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:41.308018  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:41.308223  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 247258us
I20260709 13:04:41.315759  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:41.327292  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:41.339222  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.345695  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:41.347051  8749 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:41.347501  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:41.349537  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:41.319137 14946 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.116s	user 0.013s	sys 0.009s
I20260709 13:04:41.264082 14795 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:41.295892 14755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.607s	user 0.006s	sys 0.025s
I20260709 13:04:41.264637 14994 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:41.287376 14465 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:41.366963  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:41.374486  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:41.375020  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:41.375309  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:41.376024  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:41.383759  8755 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:41.385238  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:41.370049  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:41.364908 14946 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.173s	user 0.013s	sys 0.009s
I20260709 13:04:41.274508 14882 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:41.378573 14755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.687s	user 0.007s	sys 0.025s
I20260709 13:04:41.365923 14912 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:41.383042 14952 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.006s
I20260709 13:04:41.376468 14915 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:04:41.410360  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 101034us
I20260709 13:04:41.357838 15015 consensus_queue.cc:1059] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:04:41.417189  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:41.425683  8755 raft_consensus.cc:2410] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 20 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:04:41.383545 14767 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:04:41.446229  8756 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:41.442248  9353 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:41.419474 14911 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:04:41.440814 14912 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:41.420218 15014 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:41.429513 14882 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:41.418464 14952 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:41.479281 14439 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:41.526939  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 84807.8us
I20260709 13:04:41.450485 15016 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
W20260709 13:04:41.571377  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 201418us
I20260709 13:04:41.490403 14900 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:41.574553 14825 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:41.494695 15015 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:41.576467 14994 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:41.576640  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:41.498692 14755 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:41.578934  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:41.582690 14977 raft_consensus.cc:3055] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:41.588616 14537 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:41.589718  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 214786us
I20260709 13:04:41.633312  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:41.625815  8677 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:41.634945  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:41.591349 14998 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:41.635780  8677 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:41.593748 14766 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:41.596094 14819 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:41.612219 14795 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:41.623394 15015 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:41.597286 15039 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:04:41.488996 14589 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:41.609210  8679 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
W20260709 13:04:41.613620  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:41.578032 14934 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.674s	user 0.007s	sys 0.006s
W20260709 13:04:41.611174 15034 log.cc:927] Time spent T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
W20260709 13:04:41.630999  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 52146.9us
I20260709 13:04:41.602425 14465 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:41.639333  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:41.660832  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:41.627774 14771 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:41.613293 15014 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:04:41.632953 14777 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:41.662993  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.651441 14914 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:41.632237 14916 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:41.629508 14977 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:41.637917 14998 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:41.663640  8793 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:41.640513 14767 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:41.666376 15025 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:41.724949 14760 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:41.726789 15042 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 13:04:41.727993 15043 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:04:41.666710 14875 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:41.640892 14819 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:41.655752 15015 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:41.693262  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.659192 14934 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:41.719005 15009 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:41.719295 14777 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:04:41.720748 15031 log.cc:927] Time spent T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:04:41.721840 14977 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:41.722878 14998 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:41.723531 14952 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:41.664911 14766 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:41.723877 15040 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:04:41.758996  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 61304.9us
I20260709 13:04:41.633044 14755 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:41.908185 14755 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:41.859092  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:41.880402  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.911756  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.912494  8828 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:41.913609  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:41.775503 14875 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:41.876668 14589 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:41.914952 14382 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:41.932428 14875 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:41.963071 14790 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:41.937247 14977 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:41.974483  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:41.981642  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:41.938798 14777 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:04:41.951491 14803 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.838s	user 0.010s	sys 0.005s
I20260709 13:04:41.991285  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:41.997370  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:42.004127  8754 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:41.999322  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:42.006850  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:42.012843  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:42.028757  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2425) took 1011 ms. Trace:
I20260709 13:04:42.039544  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:42.040436  8674 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:42.044063  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:41.967674 14589 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:42.044508  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.053694  8755 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:42.044984  8754 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:42.032485  8830 rpcz_store.cc:276] 0709 13:04:41.017344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.029588 (+ 12244us) service_pool.cc:224] Handling call
0709 13:04:41.030068 (+   480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.067678 (+ 37610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.067701 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:41.078972 (+ 11271us) log.cc:844] Serialized 26 byte log entry
0709 13:04:41.347771 (+268799us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:41.370924 (+ 23153us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.585352 (+214428us) spinlock_profiling.cc:243] Waited 2.84 ms on lock 0x7b1800407ce0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:04:41.585577 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.585670 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.598259 (+ 12589us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b48000cd158. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:04:42.028498 (+430239us) raft_consensus.cc:1685] finished
0709 13:04:42.028523 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.028693 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15552,"prepare.queue_time_us":26235,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":242993,"raft.run_cpu_time_us":2800,"raft.run_wall_time_us":538902,"spinlock_wait_cycles":12327552,"thread_start_us":269673,"threads_started":2,"wal-append.queue_time_us":360870}
I20260709 13:04:41.973284 14382 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:41.998818 14914 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:04:42.058988  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.060289  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:41.946193 14819 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.028244 14790 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:42.063617 15049 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 13:04:42.070978  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.008100 14777 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.075814  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.012915 14803 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:42.077780  8748 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:42.083824  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.087452  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:42.078785  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.096722  8755 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
W20260709 13:04:42.100898  8794 connection.cc:256] RPC call timeout handler was delayed by -2.0000135259998615s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
W20260709 13:04:42.101573  8794 connection.cc:256] RPC call timeout handler was delayed by -1.9730135259999315s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15500000000000003s for a response
I20260709 13:04:42.075513 14934 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.105620 14382 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:42.086591 14819 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:42.111881 14589 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:42.115360 15052 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:04:42.076184 14916 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:42.116555 15051 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:04:42.116448 14916 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:42.117786  8747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 1159) took 4366 ms (client timeout 1508 ms). Trace:
W20260709 13:04:42.118356  8747 rpcz_store.cc:269] 0709 13:04:37.751120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.061686 (+310566us) service_pool.cc:224] Handling call
0709 13:04:42.117428 (+4055742us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b54002feed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:42.117657 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4870916864}
I20260709 13:04:42.119179  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.124886 14934 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:42.123543  8794 connection.cc:256] RPC call timeout handler was delayed by -1.8874030230000471s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
W20260709 13:04:42.126655  8794 connection.cc:256] RPC call timeout handler was delayed by -1.6751135259999046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
W20260709 13:04:42.126978  8794 connection.cc:256] RPC call timeout handler was delayed by -1.6688135259998944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
I20260709 13:04:42.127444 14934 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:42.127271  8794 connection.cc:256] RPC call timeout handler was delayed by -1.5356135259999064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2036s for a response
W20260709 13:04:42.129171  8794 connection.cc:256] RPC call timeout handler was delayed by -1.5041135259999692s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20710000000000003s for a response
W20260709 13:04:42.129601  8794 connection.cc:256] RPC call timeout handler was delayed by -1.43661352599986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2146s for a response
W20260709 13:04:42.130020  8794 connection.cc:256] RPC call timeout handler was delayed by -1.4051135259999228s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21810000000000002s for a response
I20260709 13:04:42.137830  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:42.138749  8674 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 10.
I20260709 13:04:42.141669  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:42.141705  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:42.145901  8674 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:42.148725  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.147462 15003 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:04:42.153605 14876 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:42.154297 14803 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:42.153677  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.155543  8674 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:42.156065 15023 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:42.156562  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.157641  8674 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:04:42.077548 14790 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.143416 14934 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:04:42.164826 15056 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:04:42.166594  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:42.167522 14777 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:42.167598  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:42.168534  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.169348  8677 raft_consensus.cc:2368] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12.
I20260709 13:04:42.170095  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.170168  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.175673 14914 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:42.176373 15053 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:42.173413  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.178432  9388 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:42.179129  9388 raft_consensus.cc:606] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:42.180516  8674 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18. Ops: [15.10-17.11]
I20260709 13:04:42.183444  8674 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:42.173748  8677 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
W20260709 13:04:42.156069 14912 consensus_meta.cc:287] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.695s	user 0.011s	sys 0.002s
I20260709 13:04:42.189774  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:42.190200  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.148545 14382 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.195288  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:42.173836 14771 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:42.182188 14934 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:42.199311  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:42.200297  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 194014us
I20260709 13:04:42.233501  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:42.235610  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.199260  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.240816  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.243547  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.192191 14916 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:42.194691 14912 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:42.247254  9353 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:42.264338  8750 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:42.259466  8677 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:42.260265  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2422) took 1244 ms. Trace:
I20260709 13:04:42.227514 15025 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.265121  8822 rpcz_store.cc:276] 0709 13:04:41.016364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.016533 (+   169us) service_pool.cc:224] Handling call
0709 13:04:41.016869 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.018384 (+  1515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.018417 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:41.023765 (+  5348us) log.cc:844] Serialized 26 byte log entry
0709 13:04:41.379137 (+355372us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b1800428fe0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:04:41.568270 (+189133us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:41.633966 (+ 65696us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.634281 (+   315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.634389 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.243080 (+608691us) raft_consensus.cc:1685] finished
0709 13:04:42.259920 (+ 16840us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.260165 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1091,"prepare.queue_time_us":59066,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":2175,"raft.queue_time_us":29518,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":20666,"spinlock_wait_cycles":2955648,"thread_start_us":188226,"threads_started":1,"wal-append.queue_time_us":695482}
I20260709 13:04:42.234787 14934 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.230348 14876 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.198791 14929 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:04:42.240504 14790 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:42.241390 15053 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:42.242915 14785 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.194537 14382 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:42.270241 14934 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:42.240689 15003 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 13:04:42.278142 14882 consensus_meta.cc:287] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.804s	user 0.014s	sys 0.000s
I20260709 13:04:42.300302  8677 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:04:42.268966  8674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:53352
I20260709 13:04:42.301888  8674 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:42.302256  8674 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:42.310089  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1261) took 1545 ms. Trace:
I20260709 13:04:42.312202  8827 rpcz_store.cc:276] 0709 13:04:40.764964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.775202 (+ 10238us) service_pool.cc:224] Handling call
0709 13:04:40.782130 (+  6928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.948418 (+166288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.948454 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.948501 (+    47us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:40.948533 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.309863 (+1361330us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54003ab6d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:42.309913 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.310024 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381872384}
W20260709 13:04:42.313246  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.278796 15023 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.276063 14876 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.295373 15053 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:42.320943 14929 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:42.289538 15015 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:42.291935 14916 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:04:42.351234 15055 log.cc:927] Time spent T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:04:42.351914 14382 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:42.352826  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2876) took 1035 ms. Trace:
I20260709 13:04:42.353441  8719 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:42.353567  8681 rpcz_store.cc:276] 0709 13:04:41.317300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.621713 (+304413us) service_pool.cc:224] Handling call
0709 13:04:41.622211 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.622783 (+   572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.622824 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:41.625178 (+  2354us) log.cc:844] Serialized 26 byte log entry
0709 13:04:42.071630 (+446452us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:42.097842 (+ 26212us) log.cc:844] Serialized 14 byte log entry
0709 13:04:42.098121 (+   279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.098237 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.352580 (+254343us) raft_consensus.cc:1685] finished
0709 13:04:42.352619 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.352759 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22101,"prepare.queue_time_us":28243,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":49,"raft.queue_time_us":1492,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":38995,"thread_start_us":414188,"threads_started":1,"wal-append.queue_time_us":595293}
I20260709 13:04:42.274958 15025 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.345911  8823 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:04:42.360910  8823 pending_rounds.cc:85] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 13
I20260709 13:04:42.361267  8823 pending_rounds.cc:107] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:04:42.361552  8823 raft_consensus.cc:2889] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:42.346506 14766 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:42.297982 14785 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.344331 14929 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:42.321419 14901 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:42.358253 15050 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:04:42.381345  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 185901us
W20260709 13:04:42.360060  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 121153us
I20260709 13:04:42.376816  9383 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:42.358924 14767 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:42.348418 15053 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:42.353287 15058 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:42.347558 14876 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:42.379824  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 190171us
I20260709 13:04:42.322911 15023 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.385243  8793 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:42.362774 14168 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:42.385560  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 138447us
W20260709 13:04:42.390306  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 142047us
I20260709 13:04:42.379604 14901 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:42.371663 14766 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:42.411185  9388 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:42.385099 14767 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:42.409287 14168 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:42.408447  8794 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:42.413805 15062 consensus_queue.cc:1059] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:04:42.415920 14916 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:42.415786 14901 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.416401 15062 raft_consensus.cc:3055] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:42.416779 15062 raft_consensus.cc:740] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:42.407706  9355 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:42.417833  8794 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:42.391485 15058 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.413327 14767 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.385074 14914 raft_consensus.cc:697] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:42.430958 15066 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:42.431562 14767 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:42.417176 14901 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:42.416420 14916 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:42.422154 15063 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:42.438418 15063 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:42.422749 15058 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:42.437381 14916 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:42.444238 15063 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:42.437722 14998 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.428509 14929 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:42.475080 15058 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:42.538411  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.317600  8678 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:42.369345  8674 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.352795  8679 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:42.359293  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:42.460198 14998 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:42.458994 15015 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:42.456272 15023 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:42.459414 15025 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:42.461194 15038 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.844s	user 0.008s	sys 0.000s
I20260709 13:04:42.555294 14914 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.574347 15058 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:42.453531 15062 consensus_queue.cc:260] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.558316 14997 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:42.576719 14785 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:42.577548 14997 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:42.559643 14999 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:42.581058 14999 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:42.584971 14999 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:42.543481  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 133920us
W20260709 13:04:42.585824  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 226466us
I20260709 13:04:42.442360 14766 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.556731 14995 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:42.591351 14995 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:42.597393 14929 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:42.597963  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.570228  8674 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:42.601357 14382 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:42.601681  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2427) took 1583 ms. Trace:
I20260709 13:04:42.602545 14382 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:42.601917  8719 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:42.602358  8826 rpcz_store.cc:276] 0709 13:04:41.018688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.226817 (+208129us) service_pool.cc:224] Handling call
0709 13:04:41.227168 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.227548 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.227572 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:41.242762 (+ 15190us) log.cc:844] Serialized 26 byte log entry
0709 13:04:41.413961 (+171199us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:41.427584 (+ 13623us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.595082 (+167498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.595174 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.460317 (+865143us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:42.586347 (+126030us) raft_consensus.cc:1685] finished
0709 13:04:42.586603 (+   256us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.601401 (+ 14798us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":153580,"prepare.queue_time_us":36045,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":77,"raft.queue_time_us":154664,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":12096,"spinlock_wait_cycles":6139136,"thread_start_us":150470,"threads_started":1,"wal-append.queue_time_us":200511}
I20260709 13:04:42.605384 15050 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:42.604909 14382 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.606662  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2933) took 1398 ms. Trace:
I20260709 13:04:42.576786 15062 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:42.607234  8751 rpcz_store.cc:276] 0709 13:04:41.207571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.208555 (+   984us) service_pool.cc:224] Handling call
0709 13:04:41.208868 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.401351 (+192483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.401403 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:41.401466 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:41.401534 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.605916 (+1204382us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54001581d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:42.606006 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.606316 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441923328}
I20260709 13:04:42.609344  8680 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:42.609519  8719 leader_election.cc:400] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12.
I20260709 13:04:42.609936  8719 leader_election.cc:403] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:42.611399 14777 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:04:42.613564  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:42.614238  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 47707.5us
I20260709 13:04:42.616185  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.617192  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.484736 15063 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:42.616547  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.610529  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.626693 14876 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:42.631132 14766 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:42.630923  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.618492  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.640851  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1267) took 1714 ms. Trace:
I20260709 13:04:42.642800  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1264) took 1854 ms. Trace:
I20260709 13:04:42.643450  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:42.645864  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.659157  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.658711  8823 rpcz_store.cc:276] 0709 13:04:40.788791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.798978 (+ 10187us) service_pool.cc:224] Handling call
0709 13:04:40.799386 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.362396 (+1563010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.642216 (+279820us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54001937d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:42.642595 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3555662592}
I20260709 13:04:42.667861  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.669116  8752 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:42.653952 14995 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:42.671723  8753 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:04:42.675835  8678 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:42.675751  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:42.658246  8821 rpcz_store.cc:276] 0709 13:04:40.926365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.926532 (+   167us) service_pool.cc:224] Handling call
0709 13:04:40.939133 (+ 12601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.036388 (+ 97255us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:41.295334 (+258946us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.590302 (+294968us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:41.590366 (+    64us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:41.590418 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.640212 (+1049794us) spinlock_profiling.cc:243] Waited 97 ms on lock 0x7b5400267ad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:42.640298 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.640616 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":189330,"prepare.queue_time_us":281164,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":190907,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":88,"spinlock_wait_cycles":223067904,"thread_start_us":363686,"threads_started":2,"wal-append.queue_time_us":155603}
W20260709 13:04:42.703996  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 286059us
I20260709 13:04:42.705705  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:42.708786 14829 raft_consensus.cc:3055] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:42.709411 14829 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:42.715835 14997 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:42.725549  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1195) took 1716 ms. Trace:
I20260709 13:04:42.726146  8676 rpcz_store.cc:276] 0709 13:04:41.008752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.016829 (+  8077us) service_pool.cc:224] Handling call
0709 13:04:41.017103 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.017300 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:41.036652 (+ 19352us) log.cc:844] Serialized 14 byte log entry
0709 13:04:42.600146 (+1563494us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b1800369520. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:04:42.676445 (+ 76299us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:42.688911 (+ 12466us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:42.724873 (+ 35962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.725032 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.725278 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1751,"prepare.queue_time_us":19349,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":2333,"raft.queue_time_us":7166,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":17493,"spinlock_wait_cycles":8000512,"thread_start_us":47299,"threads_started":1,"wal-append.queue_time_us":1633837}
W20260709 13:04:42.740173  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 60783.2us
I20260709 13:04:42.749688 14777 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 11. Current term is 12.
I20260709 13:04:42.752393 14168 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:42.760874  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:42.761276  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55859.6us
I20260709 13:04:42.783286 15022 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:42.788605  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 179473us
I20260709 13:04:42.791864  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:42.797710 15063 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:42.796674  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.762516 14829 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:42.816583  8821 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:42.820449 14936 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:42.820968  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:42.822228 14936 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:42.826936  8564 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0e9b0850145a4c66bfe983ffdfa93a57 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:42.831374  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:42.831713  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:42.834530 14382 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.835677  8793 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:42.843592 15060 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.421s	user 0.004s	sys 0.005s
I20260709 13:04:42.860656  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:42.862659  8672 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 18: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:42.843655 15057 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:04:42.872731  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:42.889756 14802 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:42.889827  8748 raft_consensus.cc:2368] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:04:42.890880 14802 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:42.891737  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.892501  8748 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:42.893656  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.893755 14802 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:42.895553 14875 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.891s	user 0.012s	sys 0.012s
I20260709 13:04:42.896121 14875 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.897535  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:42.897725  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:42.898512  8824 raft_consensus.cc:2376] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17. Re-sending same reply.
I20260709 13:04:42.899789  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:42.900064  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:42.900144  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:42.901870  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:42.902288 14875 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:42.889914  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:42.896461  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:42.898649  8828 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:42.905591  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:42.907044 15063 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:42.911162  8825 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
W20260709 13:04:42.911576 15082 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:04:42.916671 14901 consensus_queue.cc:1059] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:42.918880 14911 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:42.918494 14382 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.919365 14911 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:42.920652 14911 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.921353 14382 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:42.925765  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.927273  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:42.928037  8672 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 12. Ops: []
I20260709 13:04:42.928232  8749 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 17: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:42.929235  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:42.929291  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:42.929998  8749 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
W20260709 13:04:42.930996  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 135157us
I20260709 13:04:42.932045  8794 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:42.934100  8794 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:42.935202 14767 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:42.935752 14767 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:42.936204 14767 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:42.937933 14911 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:42.938890 14936 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:42.940171  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:42.940830 14777 raft_consensus.cc:3055] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:42.941399 14777 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:42.942924 14777 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:42.944067  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:42.944309 14777 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:42.905828  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:42.956943  8672 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:42.960263 14929 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:42.960909 14929 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.961303 14929 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:42.962867  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 85214.9us
I20260709 13:04:42.963131  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:42.963158  9386 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:42.964004  8747 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:42.964257  8825 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:42.964299  9386 raft_consensus.cc:606] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:42.978214  8747 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:42.997754  8821 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:42.999330  8672 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:42.989805  8675 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:42.938099 14916 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.006775  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:43.006994  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:43.009590  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.010650  8829 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
I20260709 13:04:43.011264  8828 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:43.015018  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.016158  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.017033  8830 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:43.018574  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:43.019901  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:43.021340  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:43.023717  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:43.023892  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:43.025741  8673 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:04:43.029167  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.030716  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:43.008504  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.047902  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:43.049002  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:43.051963  8752 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:43.003485 15062 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:43.056470  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:43.061619  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:43.064221 15063 raft_consensus.cc:2764] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:04:43.056341  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:55676 (request call id 1178) took 3282 ms. Trace:
I20260709 13:04:43.031966 14916 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:43.070453  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.066187  8675 rpcz_store.cc:276] 0709 13:04:39.763878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.767532 (+  3654us) service_pool.cc:224] Handling call
0709 13:04:41.254709 (+1487177us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b5400174b54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f0a3d 00007f32e74c0cd0 00007f32e0a45254 00007f32e0a451e2 00007f32e0a45111 00007f32e0a4508c 00007f32e0a44342 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:42.957660 (+1702951us) spinlock_profiling.cc:243] Waited 9.45 ms on lock 0x7b4c000c8600. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e07c5a20 00007f32e07d6a29 00007f32e07d713f 00007f32e079b2ab 00007f32e07c8f3c 00007f32e07c93fe 00007f32e0a5688b 00007f32e2455495 00007f32e24a630f 00007f32e24f0a55 00007f32e74c0cd0
0709 13:04:43.041116 (+ 83456us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":750726656}
I20260709 13:04:43.079293  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.082772  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.083959  8827 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:04:43.085013 14934 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.768s	user 0.009s	sys 0.013s
I20260709 13:04:43.085624 14934 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.087920  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:43.088775  8749 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:43.090065  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:43.090890  8679 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:43.092427  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:43.093282  8747 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:04:43.103188  8822 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:43.092454  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:43.094467  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:43.110317  8829 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:43.112587  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:43.110034  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 75580.2us
I20260709 13:04:43.112704  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.116031  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:43.116923  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 183027us
I20260709 13:04:43.119475  8749 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:43.124838  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:43.147051  8748 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:43.146713  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:43.124853  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:43.126550  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:43.150205  8749 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:43.150748  8752 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:43.132157  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1179) took 1142 ms. Trace:
I20260709 13:04:43.151965  8750 rpcz_store.cc:276] 0709 13:04:41.989361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.989507 (+   146us) service_pool.cc:224] Handling call
0709 13:04:41.989868 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.267506 (+277638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.126435 (+858929us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b540034b554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:43.131941 (+  5506us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":630685952}
I20260709 13:04:43.136435  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:43.156601  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:43.157529  8752 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 18: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:43.159188  8755 raft_consensus.cc:2368] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:04:43.150091  8823 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:43.158494  8747 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:43.162060  8747 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:43.164122  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:43.169566  8679 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:43.165158  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:43.176148 14970 consensus_queue.cc:1059] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:04:43.163800 14777 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:43.177518 14777 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.183291 14767 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.179826  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:43.182365 14969 raft_consensus.cc:993] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: : Instructing follower 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:04:43.184265  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:43.185251  8722 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:43.185597 14934 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:43.202474 14802 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:43.186291  8753 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:04:43.182881 14777 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:43.203289  8753 pending_rounds.cc:85] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 11
I20260709 13:04:43.203665  8753 pending_rounds.cc:107] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:04:43.203992  8753 raft_consensus.cc:2889] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:43.187130  8747 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.206830  9353 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:43.187990  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:43.209129  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:43.210276 14888 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:43.156044 14916 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:04:43.211220 14888 raft_consensus.cc:1081] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Signalling peer 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:04:43.196362  8751 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:43.220534  8749 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:43.223078  9353 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:43.225368  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 45879.9us
I20260709 13:04:43.220780 14766 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:43.221611 15062 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:43.226605  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:43.246651  8826 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:43.242076 14671 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:43.211635 14916 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:43.251588  8679 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:43.252827  8752 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:43.231492 15062 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:43.227370 15063 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:43.255497 14802 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:43.265282 14671 raft_consensus.cc:697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:43.266841  8647 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:43.270809 14771 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:43.271986 14767 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:43.273221 14802 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:43.285194 14771 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:43.286842 14771 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:43.288779 15080 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.504s	user 0.005s	sys 0.002s
W20260709 13:04:43.299144  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 72671us
I20260709 13:04:43.308564  8751 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 12. Ops: []
I20260709 13:04:43.283991 14802 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:43.315450  8752 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:04:43.316486  8752 raft_consensus.cc:606] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:43.316944  8752 raft_consensus.cc:614] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:43.318996  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1183) took 1298 ms. Trace:
I20260709 13:04:43.319280  8645 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:43.319527  8756 rpcz_store.cc:276] 0709 13:04:41.995736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.995874 (+   138us) service_pool.cc:224] Handling call
0709 13:04:41.996240 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.999959 (+  3719us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.999993 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:42.000492 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:04:42.604239 (+603747us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b1800412480. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:04:42.672923 (+ 68684us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:42.672983 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.687843 (+ 14860us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:43.287873 (+600030us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:43.294070 (+  6197us) raft_consensus.cc:1685] finished
0709 13:04:43.294104 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.294281 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3243,"raft.queue_time_us":7585,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":9473,"spinlock_wait_cycles":24682752,"thread_start_us":58358,"threads_started":1,"wal-append.queue_time_us":698198}
I20260709 13:04:43.337568  8673 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:43.337594  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:43.338662  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 125739us
I20260709 13:04:43.338581  8676 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:43.322989 14671 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:43.347028  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 162038us
I20260709 13:04:43.339612  8679 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:55676
I20260709 13:04:43.361119  8679 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:43.361689  8679 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:43.365474  8754 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:43.367843  8719 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:43.376010  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2580) took 1020 ms. Trace:
I20260709 13:04:43.381152  8754 rpcz_store.cc:276] 0709 13:04:42.350263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.193397 (+843134us) service_pool.cc:224] Handling call
0709 13:04:43.254049 (+ 60652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.366185 (+112136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.370218 (+  4033us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:43.378717 14911 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:43.373251 14936 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:04:43.415899  8794 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:43.387393 15047 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.415940  8822 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:43.385897 14911 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.351784 15062 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.398491 14829 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:04:43.398027 14901 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:04:43.422847  8825 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:43.433171  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.432888 14916 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:43.440512 14936 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:43.441099 14936 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:43.423293  8754 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
W20260709 13:04:43.429811  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 81268.3us
I20260709 13:04:43.455780  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.456043 15047 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:43.456701  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.456856  8827 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:04:43.428382 14911 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:43.461045  8747 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:43.456446  8794 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:43.466766 15047 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:43.399763 14929 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.492549  8823 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:43.502287  8754 raft_consensus.cc:606] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:43.503130  8754 raft_consensus.cc:614] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:43.504626  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:43.507659  8679 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:43.516839  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 185038us
W20260709 13:04:43.517086  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 128320us
I20260709 13:04:43.442229 14936 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:43.519416  8676 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:43.463413 14777 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.515916 14916 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:43.535246  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 412229us
I20260709 13:04:43.544440  8679 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:43.527693 14771 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:04:43.550856  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 230581us
W20260709 13:04:43.562845  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 106545us
I20260709 13:04:43.565799  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:43.535326 14916 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.523568 14540 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.159s
I20260709 13:04:43.581056 15100 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:43.582365 15100 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:43.582515 14777 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:43.573556 14936 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:43.585973 14875 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:43.589798  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2573) took 1321 ms. Trace:
I20260709 13:04:43.581102 14929 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:43.586481 14875 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.590994 14875 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:43.590325  8750 rpcz_store.cc:276] 0709 13:04:42.268501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.159414 (+890913us) service_pool.cc:224] Handling call
0709 13:04:43.189008 (+ 29594us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:43.189462 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:43.189491 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:43.190085 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 13:04:43.501988 (+311903us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:43.502044 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.502170 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:43.589493 (+ 87323us) raft_consensus.cc:1685] finished
0709 13:04:43.589520 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.589689 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3208,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":311409,"threads_started":1,"wal-append.queue_time_us":397368}
I20260709 13:04:43.595508  8677 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:04:43.601794  8747 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:43.575721 14916 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:43.611328  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:43.612686  8677 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:43.615350  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 85904.1us
W20260709 13:04:43.620471  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:43.620692 15098 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:04:43.621760 15090 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.310s	user 0.003s	sys 0.000s
W20260709 13:04:43.623812  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.627418  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:43.628556  8750 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:43.631070  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2584) took 1279 ms. Trace:
I20260709 13:04:43.634796  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2583) took 1283 ms. Trace:
I20260709 13:04:43.631695  8753 rpcz_store.cc:276] 0709 13:04:42.351375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.205182 (+853807us) service_pool.cc:224] Handling call
0709 13:04:43.205689 (+   507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:43.206232 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:43.206260 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:43.213164 (+  6904us) log.cc:844] Serialized 26 byte log entry
0709 13:04:43.353498 (+140334us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:43.353565 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.353680 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:43.630701 (+277021us) raft_consensus.cc:1685] finished
0709 13:04:43.630746 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.630975 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1163,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":165,"thread_start_us":99856,"threads_started":1,"wal-append.queue_time_us":268486}
I20260709 13:04:43.610733 14540 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 13:04:43.612828 14901 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.025s
W20260709 13:04:43.651492  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 206862us
I20260709 13:04:43.652823  8722 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:43.637447  8755 rpcz_store.cc:276] 0709 13:04:42.351113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.201480 (+850367us) service_pool.cc:224] Handling call
0709 13:04:43.202019 (+   539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.202460 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:43.346646 (+144186us) log.cc:844] Serialized 14 byte log entry
0709 13:04:43.634326 (+287680us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:43.634385 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:43.634434 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.634511 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.634715 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":267885,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"raft.queue_time_us":47593,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":114,"thread_start_us":332806,"threads_started":3,"wal-append.queue_time_us":248182}
I20260709 13:04:43.662431  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:43.665516  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:43.666373  9380 raft_consensus.cc:3055] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:43.645357 14911 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.668856 14911 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:43.666782  9380 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:43.671929  9380 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:43.679749  9380 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:43.658854 14771 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:43.680531 14771 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:43.680985 14771 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:43.681604  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 75194.3us
I20260709 13:04:43.682999  8645 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:43.684497  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:43.690383  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.686177  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.709657 15100 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:43.687844  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.689075 14382 raft_consensus.cc:3055] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:43.711293  8677 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:04:43.711549 14382 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:43.712546 14382 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.713044 15062 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:43.689312  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.661559  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:43.675495  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:43.714551  8827 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:43.714735  8680 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:43.715054 15107 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:43.691252  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.713828 14382 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:43.716326  8826 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:43.692027  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.673265 14875 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.677934 14802 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:43.709983  8828 raft_consensus.cc:2410] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:04:43.687479  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.711354  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.715531 15107 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:43.717541  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:43.733824  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.740897  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.746706  8673 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:43.749223  8829 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:43.749431 15102 raft_consensus.cc:2764] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:04:43.750428 15014 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:43.751263 15014 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:43.752421 15094 log.cc:927] Time spent T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Append to log took a long time: real 0.430s	user 0.003s	sys 0.002s
I20260709 13:04:43.754086 15014 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:43.754937  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 97874.4us
I20260709 13:04:43.755818  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:43.750476 14888 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.756752 14888 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:43.757318  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:43.758226 15109 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:43.758749 15109 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:43.759153 15109 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:43.759243 14888 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:43.760215 14465 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:43.760677 14465 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:43.761104 14465 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:43.763737  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 79366.2us
I20260709 13:04:43.771692 14465 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:43.774255 14465 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:43.775094  8826 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 16: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:43.771580 15109 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.777225 15109 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:43.741878 14540 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:43.781419 14802 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:43.781351 14540 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.735001 15107 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:43.791843 14916 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.803815  8756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:04:43.804859  8756 raft_consensus.cc:606] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:43.742406 14911 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:43.817771  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:04:43.823587  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 88702us
I20260709 13:04:43.831607  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:04:43.835024  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 173500us
I20260709 13:04:43.836813  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.839946  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:43.840957  8755 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:43.840914 14888 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:43.842044 14888 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.842450 14888 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:43.843520  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:43.844269  8747 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:43.845600 15063 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.846167 15063 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.848834 15063 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:43.850857  8822 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:43.852188  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.852548 14916 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:43.853749  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.859158  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.860028  8825 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:04:43.862046  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.863097  8825 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:43.863297  8722 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:43.865653 14940 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:43.867311 15027 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:43.867738 15027 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:43.869164  8828 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:43.868773 15027 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:43.873387 15027 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:43.874636  8825 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:43.875258 14540 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:43.875949  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.878324  8756 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:43.878947 14911 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:43.873560  8826 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:04:43.880133 14911 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:43.885466  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:43.887181  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 179799us
I20260709 13:04:43.890466  9383 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 12.
I20260709 13:04:43.892688  9380 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:43.892359 15110 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:43.898221  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.906107  8680 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:43.908231  8719 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:43.898535 15110 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:04:43.916455  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 53247.8us
I20260709 13:04:43.905100  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:43.917945  8722 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:43.921463  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 156343us
I20260709 13:04:43.920850  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:43.925001  8672 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:43.925768  9355 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:43.932614  8679 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:43.935529  8826 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:43.938177 14969 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:43.938829 14969 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.939246 14969 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:43.940076 14875 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:43.927703  8647 leader_election.cc:336] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.555s (SENT)
I20260709 13:04:43.908311 15102 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:04:43.944367  8647 leader_election.cc:336] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.562s (SENT)
I20260709 13:04:43.945132 15102 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.945863  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:43.947677  8647 leader_election.cc:336] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.587s (SENT)
I20260709 13:04:43.948452 14802 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:43.949923  8828 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:43.948210  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:43.953337  8829 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:43.921655 14382 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:43.959560  8722 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:43.922899 15110 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:43.948133 15102 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:43.965138 15104 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:43.971175 14936 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:43.971287 15104 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:43.971868  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:43.972009 15104 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:43.975281 14970 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:43.978920 15109 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:43.979751 14802 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:43.984050 14382 raft_consensus.cc:697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:43.984176 14465 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:43.985548 14382 consensus_queue.cc:237] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.988410  8722 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:43.988636 14786 consensus_queue.cc:1059] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:43.952580 15105 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:43.989630 14671 raft_consensus.cc:993] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: : Instructing follower e814ed2013244fa6be42623ea5c0c7b9 to start an election
I20260709 13:04:43.989619 14901 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:43.991524 15107 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:43.992223 14901 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:43.992720 14901 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:43.994823 14671 raft_consensus.cc:1081] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Signalling peer e814ed2013244fa6be42623ea5c0c7b9 to start an election
I20260709 13:04:43.995729  8722 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:43.997742 14936 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:43.998291 14936 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:43.998760 14936 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:44.000330  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:44.001595  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.002381  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.002486  8828 raft_consensus.cc:2368] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15.
I20260709 13:04:44.003244  8827 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:44.003978  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.997751  8823 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:55168
I20260709 13:04:44.004863  8823 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:44.005261  8823 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:44.005784 14786 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:44.003387  8824 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:44.006246 14786 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:44.002290  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.007192  8822 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:44.008221  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.015635 14786 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:44.017639 14786 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:44.019905  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 57217.3us
I20260709 13:04:44.023425  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:44.024116  8829 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:44.024828  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.026208  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.047804  8824 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:44.027056  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.028440  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.028577 15107 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:44.029258  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:44.037293 15109 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:44.038825 15104 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:44.039114  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.040791  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.041085  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.042546  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:43.907554 14771 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:44.045199  8645 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:44.058038  8830 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:44.059441  8826 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 16: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:44.054158  8823 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:44.065173  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.061430 14465 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:44.069497  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.074198 15102 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:44.079694 14465 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:44.080088 15102 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:44.080282 14465 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:44.080605 15102 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:44.081758 14936 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:44.082105 14777 consensus_queue.cc:1059] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:44.083488  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.084354 14936 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:44.085031 14936 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.085554 14936 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:44.087729 14936 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:44.089172 14969 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:44.090319  8826 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:44.092319 14671 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:44.095161  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:44.098737  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.099063 14771 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:44.099457  8827 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:44.102200  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.102955  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.103169  8830 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:04:44.104130  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.102536  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.105520  8829 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:44.117985  8828 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:44.119585  8824 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:44.119712  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.120685  9383 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
W20260709 13:04:44.145200  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 197592us
I20260709 13:04:44.162712  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.129323  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:44.121781  8645 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:44.275878  8825 raft_consensus.cc:2410] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:04:44.234350  8827 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:44.198813  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:44.277588  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.184969  8747 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:44.278398  8828 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:44.122789  8823 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:44.129570  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:44.280025  8756 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:44.280478  8829 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
W20260709 13:04:44.151391  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.281986  9355 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:44.129976  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.123494  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.233762  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:44.291558  8754 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:44.293704  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.097560 14969 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:44.292479  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 170797us
I20260709 13:04:44.168866 15104 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:44.265488 14767 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.121480 15107 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:44.062569 14888 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:44.308521 14382 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:44.316591 15111 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:44.321486  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:44.349838  8755 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:44.322518  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:44.302534 15117 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:44.323088  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:44.323454  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:44.324486  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:44.182855 15110 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:44.258384 14911 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.336423  9380 raft_consensus.cc:3055] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:44.360519  9380 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:44.199990 14901 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:44.337428  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:44.351009  8753 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:04:44.373616  9380 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:44.353335 15111 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:44.354589 15117 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:04:44.379230  8647 leader_election.cc:336] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.757s (SENT)
W20260709 13:04:44.387890  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 186933us
I20260709 13:04:44.359742 15104 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:44.390388  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:44.389374  9380 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:44.392216  8722 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:44.393138  8828 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 11.
I20260709 13:04:44.351590 14767 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:44.384732 15117 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:44.384284 14802 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:44.407588  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:44.407588  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:44.409293  8751 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:44.412604  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 75330.6us
I20260709 13:04:44.413367  8749 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:04:44.413497  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:44.414662 15048 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:04:44.418327  8794 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:44.415584  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:44.416517  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:44.417205  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:44.378093 14911 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:44.414920  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:44.420243 14802 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:44.421952  8821 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:04:44.405042 15100 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:44.404809 15105 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:44.422655 14540 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:44.417868 15126 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:04:44.428994 15111 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:44.409888 14465 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:44.429958  9380 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:44.395255 15104 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:44.436753  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 64610.8us
I20260709 13:04:44.433333  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:44.436343  8796 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:44.437500  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:44.423779 14802 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:44.444342  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:44.442382  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:44.448410  8794 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:44.452379  8647 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.812s (SENT)
I20260709 13:04:44.452888  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:44.455192  8677 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:44.451012 14802 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:44.458411  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:44.484474  8747 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:04:44.485675  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.486423  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:44.486701  8830 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:44.481834  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 68408.2us
W20260709 13:04:44.492290  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 43972.2us
I20260709 13:04:44.491612  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:44.495484  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:44.467744 14540 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:44.469861 15057 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:44.459424 15110 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:44.501163  8747 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:44.483528 15062 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:44.512513 14936 raft_consensus.cc:3055] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:44.513391 14936 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:44.520277  8794 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:04:44.524976  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 91701.1us
I20260709 13:04:44.528868  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:44.534318  8756 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:44.540731  9380 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:44.499025 15100 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:44.481469 15109 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:04:44.554258 15062 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:44.524183 15057 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:44.571533  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 88091.1us
I20260709 13:04:44.502414 14888 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:44.515498 15104 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:44.577461  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:44.582335  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 136615us
W20260709 13:04:44.595161  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 103395us
I20260709 13:04:44.599095  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:44.604087  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:44.605450  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:44.607292  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 87157.2us
W20260709 13:04:44.605693  8793 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.531s (SENT) [suppressed 33 similar messages]
I20260709 13:04:44.568006 15062 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.620879  8672 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:44.622797  8821 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:44.623636  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:44.624137  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:44.624935  8747 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:44.625679  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.626281  8821 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:44.627012  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.629715  8828 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:04:44.630113  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:44.632840  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.633637  8829 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:44.604410 14901 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:44.641225  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.642676  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.643924  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.600415 14540 raft_consensus.cc:697] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:44.651578  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:44.621869 15062 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:44.655594  8825 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:44.657243  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.546087 14936 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:44.663195  8673 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:44.671228  8821 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
W20260709 13:04:44.680883  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.681890  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.684432  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:44.647989 14911 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:44.674062 14465 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:44.680960 14767 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:44.705768  8830 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:44.710412  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:44.706727  8647 leader_election.cc:336] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.891s (SENT)
I20260709 13:04:44.807868  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:44.725418  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:44.746383  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.792292  8826 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:44.777673  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:44.818253  8754 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:44.798003 15105 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:44.764586  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:44.827682  8749 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:04:44.830978  8754 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:44.854293  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:44.865908  8825 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:44.866426  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:44.791507  8751 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
W20260709 13:04:44.865301 15128 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:04:44.891448 15135 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:44.898168  9355 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:44.871169 15102 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.745s	user 0.012s	sys 0.000s
I20260709 13:04:44.755023 15057 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:44.899114 15135 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:44.872776  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 275562us
W20260709 13:04:44.905148 15121 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.484s	user 0.003s	sys 0.001s
I20260709 13:04:44.865190 14540 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:44.909812  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 107340us
I20260709 13:04:44.945724 15133 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.946645 15133 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:44.953974  8645 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:44.950241 15133 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:44.697036 15127 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:44.966039 15138 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:44.967993 14802 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:44.954511 15135 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:44.944049 15102 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:44.991822 15135 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:44.992730 15127 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:44.992882 15102 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:44.994793 15117 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.578s	user 0.012s	sys 0.002s
I20260709 13:04:44.995374 15117 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:44.997087 15062 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:44.997933 15117 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:44.998687  8823 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:04:45.004381  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 53107.1us
I20260709 13:04:44.977065  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:44.939236 14767 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.953125 15131 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:04:44.986027 15107 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.636s	user 0.006s	sys 0.007s
I20260709 13:04:45.030172  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:45.045388  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:44.994024 15127 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:45.045086  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:45.056785  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 100630us
I20260709 13:04:45.058385  8794 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:44.999396 15133 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
W20260709 13:04:45.063127  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 86170.5us
I20260709 13:04:45.069860  8645 leader_election.cc:400] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:04:45.070288  8645 leader_election.cc:403] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:04:45.047055 15107 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:45.080660 14540 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:45.053686 15135 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.084849  8681 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:45.063024 14802 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:45.038787 14767 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:45.090939  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 384294us
I20260709 13:04:45.081719 15149 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:45.110541 15149 raft_consensus.cc:697] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:45.112068 15149 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:45.114959  8647 leader_election.cc:336] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.904s (SENT)
I20260709 13:04:45.115617  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:45.120146  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55106.6us
I20260709 13:04:45.104979 15027 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:04:45.138545  8647 leader_election.cc:336] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.132s (SENT)
I20260709 13:04:45.139223  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:45.140089 14802 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:45.142776 15117 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:45.143958 15117 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:45.144423 15117 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:45.145470 15117 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.146689  8747 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:45.147369 15117 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:45.150887 15147 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:45.166111  8747 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:45.099529 15135 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.123663 15062 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:45.207913 15131 raft_consensus.cc:697] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:45.224308 14802 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:45.200189 14540 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:45.125272 15027 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:45.227257 15131 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:45.228400 15027 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:45.228247 14540 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.230103 14767 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:45.249847  8645 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:45.203617 15102 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.047s
W20260709 13:04:45.271159 15110 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.010s	sys 0.041s
W20260709 13:04:45.284595 15110 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.011s	sys 0.042s
W20260709 13:04:45.279107  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 220841us
I20260709 13:04:45.248901 14540 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:45.283799 15148 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Leader election lost for term 11. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12.
I20260709 13:04:45.283784 15138 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:04:45.296646 15138 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:45.297158 15138 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:45.300041 15135 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:45.300817 15135 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:45.300711 15027 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:45.304725 15149 raft_consensus.cc:3055] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:45.305282 15149 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:45.306264 15149 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.277211 15147 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:45.311000 15147 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:45.308123 15149 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:45.311623 15147 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:45.316619 15100 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:45.323235  8794 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:45.334683  8645 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:45.339772 15127 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:45.302263 15135 consensus_queue.cc:1059] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 13:04:45.340847  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 43642.4us
I20260709 13:04:45.342165  8794 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:45.345167  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:45.345167 15162 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:45.347451  8673 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:45.347820 15135 raft_consensus.cc:3055] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:45.348366 15135 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:45.349493 15135 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.298184 15138 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.351734 15138 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:45.331753 15100 consensus_queue.cc:1243] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:45.356779 15027 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:45.357077 15135 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:45.366869 15162 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:45.368331 15162 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.369072 15127 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:45.371167 15117 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:45.374490 15100 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:45.340749 14540 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:45.378086  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:45.379305 15117 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:04:45.380453 15158 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:04:45.384495  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:45.391654 15117 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:45.370460 15127 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.404116  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:45.414201 15149 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:45.417543  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.346513  8794 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:45.428982 14382 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:45.429247  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 82759.7us
I20260709 13:04:45.426586  8647 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:45.459908  8830 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:45.458477  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:45.465360  8794 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:45.465803  8828 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:45.459445  8822 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:45.439110  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:45.468163  8826 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:45.439568  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:45.480309  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 46019.8us
I20260709 13:04:45.430557 15130 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:45.495803  8672 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:45.437022 14465 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.433286 14936 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:45.498996  8794 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:45.430382 14540 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:45.429719 15107 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.513289  8821 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:45.470948 15062 raft_consensus.cc:2764] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:04:45.429001 15143 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:04:45.518847  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 451030us
I20260709 13:04:45.523190 15143 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.501685 14802 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:45.524834 15133 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:45.525839 15143 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:45.525940 15107 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:45.527064 14802 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:45.528215 14802 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.528585 15107 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:45.529803  8825 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:45.528501  8823 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:45.521517 15014 consensus_queue.cc:1059] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:04:45.543289  8828 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
W20260709 13:04:45.545950  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 211402us
I20260709 13:04:45.480885 14382 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:45.553472  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 129066us
I20260709 13:04:45.517835 14540 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.556998  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:45.578734  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 140766us
I20260709 13:04:45.581286  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:45.533454 15147 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.503778 14771 consensus_queue.cc:1059] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:04:45.551091 15107 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.563851 14540 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:45.583614  8722 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:45.512543 14465 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.515486 15130 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:45.607275  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:45.618438  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:45.619390  8676 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:45.624747  8672 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:45.625023  8681 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:45.626206  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:45.626940  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:45.626538  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:45.628417  8672 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:45.632563  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:45.633869  8681 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
W20260709 13:04:45.635179  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 66186.5us
I20260709 13:04:45.637362  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:45.639511  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:45.639371  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:45.646557  8675 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:45.618204 15127 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:45.651532 15127 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.654265 15127 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:45.598248 15168 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:45.616385 15133 raft_consensus.cc:697] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:04:45.664604  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:45.661775  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 144857us
I20260709 13:04:45.662497  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:45.667521  8681 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:45.630180 14771 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:45.654997 15171 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:45.599212 15107 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.678027 15171 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:45.613085 15105 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:04:45.639602 15149 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:45.679952 15107 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:45.682834 14465 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:45.670488 14767 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.686666 14767 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.688920 14767 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:45.689282  8750 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:04:45.695782  8680 raft_consensus.cc:2410] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:04:45.671279  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:45.630967 15110 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.665951  8719 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:45.712356  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 46560us
I20260709 13:04:45.712466 14465 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:45.713203 14465 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.676713 15147 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:45.713717 14465 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:45.677652 15162 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.715334 15162 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.717839 15162 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:45.718839  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:45.719842  8821 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 16: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:04:45.723158  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 158296us
I20260709 13:04:45.723441  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:45.725128 15107 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:45.726154  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:45.670435 14771 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:45.728147  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:45.753798  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:45.691498 15133 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.753813  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:45.755489  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:45.757596  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 120417us
I20260709 13:04:45.764030  8750 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:45.764580  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:45.754643  8674 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 10.
I20260709 13:04:45.706555  8826 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:45.773628  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:45.774948  8749 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 10.
I20260709 13:04:45.803471  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:45.803247  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:45.798130 15105 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.752095 15110 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:45.755537 15148 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.823359  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:45.812988 14771 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.751781 14767 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:45.845770  8750 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
W20260709 13:04:45.846267  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 81828.5us
W20260709 13:04:45.867142  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 259984us
I20260709 13:04:45.867456  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:45.868029  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:45.847694  8753 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:45.760627 15162 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:45.863700  8674 raft_consensus.cc:2376] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13. Re-sending same reply.
W20260709 13:04:45.872627  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 147816us
I20260709 13:04:45.877928  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:45.869750  8676 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:45.867765  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:45.848407 15107 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.851334 15147 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:45.854894 15175 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.882424  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:45.932380  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:45.929314  8677 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
W20260709 13:04:45.938282  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 57262.5us
I20260709 13:04:45.941385  8681 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:45.943089  8753 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:45.864728 15105 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:45.929275 15149 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:45.955869  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:45.930254 14936 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:45.959131 15178 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:45.959494 15176 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:04:45.968116 15174 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:45.989392 15178 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:45.989861 15174 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:45.991139 15155 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:45.992413 15174 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:45.989435  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 122069us
I20260709 13:04:45.925043 15107 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:45.968847  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:45.993995  8753 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:45.969985  8674 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:04:45.970435 15110 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:45.968684  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:45.873391 15148 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:45.995545 15107 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:45.996083  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:45.996802  8752 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:45.996842 15148 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:45.947260 14771 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:45.878918 15162 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:46.000057 15162 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:46.001859  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:46.003311  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:45.975574 15188 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.004519 15188 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:46.005198 15130 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:45.971340  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:46.011057  8678 raft_consensus.cc:2376] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14. Re-sending same reply.
I20260709 13:04:45.992349 15178 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:45.976689 14786 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:46.012977  8750 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:45.976456 15105 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:45.979696  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:45.983793 15181 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:45.984968 14540 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:46.015638  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:46.015954 15181 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:46.016533  8748 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:46.017589 14540 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:45.953694 15172 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.018522 15172 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:46.019376  8751 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:46.021351 15172 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:46.022270  8754 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:45.986310 14465 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:46.024884 14465 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:45.988013  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:46.018451 15181 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:45.992522 15155 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:04:46.025730  8747 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:04:46.025888  8676 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:45.993291 15174 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:46.026539 15155 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:46.026805 15174 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:46.027033 15155 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:46.027853 15174 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:46.028185 15155 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:46.029322 15174 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:46.029538 15155 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:45.993904 15154 raft_consensus.cc:3055] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:46.030157 15154 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:45.977911  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:46.018004  8748 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:45.974416 15189 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:46.033705  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:46.003932  8753 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:46.036758  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.038569  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:46.068264  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:45.944348 15175 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:46.041909  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:46.015664 15183 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:04:46.070097  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:46.067766  8719 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:46.089077  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:46.089764  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 221799us
I20260709 13:04:46.088203  8647 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:46.090082  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:46.047622  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:46.091369  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:46.025027 15188 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:46.100483  8678 raft_consensus.cc:2410] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:04:46.099277  8674 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:46.101634  8722 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:46.103235  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:46.104280  8679 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 22: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:46.105296  8672 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:46.105399  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:46.106671  9380 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 14: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:46.065779 15181 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.114773  8722 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:46.117938  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 47959.4us
I20260709 13:04:46.070514 14382 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:46.088912 15148 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:46.093266 15062 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:46.086792 14540 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:46.122809  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:46.125150  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:46.148492  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:46.134402  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:46.134821  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:46.135711  8793 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:46.134402  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:46.137147  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:46.137032  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:46.139991  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:46.112574 15172 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:46.144615  8830 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: [13.8-16.9]
I20260709 13:04:46.145426  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:46.147127  8827 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:04:46.195317  8794 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:46.196274  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:46.196393  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:46.197268  8672 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:04:46.120678 14936 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:46.150067  8822 raft_consensus.cc:2376] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16. Re-sending same reply.
I20260709 13:04:46.204568  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:46.154356  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:46.151034 14767 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:46.153461  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:46.199563  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 319553us
W20260709 13:04:46.202579  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68282.3us
I20260709 13:04:46.137912 15127 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:46.211767  8678 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:46.215443  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:46.292092  8756 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:46.212596 15191 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 13:04:46.297188  8747 raft_consensus.cc:2410] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:04:46.297365 15138 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:46.297771  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:46.298586  8756 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 14: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:46.220818  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:46.293231  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 88675us
I20260709 13:04:46.153107 15148 raft_consensus.cc:697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:46.210922 15143 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:46.304594  8748 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:46.148478 14382 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:46.218357  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:46.031297 15154 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:46.132232 15181 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:46.212319 14540 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:46.329010  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:46.342132  8678 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:46.327926  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:46.323762  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 188133us
I20260709 13:04:46.309752  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:46.310937  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:46.344445  8822 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:46.344949  8756 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:46.328580 15148 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:46.346900 15188 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.347440 15188 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:46.348466 14465 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.349031 14465 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:46.349905 15188 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:46.351066  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:46.346036 15154 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:46.355082  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:46.354364  8751 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:46.345175  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:46.362975  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 144776us
I20260709 13:04:46.290647 15127 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:46.289994 15105 raft_consensus.cc:993] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: : Instructing follower 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:04:46.370365  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:46.388286  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 438609us
I20260709 13:04:46.802678  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:46.458629  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:46.291306 14767 raft_consensus.cc:697] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:46.515374  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:46.519037 15194 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:04:46.331418 15138 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:46.736485 15041 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.219679  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:46.516723 15182 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.208s
I20260709 13:04:46.790251 15143 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:46.346565  8756 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:04:46.792636  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 399193us
I20260709 13:04:46.846153  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:46.504230 15132 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.363423 14936 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.189s
W20260709 13:04:46.795868 15149 consensus_meta.cc:287] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.819s	user 0.018s	sys 0.000s
I20260709 13:04:46.795957  8645 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:46.796520 15168 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:46.504609 15150 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:46.289149 15196 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:46.798305 15135 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.798489 15181 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:46.340020 15147 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.126s
I20260709 13:04:46.806694  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2596) took 1051 ms. Trace:
I20260709 13:04:46.891788  8754 rpcz_store.cc:276] 0709 13:04:45.755746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.805446 (+ 49700us) service_pool.cc:224] Handling call
0709 13:04:45.979512 (+174066us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b5800083198. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:04:46.381709 (+402197us) spinlock_profiling.cc:243] Waited 6.57 ms on lock 0x7b540034ab54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:46.806597 (+424888us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19544704}
W20260709 13:04:46.796554  8753 consensus_meta.cc:287] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.706s	user 0.016s	sys 0.000s
I20260709 13:04:46.892987  8753 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:04:46.893632  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 2988) took 1053 ms. Trace:
I20260709 13:04:46.893998  8753 rpcz_store.cc:276] 0709 13:04:45.839753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.003053 (+163300us) service_pool.cc:224] Handling call
0709 13:04:46.893560 (+890507us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:46.817567  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.821784  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:46.836063  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:46.843068  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:46.828811 14771 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.831s	user 0.021s	sys 0.006s
I20260709 13:04:46.822589 15133 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.905241  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:46.906067  8674 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:46.817193 15191 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:46.831791 15207 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:46.887217  8722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.029s	sys 0.024s
I20260709 13:04:46.845449 15143 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.899080  8749 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19. Ops: [16.10-18.11]
I20260709 13:04:46.900276  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 2987) took 1061 ms. Trace:
I20260709 13:04:46.870728 15196 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.908531 15157 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:46.891269 15135 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:46.870432 15149 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:46.909227 15171 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.883731 15150 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:46.876698 15168 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:46.876123 14465 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:46.912289 15154 consensus_meta.cc:287] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.547s	user 0.010s	sys 0.003s
I20260709 13:04:46.843978 15041 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:46.812500  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:46.961045  8722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.031s	sys 0.030s
I20260709 13:04:46.961294  8755 rpcz_store.cc:276] 0709 13:04:45.839393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.839560 (+   167us) service_pool.cc:224] Handling call
0709 13:04:45.839965 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:45.840392 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:45.840419 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:45.840945 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 13:04:46.104006 (+263061us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:46.104056 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.104129 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:46.899988 (+795859us) raft_consensus.cc:1685] finished
0709 13:04:46.900012 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:46.900190 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5429,"raft.queue_time_us":233,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":23367,"thread_start_us":175019,"threads_started":1,"wal-append.queue_time_us":317238}
I20260709 13:04:46.971901 15196 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:46.920687 14540 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:46.935912  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:46.908603 15212 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:46.862015 15132 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:46.931068 15191 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:46.967082 15143 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:46.995246  8823 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:46.930995 15102 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:47.001396  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 691696us
W20260709 13:04:46.934346  8647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.003s	sys 0.020s
I20260709 13:04:47.002549  8676 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:46.978593 15127 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:46.920101 15155 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.890s	user 0.018s	sys 0.000s
I20260709 13:04:46.920614 14767 consensus_queue.cc:237] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:46.931442 15207 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.986788 15157 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:47.180112 15132 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:47.046757 15175 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:47.167819  8722 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:47.176121 15154 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.987468 15182 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.187372 15154 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.187758 15138 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.187695 15182 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.188618 15138 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:47.189201 15138 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.189935 15154 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:47.047816  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 251945us
I20260709 13:04:47.190974 15182 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:46.988543 15209 raft_consensus.cc:1081] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Signalling peer 0e8c59a1f0394eae8ce33b99f10efadc to start an election
W20260709 13:04:46.917505  8706 heartbeater.cc:646] Failed to heartbeat to 127.7.89.190:45099 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.7.89.190:45099 timed out after 15.000s (SENT)
W20260709 13:04:47.138872  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 308882us
I20260709 13:04:47.068105  8719 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:47.193185 15191 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:46.998756 14465 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.193984 14465 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:47.065333  8677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2905) took 1436 ms (client timeout 1670 ms). Trace:
W20260709 13:04:47.167053  8647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.003s	sys 0.041s
W20260709 13:04:47.197009  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 896716us
I20260709 13:04:47.179587 15157 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:47.196604  8677 rpcz_store.cc:269] 0709 13:04:45.624774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.625864 (+  1090us) service_pool.cc:224] Handling call
0709 13:04:47.058278 (+1432414us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b5400175a54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:47.060947 (+  2669us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307597568}
I20260709 13:04:46.986123 15171 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.175558 15102 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.180969 15155 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.191378 15207 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:47.173170  8828 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:47.208940 15209 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:47.138561 15192 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:46.919157 15133 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.211081 15208 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:47.230520 15208 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.230970 15208 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:47.205755  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 137805us
I20260709 13:04:47.213617  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:47.235034 15219 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:47.236406 15219 raft_consensus.cc:3055] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:47.236589 15182 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.237005 15219 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:47.237210 15182 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.215906 15041 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:47.215950 15102 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.236083  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1276) took 1094 ms. Trace:
I20260709 13:04:47.239821  8675 rpcz_store.cc:276] 0709 13:04:46.141434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.235151 (+1093717us) service_pool.cc:224] Handling call
0709 13:04:47.235537 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.235750 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.235776 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.235824 (+    48us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:47.235858 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.235918 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.236021 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.240867 15171 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.241370  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:47.241654 15171 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:47.242101  8675 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:47.071990 14771 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:47.242925 14771 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.243777 15182 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:47.244509 15175 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:47.245296 14771 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.245090 15175 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.247339 15175 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:47.248266  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:47.248684 15175 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.207691  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:47.249990 15172 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:47.250542 15172 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:47.250871 15155 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.251547  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:47.253177 15155 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:47.254112  8830 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:04:47.206820  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.151616 15213 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.205307  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:47.256814  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 51650.5us
I20260709 13:04:47.040412 14540 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:47.137725 15135 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:47.196255 15132 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:04:47.196269 14465 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:47.249782 15206 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:47.122586 15174 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.090s	user 0.013s	sys 0.007s
W20260709 13:04:47.138713 15148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.021s	sys 0.043s
I20260709 13:04:47.259667  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.218849  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2510) took 1085 ms. Trace:
I20260709 13:04:47.261685  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2605) took 1457 ms. Trace:
I20260709 13:04:47.221155  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1274) took 1080 ms. Trace:
I20260709 13:04:47.301654  8677 rpcz_store.cc:276] 0709 13:04:46.140769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.219954 (+1079185us) service_pool.cc:224] Handling call
0709 13:04:47.220430 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.220683 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.220721 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.220782 (+    61us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:47.220830 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.220910 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.221073 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.304272  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1277) took 1162 ms. Trace:
I20260709 13:04:47.304791  8677 rpcz_store.cc:276] 0709 13:04:46.141750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.303050 (+1161300us) service_pool.cc:224] Handling call
0709 13:04:47.303535 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.303787 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.303827 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.303889 (+    62us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:47.303937 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.304010 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.304188 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.227218  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2608) took 1112 ms. Trace:
I20260709 13:04:47.131286 15205 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:47.203392 15212 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:47.327271  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.327883 15212 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.242285 15171 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.329284  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.287210  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:47.330974  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.332229  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:47.332926  8756 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:04:47.334055 15171 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.282485 15102 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:47.340480  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.340806  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.341509  9381 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:47.341652  9382 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:04:47.286688  8564 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0e9b0850145a4c66bfe983ffdfa93a57 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:47.206429 15168 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:47.223261 15202 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
W20260709 13:04:47.342576  8564 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 58448.5us
I20260709 13:04:47.327316 15196 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.308749  8677 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:47.238221 15219 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 6, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.347178  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2928) took 1145 ms. Trace:
I20260709 13:04:47.229347 15133 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:47.347738 15219 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:47.347548  8677 rpcz_store.cc:276] 0709 13:04:46.202194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.306059 (+1103865us) service_pool.cc:224] Handling call
0709 13:04:47.306515 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.346890 (+ 40375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.347108 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.349399  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:47.350095  8677 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:47.350536 15195 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.351132  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:47.352208 15217 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:47.320437  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.352825 15217 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.353560 14465 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:47.354266 14465 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.286221 15175 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.357910 14465 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.358669 14465 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:47.359400 14465 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.360036 15218 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:47.361611  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:47.362466  8755 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:04:47.364815 15138 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.367669 15182 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:47.367924  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:47.368654  8750 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:47.368484 15182 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.283677 15150 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.282490 15107 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:47.371377 15107 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:47.323798  9385 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:47.324496  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.372493 15107 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.299623  8828 rpcz_store.cc:276] 0709 13:04:46.133889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.134075 (+   186us) service_pool.cc:224] Handling call
0709 13:04:47.218521 (+1084446us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003de5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:47.218713 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2353969920}
I20260709 13:04:47.219539 15209 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:47.374773  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:47.375563 15209 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.376679  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:47.377064  8747 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:47.377573  8828 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:04:47.307070  8748 rpcz_store.cc:276] 0709 13:04:46.114596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.194719 (+ 80123us) service_pool.cc:224] Handling call
0709 13:04:46.215148 (+ 20429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:46.305258 (+ 90110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.226895 (+921637us) spinlock_profiling.cc:243] Waited 89.1 ms on lock 0x7b540038b3d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:47.227145 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205038336}
I20260709 13:04:47.378389  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.300521  8749 rpcz_store.cc:276] 0709 13:04:45.804344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.806366 (+  2022us) service_pool.cc:224] Handling call
0709 13:04:46.454699 (+648333us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b5800083198. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:04:46.893570 (+438871us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:46.961245 (+ 67675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.261404 (+300159us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b540038aed0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:47.261601 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1013234688}
I20260709 13:04:47.326261 15205 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.293375  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:47.380067  8675 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:47.380636  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:47.216593 15203 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:04:47.287599 14540 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.382373  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1231) took 1540 ms. Trace:
I20260709 13:04:47.298341 14936 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:47.382336 15205 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.384222 14936 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.385306 14540 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:47.299044 15148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.021s	sys 0.044s
I20260709 13:04:47.388684 15102 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:47.389824 15102 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.394948 14936 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.396363  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.344331  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2623) took 1306 ms. Trace:
I20260709 13:04:47.344730 15211 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:47.399559  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:47.400324  8680 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:47.400964  8677 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:47.403409  8753 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:04:47.345247 15207 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.345621  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:47.414800  8754 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:04:47.255216 15213 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.348397 15196 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:47.373303  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:47.296025 15135 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:47.379181  9387 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:47.330579 15206 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.404999  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:47.398511  8681 rpcz_store.cc:276] 0709 13:04:46.037896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.038870 (+   974us) service_pool.cc:224] Handling call
0709 13:04:46.039377 (+   507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:46.067352 (+ 27975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:46.067383 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:46.088078 (+ 20695us) log.cc:844] Serialized 26 byte log entry
0709 13:04:46.364881 (+276803us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:46.429594 (+ 64713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.429743 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:47.344078 (+914335us) raft_consensus.cc:1685] finished
0709 13:04:47.344102 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.344262 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27433,"raft.queue_time_us":28068,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":18184,"thread_start_us":242065,"threads_started":1,"wal-append.queue_time_us":872484}
I20260709 13:04:47.383891  8752 rpcz_store.cc:276] 0709 13:04:45.841466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.009326 (+167860us) service_pool.cc:224] Handling call
0709 13:04:46.009812 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:46.010279 (+   467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:46.010323 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:46.086252 (+ 75929us) log.cc:844] Serialized 26 byte log entry
0709 13:04:46.307684 (+221432us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:46.309145 (+  1461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.309236 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:47.382113 (+1072877us) raft_consensus.cc:1685] finished
0709 13:04:47.382140 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.382303 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5045,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":103,"thread_start_us":184923,"threads_started":1,"wal-append.queue_time_us":888037}
I20260709 13:04:47.419611  8677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:53352
I20260709 13:04:47.421264  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:47.366196 15138 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:04:47.443522  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.362425 14465 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.428570  9353 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:47.449333  8827 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 11.
I20260709 13:04:47.454456  9353 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:47.434785  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.358397 15175 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.455483  9384 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:04:47.455892  8646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.011s	sys 0.039s
I20260709 13:04:47.425873  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:47.412489 15211 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.457052  8680 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:47.457633  9353 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:47.432018  8821 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
W20260709 13:04:47.456336  8646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.011s	sys 0.039s
W20260709 13:04:47.458992  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.463737  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:47.485883  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:47.435482 15218 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:47.487423 15218 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.489858 15218 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:47.490706  8677 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:47.491178  8677 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:47.442559 15154 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:47.493566 15154 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.493808 15207 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:47.434136 15143 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.497658 15143 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:47.451882 15147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.031s	sys 0.019s
W20260709 13:04:47.475414  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.498869 15217 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.499891  8826 raft_consensus.cc:2368] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18.
I20260709 13:04:47.459628 14829 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:47.484010  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:47.491044 15188 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:47.494046 15154 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:47.447934 14936 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.474206  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:47.501036  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:47.501892  8751 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:47.500952  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:47.481436 15172 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.503645  9386 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:47.506413  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:47.462996 15189 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:47.518877  8756 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:47.460137 15211 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:47.502236  8677 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.351162 15195 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.535845  8794 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:47.533365 15133 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:47.538476 15175 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:47.538632  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:47.539553 15175 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:47.539678  8751 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:47.540627 15175 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.541570  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:47.538465 15207 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:47.542886 15172 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:47.543527 15207 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.549898 15207 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.550917  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:47.550837 15205 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:47.553256 15206 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:47.520943 15189 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.553435  8749 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:47.498384 15143 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:04:47.553957 15189 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:47.554733 14540 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:04:47.515198 15182 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.515705 15188 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.557991 15188 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:47.561546 15208 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.563807  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:47.564654  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:47.523973  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:47.517078  8755 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:04:47.529911  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:47.568635  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.023s	sys 0.010s
W20260709 13:04:47.569300  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.024s	sys 0.010s
I20260709 13:04:47.572624  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:47.573565  8680 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:04:47.576120  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:47.512220 14829 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.582338  8756 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:47.599105  8752 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:47.582749 15110 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.602715 15110 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.533131 14936 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:47.572052 15210 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.398s	user 0.005s	sys 0.002s
I20260709 13:04:47.605476 14936 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:47.583050 14829 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:47.607836 15188 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.609541 15208 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:47.611815  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:47.618369  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:47.624212 15195 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.625301  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2508) took 1501 ms. Trace:
I20260709 13:04:47.669636  8821 rpcz_store.cc:276] 0709 13:04:46.124154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.124333 (+   179us) service_pool.cc:224] Handling call
0709 13:04:47.625089 (+1500756us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540016a654. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:47.625217 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2499821952}
I20260709 13:04:47.627748  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2511) took 1488 ms. Trace:
I20260709 13:04:47.635524  8794 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:47.670832  8823 rpcz_store.cc:276] 0709 13:04:46.139461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.139649 (+   188us) service_pool.cc:224] Handling call
0709 13:04:47.627500 (+1487851us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b54003ab6d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:47.627664 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1802360704}
I20260709 13:04:47.641526  8751 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:04:47.647771  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:47.672540  8674 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:04:47.614516 15223 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.672024  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:47.674033  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:47.678436  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:47.679950  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:47.681042 15155 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:47.681605 15155 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.682088 15155 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:47.674882  8822 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:47.683488  9386 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:04:47.682600  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:47.686280  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:47.685016  8823 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:47.685572  8821 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:47.687157  8674 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:47.705996  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.706764  9384 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:47.707098 15188 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:47.701086 14767 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:47.708074 14767 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.708355  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1330) took 1490 ms. Trace:
I20260709 13:04:47.708434 14767 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:47.708724  8796 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:47.710780 15147 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:47.711458 15147 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.713907 15147 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:47.717074  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:47.717886  8674 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:47.668771 15213 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.715124  8824 rpcz_store.cc:276] 0709 13:04:46.217750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.371735 (+153985us) service_pool.cc:224] Handling call
0709 13:04:47.628495 (+1256760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.628800 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.628837 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.628898 (+    61us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:47.628945 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.629024 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.708163 (+ 79139us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54003ab6d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:47.708288 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2889158400}
I20260709 13:04:47.682866 15196 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:47.749554  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:47.741900  8680 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:47.752206  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:47.755311  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2516) took 1605 ms. Trace:
I20260709 13:04:47.729563  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:47.740935  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 531537us
I20260709 13:04:47.643386 15143 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:04:47.715790 14802 raft_consensus.cc:476] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Not starting pre-election -- already a leader
I20260709 13:04:47.714709 15218 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.760303  8823 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:47.755887  8829 rpcz_store.cc:276] 0709 13:04:46.149954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.150882 (+   928us) service_pool.cc:224] Handling call
0709 13:04:46.202371 (+ 51489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:46.289429 (+ 87058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:46.289488 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:46.290179 (+   691us) log.cc:844] Serialized 26 byte log entry
0709 13:04:46.679535 (+389356us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:46.679607 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.679724 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:47.556333 (+876609us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:47.748358 (+192025us) raft_consensus.cc:1685] finished
0709 13:04:47.748396 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.755212 (+  6816us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":76104,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":158,"spinlock_wait_cycles":30077184,"thread_start_us":388742,"threads_started":1,"wal-append.queue_time_us":797314}
I20260709 13:04:47.763242  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:47.763764  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:47.768769  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 311186us
I20260709 13:04:47.770638  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:47.743432 15205 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.719403 15110 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.746661 15150 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:47.701938 15206 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:04:47.784917  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.826150  9385 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:47.827205  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.849444  8722 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:47.848392  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:47.835042  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 64554.9us
W20260709 13:04:47.831734  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68555.4us
I20260709 13:04:47.704586 15131 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:04:47.855553  8722 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:47.742540 15154 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.701773 15223 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:47.880620  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 171972us
I20260709 13:04:47.782948 15196 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:47.772872 15192 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:47.892822 15192 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:47.856038  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:47.876811  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:47.905582  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 49621.1us
W20260709 13:04:47.907275  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 51427.9us
I20260709 13:04:47.912483 15192 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:47.914511  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:47.919953  8751 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:47.829282 15133 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:47.923839  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 160136us
I20260709 13:04:47.847672 15174 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:47.926025  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 70567.4us
I20260709 13:04:47.856663 15216 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:47.932364  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:47.929257  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.877465 15131 consensus_queue.cc:1243] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:47.776080 15218 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.779763 14802 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.143s
I20260709 13:04:47.862785 15150 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:47.884608 14936 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:47.917930 15222 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:47.918440 15196 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.786510 15205 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:47.938129 15216 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.941262  8722 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:47.948400  8751 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:47.920495 15228 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:47.949940  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:48.005244 15228 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:47.956838  8755 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:48.006345 15228 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.007195 15154 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:47.944628  8645 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:48.008195 15150 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:48.008808  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 64279.6us
I20260709 13:04:47.939539 15107 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:47.968597 14802 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:47.970330  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:47.963877 15196 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:47.973642 15229 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:47.975797 15133 raft_consensus.cc:3055] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:47.977488 15205 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:48.049093  8674 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:48.049866  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:47.987756  8676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 2927) took 1836 ms (client timeout 1965 ms). Trace:
I20260709 13:04:47.991087  8719 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:48.052202  8676 rpcz_store.cc:269] 0709 13:04:46.151476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.220462 (+ 68986us) service_pool.cc:224] Handling call
0709 13:04:47.987528 (+1767066us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b54002567d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:47.987680 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1616188160}
I20260709 13:04:47.997176 14829 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:48.004526 15227 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 13:04:48.010768  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 52586.9us
W20260709 13:04:48.011134  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 69950us
I20260709 13:04:48.038334  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:48.046810  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 140339us
I20260709 13:04:47.954506 14936 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:48.052546  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 61605.2us
I20260709 13:04:47.995076  8824 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:47.957739 15222 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:48.010900 15216 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:48.063802  8719 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:48.035212 14802 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.064746  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:48.032279 15131 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:48.069146  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:48.067476  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1273) took 1927 ms. Trace:
I20260709 13:04:48.035951 15208 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:48.051645 15181 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.078531  8719 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:48.075136  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:48.049546 15229 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.049332 15133 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:48.078871  8828 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:48.054888 15110 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.066025 15195 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.069927 15222 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:47.959738 15189 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.066556 15205 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.063680 14382 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:48.088577  8719 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:48.086551  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 55184.3us
I20260709 13:04:48.079844  8672 rpcz_store.cc:276] 0709 13:04:46.140420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.101445 (+961025us) service_pool.cc:224] Handling call
0709 13:04:47.145011 (+ 43566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.145348 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:47.664783 (+519435us) log.cc:844] Serialized 14 byte log entry
0709 13:04:48.067004 (+402221us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:48.067066 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:48.067122 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.067211 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.067382 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":616340,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":46867,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"thread_start_us":869831,"threads_started":3,"wal-append.queue_time_us":396121}
I20260709 13:04:48.066810 14936 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:48.092971  9353 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:04:48.102427 15175 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.001s	sys 0.019s
I20260709 13:04:48.090260 15181 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.076872 14802 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:48.102077  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:48.082302 15212 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.008s	sys 0.016s
I20260709 13:04:48.102849  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:48.103837  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1279) took 1242 ms. Trace:
I20260709 13:04:48.110219  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:48.133270  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:48.089495 15229 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:48.081398 15208 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.076184 15209 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:48.077946 15192 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:48.136649  8719 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:48.138865  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 67661.3us
I20260709 13:04:48.142190  8828 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:48.140722  8675 rpcz_store.cc:276] 0709 13:04:46.861898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.380809 (+518911us) service_pool.cc:224] Handling call
0709 13:04:47.404725 (+ 23916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.404960 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.404988 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.405035 (+    47us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:47.405075 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.405130 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.103640 (+698510us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b54003cf850. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:48.103769 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54192000}
I20260709 13:04:48.144806  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:48.106724 14382 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.149523 15231 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:48.150074 15253 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
W20260709 13:04:48.151463 15243 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:04:48.108566 15107 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:48.081845 15131 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:48.144052 15175 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.001s	sys 0.019s
I20260709 13:04:48.138334 15217 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:48.175623 15217 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.175927 15131 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:48.176102 15217 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:48.139747 15212 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.009s	sys 0.017s
I20260709 13:04:48.166723  8647 leader_election.cc:400] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18.
I20260709 13:04:48.083647 15246 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:04:48.177093  8647 leader_election.cc:403] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:48.168372  9388 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:48.170192  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:48.179162  8828 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:48.100773 15195 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.139926 14786 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:48.109025 15205 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.162663 15133 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.141515 15157 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:04:48.102646 15110 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:48.228637  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 117273us
W20260709 13:04:48.185829 15249 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:04:48.198860  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:48.149555 15247 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:48.205215  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:48.169627 14382 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:48.173427 15231 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.229936  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:48.148027 15208 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:48.236490 14829 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:48.264269  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:48.265273  8822 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:48.266922 15218 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:48.269222 15172 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:48.269814 15172 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:48.270289 15172 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:48.272390 15205 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:48.238338 14786 raft_consensus.cc:697] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:48.195899 15189 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:48.244632  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 206396us
I20260709 13:04:48.178650 15107 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.195762 15135 raft_consensus.cc:3055] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:04:48.218746 15211 consensus_meta.cc:287] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.667s	user 0.009s	sys 0.003s
I20260709 13:04:48.255029 15260 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:48.330729  8793 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:48.256608 15265 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:48.258473  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:48.259543 14771 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:48.275334  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:48.278784  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:48.337241  8828 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:48.208046 15223 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:48.163290 15209 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.219060 15062 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.318920 15271 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18.
W20260709 13:04:48.322952  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 160644us
I20260709 13:04:48.350831  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:48.324087  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:48.324527  8675 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:48.334566  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:48.335585  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 105781us
I20260709 13:04:48.344806  8674 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:04:48.355048  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.356266  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.357005  8674 raft_consensus.cc:2393] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:48.347755  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.358191  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.358850  8680 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:48.359898  8680 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:48.180199 15254 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.158s
I20260709 13:04:48.276154 15228 raft_consensus.cc:2764] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:04:48.364243  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:48.316147 15107 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:48.312529 15135 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:48.208531 15256 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.149s
I20260709 13:04:48.372206  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.275966 15231 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:48.370025  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:48.368929  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 170152us
I20260709 13:04:48.327406 15205 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:48.339644 15265 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:48.369006  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:48.378805  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:48.377534 15219 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:48.385306 15135 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.396960  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.365833 15143 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.413458  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.399665  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:48.414590  8793 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:48.405719  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:48.330730 15211 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.383044 15247 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:48.343029 15246 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:04:48.416040  8680 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:48.416347  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:48.417224  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:48.418246  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:48.420810  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.397768  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2615) took 2920 ms. Trace:
I20260709 13:04:48.418200 14786 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.422822 15244 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:48.416637  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:48.382217 15206 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:48.428174  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 51775.9us
I20260709 13:04:48.412439 15223 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:48.437654  8754 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:48.462565  8826 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:48.463145  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.399019 15265 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:48.462080  8722 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:48.361071 15062 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.458963  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:48.460300  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.469434  8674 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:48.469906  8681 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 11: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:48.396188 15102 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:04:48.478497 15246 consensus_queue.cc:1243] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:48.428766 15245 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:48.463703 14540 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:04:48.460045 15174 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:48.480595  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.489537  8679 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:48.474067 15252 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:48.492311  8793 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:48.492435  8647 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:48.439968  8673 rpcz_store.cc:276] 0709 13:04:45.462963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.463769 (+   806us) service_pool.cc:224] Handling call
0709 13:04:45.491752 (+ 27983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:45.491996 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:45.600549 (+108553us) log.cc:844] Serialized 14 byte log entry
0709 13:04:46.869710 (+1269161us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b1800496420. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e20a14bc 00007f32df3ef48b 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a
0709 13:04:47.518700 (+648990us) log.cc:844] Serialized 14 byte log entry
0709 13:04:48.105135 (+586435us) log.cc:844] Serialized 14 byte log entry
0709 13:04:48.375324 (+270189us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:48.375390 (+    66us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:48.375441 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.375595 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.382980 (+  7385us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21469,"prepare.queue_time_us":818958,"prepare.run_cpu_time_us":177,"prepare.run_wall_time_us":6337,"raft.queue_time_us":537718,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":4079,"spinlock_wait_cycles":88692992,"thread_start_us":1662178,"threads_started":7,"wal-append.queue_time_us":745458,"wal-append.run_cpu_time_us":1619,"wal-append.run_wall_time_us":1104410}
I20260709 13:04:48.460676 15195 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:48.415197 15205 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.420598 15143 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.463016 15264 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:48.435943 15219 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.511432  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:48.478639 15110 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:48.526328  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:48.532043  8793 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:48.502271 15254 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:48.520859 14767 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.812s	user 0.014s	sys 0.000s
I20260709 13:04:48.537957  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:48.536470  8673 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:48.538861  8673 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:48.539901 15206 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:48.540962 15062 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:48.541409 15268 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.542009 15268 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.540103  8673 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.544044  8673 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:48.547778  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2649) took 1006 ms. Trace:
I20260709 13:04:48.548281  8681 rpcz_store.cc:276] 0709 13:04:47.541065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.541251 (+   186us) service_pool.cc:224] Handling call
0709 13:04:48.547499 (+1006248us) spinlock_profiling.cc:243] Waited 924 ms on lock 0x7b54003a2fd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:48.547703 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2125003136}
I20260709 13:04:48.549899  8676 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:04:48.536963 15110 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:48.550469 15267 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.112s
I20260709 13:04:48.551741 15211 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:48.552405 15211 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:48.552930 15211 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.526028 15264 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.556829 15264 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:48.557878 15209 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:48.540257 15262 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:48.560662 15262 raft_consensus.cc:697] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:48.561712 15262 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.562866 15205 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:48.514889 15181 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:48.559985  8748 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:48.568603  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:48.540822 15276 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:48.519304 15150 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:48.571264  8678 raft_consensus.cc:2410] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:04:48.565405 15223 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:48.575917  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 113950us
I20260709 13:04:48.580691  8754 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:04:48.581076  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:48.615473  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 246700us
I20260709 13:04:48.616643  8719 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:48.620415  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:48.620522  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:48.566354 15196 consensus_meta.cc:287] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.503s	user 0.013s	sys 0.000s
I20260709 13:04:48.619480  9353 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:48.626729  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:48.503640 15244 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:04:48.632241  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:48.624845  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:48.615502 15105 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.777s	user 0.011s	sys 0.006s
I20260709 13:04:48.523988 15208 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.570509 14540 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:48.638062  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:48.638136 15105 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.800s	user 0.011s	sys 0.006s
W20260709 13:04:48.638599  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 57630.9us
I20260709 13:04:48.633380 15195 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:48.673559  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:48.641002  8719 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:48.615634 15062 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:48.566573 15172 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:48.673843  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47229.3us
I20260709 13:04:48.572983 14767 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.637876 14829 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:48.682008  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 191687us
I20260709 13:04:48.665548  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:48.636812 15205 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:48.709394 15205 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.640884 15175 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:48.711998  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:48.714207 15105 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:48.717353 15135 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:48.720162  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:48.721186  8719 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:48.721378 14802 consensus_meta.cc:287] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.569s	user 0.005s	sys 0.005s
I20260709 13:04:48.722054  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:48.721838 14802 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.724612 15281 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:48.725143 15281 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.725505 15281 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:48.644233 15181 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:48.572175 14936 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.683928 15280 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:48.690946 15211 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:48.692732 15172 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:48.694495 15262 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:48.700342  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:48.705016 15213 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.705732  8829 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 17: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:48.706045 14829 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:48.706847 15174 consensus_queue.cc:1059] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:48.707523 15284 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:48.708755 14767 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:48.728094  8719 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:48.680689  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:48.728931  9353 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:48.726881 15279 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:48.744815  8673 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:04:48.773090  8673 pending_rounds.cc:85] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 10
I20260709 13:04:48.773401  8673 pending_rounds.cc:107] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:04:48.773651  8673 raft_consensus.cc:2889] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:48.618665 15229 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.678375 14540 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:48.743567 14382 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.786815  8754 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:48.779928  8681 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:48.789736  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.791998  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.793203  8681 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:48.794855 14382 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:48.799700  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.801156  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:48.802258  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.761122 15280 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.803035 15280 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:48.803673  8750 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:04:48.807035 15143 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:48.760258 15213 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.808177  8679 raft_consensus.cc:3055] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:48.808565  8679 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:48.809659 15213 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:48.809343  8679 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.811779  8747 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:48.812927 15208 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:48.815008 15287 raft_consensus.cc:3055] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:48.815531 15287 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:48.816584 15287 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.818007 15287 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:48.819957 14802 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:48.823086 15275 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.360s	user 0.004s	sys 0.000s
I20260709 13:04:48.825382 15110 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:48.826041 15110 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.828315 15245 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:48.828918 15245 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:48.829782 15245 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:48.830804  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 150219us
W20260709 13:04:48.831128 15149 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 13:04:48.831310 15150 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:48.831784 15150 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.832158 15150 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:48.834590  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:48.835532 15274 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:04:48.777951 15204 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:48.836267  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:48.836735 15204 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:48.837203 15204 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:48.837299  8676 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:48.784898 15105 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:48.839174 15105 raft_consensus.cc:697] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:48.840154 15105 consensus_queue.cc:237] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:48.775486 15217 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.594s	user 0.011s	sys 0.003s
I20260709 13:04:48.810571  8679 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:48.828425 15110 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:48.845548  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.635663 15196 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.849850  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:48.850049  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.852988  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:48.769830 15252 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:48.760409 15262 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.856994  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:48.857259 15245 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:48.887408 15174 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.892733 15262 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:48.892820 15174 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.894907 14767 consensus_queue.cc:1059] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:48.896454 14767 raft_consensus.cc:3055] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:48.896929 14767 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:48.857699  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:48.897838 15206 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:48.857867 15107 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.898447 15206 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.900372 15107 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:48.900786 15206 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:48.901252  8748 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:48.846235 15279 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:48.902393 15279 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.903143  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:48.866745 15217 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:48.868547 15155 consensus_meta.cc:287] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.186s	user 0.011s	sys 0.004s
I20260709 13:04:48.869035 15265 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.869611  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:48.860095 15216 consensus_meta.cc:287] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.776s	user 0.012s	sys 0.001s
I20260709 13:04:48.880725 15276 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:48.880066  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 45118.1us
I20260709 13:04:48.887995 15256 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:48.939457  8793 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:48.908063 15172 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:48.878564 15231 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.890302  8675 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:04:48.882700 15149 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:48.981382  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:48.863296 15222 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.753s	user 0.004s	sys 0.009s
I20260709 13:04:48.888192 14382 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:48.915795  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:48.983181  8748 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:48.984220  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:48.984668 15229 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:48.984752 15149 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.985584  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:48.917575  8753 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:48.987052  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:48.987447 15149 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:48.987819  8753 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 11: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:48.920403 15062 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:48.921041  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:48.928138  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:48.928568  8722 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:48.884377  8681 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:48.940994  8796 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:48.981052  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:48.860065 15162 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:49.021608  8827 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 1335) took 1776 ms (client timeout 2134 ms). Trace:
I20260709 13:04:48.872002 15195 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:49.019534  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:49.019800  8679 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:04:49.023355  8821 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
W20260709 13:04:49.024245  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 95742.2us
I20260709 13:04:48.921245 15189 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.061s
W20260709 13:04:49.022156  8827 rpcz_store.cc:269] 0709 13:04:47.245651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.247950 (+  2299us) service_pool.cc:224] Handling call
0709 13:04:49.021399 (+1773449us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b540009ced4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:49.021545 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460704000}
I20260709 13:04:49.028860  8751 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:04:48.981489 15168 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:04:48.923990 15155 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:48.923522 15284 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:04:48.961483 15143 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:49.062059  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.065294  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:49.071393  8678 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:49.021621 14771 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:49.078555  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.079625  8827 raft_consensus.cc:2410] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:04:48.934846 15269 consensus_queue.cc:1059] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.104s
I20260709 13:04:48.931154 14767 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:48.934993 15213 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 13:04:49.105341  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:49.006407 15276 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:04:49.112259  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.034798 15172 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:49.105724  8825 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:49.123397  8679 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
W20260709 13:04:49.111651  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 170723us
I20260709 13:04:49.081998  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2500) took 1197 ms. Trace:
I20260709 13:04:48.929344 14829 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:49.147743  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 208377us
I20260709 13:04:49.073987 15284 consensus_queue.cc:1243] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:48.996929 15222 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.152516  8827 rpcz_store.cc:276] 0709 13:04:47.884304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.078259 (+1193955us) service_pool.cc:224] Handling call
0709 13:04:49.081923 (+  3664us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:49.159884  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.160869 15222 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.161303  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.161767 15192 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:04:49.108033  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.162227 15192 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:49.163211 15192 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.163933 15280 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.164655 15192 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.166778 15280 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.168177 15196 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.169028 15196 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:49.169407  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:49.170102  9388 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:49.170513  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:48.962406 15252 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 13:04:49.156594  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:49.173579  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:49.174913  8672 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.175712  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:49.176208 15289 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:04:49.178237  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:49.173334  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2498) took 1289 ms. Trace:
I20260709 13:04:48.932758 15216 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.179549  8827 rpcz_store.cc:276] 0709 13:04:47.883691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.161012 (+1277321us) service_pool.cc:224] Handling call
0709 13:04:49.173085 (+ 12073us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:49.108088 15267 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.190965 15267 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.193562 15267 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:48.982080 15256 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.031823 15162 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.195969 15162 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.196031 15265 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.197780  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:49.144990 15143 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.199600 15231 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:49.201843 15256 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.202886  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:49.204557  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:49.205456  8679 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 17: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:49.212462 15281 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.081483 14540 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:49.184698 15222 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:49.185009  9388 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:49.223356 15222 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.224622  8796 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:49.031545 15062 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.185959  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:49.187201  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:49.198016  8675 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:49.225927 15205 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.226707  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2518) took 1038 ms. Trace:
I20260709 13:04:49.226783 15205 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:49.227102  8827 rpcz_store.cc:276] 0709 13:04:48.188465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.186733 (+998268us) service_pool.cc:224] Handling call
0709 13:04:49.226639 (+ 39906us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:49.227511 15205 raft_consensus.cc:697] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:49.228333  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.229223  8827 raft_consensus.cc:2410] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:04:49.229640  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2496) took 1346 ms. Trace:
I20260709 13:04:49.231534 15217 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:49.232052  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2495) took 1349 ms. Trace:
I20260709 13:04:49.151454 15127 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:49.232920 15127 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.233779  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:49.235569 15127 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.235618  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.236649  8823 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:49.235697 15062 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.232487  8824 rpcz_store.cc:276] 0709 13:04:47.882653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.882836 (+   183us) service_pool.cc:224] Handling call
0709 13:04:47.883197 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.063260 (+180063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.231784 (+1168524us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b540032f854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:49.231981 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256269184}
W20260709 13:04:49.216557  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 46139.2us
I20260709 13:04:49.216061 15213 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:49.238775  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.239326 15213 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.217180 15107 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:49.242036 15107 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.242568 15107 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:49.247668  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.248134  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:49.239548  8824 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:49.250346  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2519) took 1061 ms. Trace:
I20260709 13:04:49.251148  8824 rpcz_store.cc:276] 0709 13:04:48.188803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.238330 (+1049527us) service_pool.cc:224] Handling call
0709 13:04:49.250247 (+ 11917us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:49.252749  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.253757  8824 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:49.255219  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.231127  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1243) took 1808 ms. Trace:
I20260709 13:04:49.237478  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:49.261621  8749 rpcz_store.cc:276] 0709 13:04:47.422298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.422472 (+   174us) service_pool.cc:224] Handling call
0709 13:04:47.422890 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.554021 (+131131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.230868 (+1676847us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54001040d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:49.231064 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299047296}
I20260709 13:04:49.263959  8749 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:49.264755  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:49.149619 15276 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:49.104558 15155 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:49.257017  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:49.248764  8823 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:49.268899  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.272079  8827 rpcz_store.cc:276] 0709 13:04:47.883029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.228023 (+1344994us) service_pool.cc:224] Handling call
0709 13:04:49.229561 (+  1538us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:49.273641  8672 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:49.274040  8645 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:49.315929  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:49.317811  8645 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:49.183471  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:49.206501  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:49.276003  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:49.284832  8753 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:49.287909  9383 raft_consensus.cc:2393] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 13: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:49.295154  8749 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:04:49.295362  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.321161  8823 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:49.322505  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:49.323395  8823 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:49.298422  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:49.300594  8751 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:49.327251  8678 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:49.349771  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:49.258553 15127 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:49.328359  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:49.260493 15174 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:49.275727 15222 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:49.110322 14936 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.340165  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:49.358769  8749 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:04:49.336623  8754 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:49.328655  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2517) took 1140 ms. Trace:
I20260709 13:04:49.347139  8677 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:49.360567  8823 rpcz_store.cc:276] 0709 13:04:48.188130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.322202 (+1134072us) service_pool.cc:224] Handling call
0709 13:04:49.328581 (+  6379us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:49.354655  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:49.327781  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:49.357807  8748 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:04:49.324301 15223 consensus_meta.cc:287] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.003s
I20260709 13:04:49.340561  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:49.383273  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 55586.2us
I20260709 13:04:49.340973 15149 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:04:49.327826 14802 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:49.384307  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:49.363570  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.361759  8672 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:49.383098  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:53352 (request call id 2638) took 2121 ms. Trace:
I20260709 13:04:49.333137 15295 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:49.387578  8677 rpcz_store.cc:276] 0709 13:04:47.240140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.419444 (+179304us) service_pool.cc:224] Handling call
0709 13:04:49.342171 (+1922727us) spinlock_profiling.cc:243] Waited 53.5 ms on lock 0x7b540033a4d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f0a3d 00007f32e74c0cd0 00007f32e0a45254 00007f32e0a451e2 00007f32e0a45111 00007f32e0a4508c 00007f32e0a44342 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:49.361102 (+ 18931us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123060480}
I20260709 13:04:49.339298 15147 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:49.332928 15220 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:49.386111  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:49.300953  8824 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:04:49.360442 15150 consensus_meta.cc:287] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.525s	user 0.012s	sys 0.000s
W20260709 13:04:49.390759  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:49.396822  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.348100 15213 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.393815  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:49.390826  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:49.389796  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:49.403832  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:49.404711  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:49.406039  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:49.408761  8678 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.409237  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:49.339581 15155 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:04:49.341482 15219 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:49.388085 15223 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:49.408427 15150 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.357306 15127 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.396106 15220 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:49.412063 15204 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.574s	user 0.009s	sys 0.004s
I20260709 13:04:49.409642 14540 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:49.416049 15213 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:49.359365 15102 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:04:49.411893 15294 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:49.390391 15174 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:49.387938 14802 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:49.416579  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:49.433100  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:49.427134  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:49.386952 15256 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:04:49.349589 15205 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.445175  8676 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.438273 15102 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:04:49.446117  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 416639us
I20260709 13:04:49.449815  8823 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:04:49.450385  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.355288 15107 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.409991 15222 consensus_queue.cc:1059] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:04:49.360325 15276 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.435686 15245 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:49.441860 15174 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.490828  8679 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:49.491559  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:49.458079  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:49.463534  8754 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:49.457542  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2508) took 1421 ms. Trace:
I20260709 13:04:49.435981 15220 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:49.483536  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:49.485175  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:49.399220 15295 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:49.449735 15135 consensus_meta.cc:287] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.732s	user 0.017s	sys 0.000s
I20260709 13:04:49.450974 14540 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:49.447214 14802 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.441643 15294 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:49.451210 15233 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.487509 15245 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:49.540992 15245 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.453363 15102 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:49.545603 15102 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.547387 15102 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:49.498833  9388 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:49.439953 15204 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.551654 15204 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:49.515821  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:49.552898 15217 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:49.553985  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 1258) took 1162 ms. Trace:
I20260709 13:04:49.555229  8751 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19. Ops: [16.13-18.14]
I20260709 13:04:49.521504  8676 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:49.522507 15127 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:49.491959 15174 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.558678  9387 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:49.526779 15294 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.527710 15133 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:49.519825 15216 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:49.529162 15281 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:49.537909  8673 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:49.494310  8828 rpcz_store.cc:276] 0709 13:04:48.036367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.456430 (+1420063us) service_pool.cc:224] Handling call
0709 13:04:49.456867 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.457123 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.457163 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:49.457229 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:49.457277 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.457341 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.457464 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:49.565443  8678 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:49.571976  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:49.572914  8828 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:49.565867  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 50113.6us
I20260709 13:04:49.576529  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:49.562130  8747 rpcz_store.cc:276] 0709 13:04:48.391198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.393877 (+  2679us) service_pool.cc:224] Handling call
0709 13:04:49.553628 (+1159751us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b540038bdd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:49.553864 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":824145664}
I20260709 13:04:49.579054  8747 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:49.583112  8747 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:49.585285  8722 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:49.584146 15107 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:49.591491 15280 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:49.592681 15271 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:49.593176 15271 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:49.594389 15244 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.595016 15244 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.594228  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:49.599187  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:49.599560  8822 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.600133  9388 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:49.615608  8722 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:49.512972 15279 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:49.587545 15281 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:49.634663  8722 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:49.636138  8828 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:49.639559  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:49.606427 15244 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.607131 15213 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.649323  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:49.566958 15233 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.594200 15271 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.655748  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:04:49.660792  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 137064us
I20260709 13:04:49.661657  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:49.662946  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:49.663267  9355 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.560s (SENT) [suppressed 12 similar messages]
I20260709 13:04:49.645115 15217 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:49.664700  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:49.649019 15107 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:49.627576 15280 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:04:49.666325  8722 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:49.650673 15279 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:49.667857  8754 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:49.672966  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:49.653538 15269 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:49.667119 15217 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.675433  8722 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:49.666795 15281 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:49.669343 15231 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:49.671133 15107 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:49.656531 15213 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.677572 15265 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:49.678997 15150 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:49.700347 15299 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 13:04:49.685154 15271 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.684520 15217 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:49.696764 15252 raft_consensus.cc:3055] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:49.704924 15105 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:49.655412 14540 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.723735  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2497) took 1840 ms. Trace:
I20260709 13:04:49.709708 15107 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.688387 15281 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.733520  8824 rpcz_store.cc:276] 0709 13:04:47.883406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.414427 (+1531021us) service_pool.cc:224] Handling call
0709 13:04:49.415075 (+   648us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.432419 (+ 17344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.432449 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:49.446995 (+ 14546us) log.cc:844] Serialized 26 byte log entry
0709 13:04:49.637550 (+190555us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:49.674054 (+ 36504us) log.cc:844] Serialized 14 byte log entry
0709 13:04:49.674317 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.674405 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:49.723462 (+ 49057us) raft_consensus.cc:1685] finished
0709 13:04:49.723489 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.723658 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16705,"prepare.queue_time_us":79383,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":23297,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":10856,"thread_start_us":164974,"threads_started":2,"wal-append.queue_time_us":210295}
I20260709 13:04:49.730014 15213 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:49.743002  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:49.743775  8824 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:49.737545 15105 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:49.746835 15105 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:49.749065  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.718588 15252 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:49.740967 15281 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.764109  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2527) took 2036 ms. Trace:
I20260709 13:04:49.764690  8826 rpcz_store.cc:276] 0709 13:04:47.727650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.727833 (+   183us) service_pool.cc:224] Handling call
0709 13:04:47.751809 (+ 23976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.466120 (+714311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.763802 (+1297682us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b54003de0d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:49.764032 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1633856256}
I20260709 13:04:49.766075  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.766480 14771 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:49.767099 14771 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:49.767158  8825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2506) took 1833 ms (client timeout 1856 ms). Trace:
W20260709 13:04:49.767627  8825 rpcz_store.cc:269] 0709 13:04:47.934085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.101810 (+167725us) service_pool.cc:224] Handling call
0709 13:04:49.766921 (+1665111us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b5400126954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:49.767089 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2221794688}
I20260709 13:04:49.768843  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:49.769639  8825 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:49.771181  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:49.771696  8825 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:49.778765  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:49.779349 15301 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.104s	user 0.002s	sys 0.000s
I20260709 13:04:49.780277  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:49.780359 15149 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:49.780938  8826 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:49.780992 15149 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:49.786311  9388 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:49.788024  9384 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:49.791033 15252 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.792502 15252 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:49.795877  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.797454  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:49.799072  9386 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:04:49.805361  9353 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:49.805503 15248 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:49.809621 15216 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:49.811537  8676 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:04:49.812001  8676 pending_rounds.cc:85] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 10
I20260709 13:04:49.812232  8676 pending_rounds.cc:107] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:04:49.812459  8676 raft_consensus.cc:2889] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:49.812729 15268 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:49.816210  8829 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2513) took 1650 ms (client timeout 1763 ms). Trace:
W20260709 13:04:49.816674  8829 rpcz_store.cc:269] 0709 13:04:48.165910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.258064 (+ 92154us) service_pool.cc:224] Handling call
0709 13:04:49.815992 (+1557928us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b540032e954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:49.816138 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":856371968}
W20260709 13:04:49.817729  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.817916  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:49.818357  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.821254  9355 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:49.750133 15295 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.822018 15135 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.822635 15233 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:49.822562 15135 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.824928 15135 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:49.827469 15135 consensus_queue.cc:1059] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:49.828689 15256 raft_consensus.cc:3055] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:49.829128 15256 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:49.830009 15256 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.831267 15256 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:49.831633 15222 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:49.834676 15219 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:04:49.836663 15288 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.549s	user 0.000s	sys 0.002s
W20260709 13:04:49.757264 15192 consensus_meta.cc:287] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.006s
I20260709 13:04:49.837779 15222 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:49.782377 15149 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.839540 15222 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:49.840705 15147 raft_consensus.cc:3055] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:49.841145 15147 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:04:49.841995  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.843200  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:49.845877  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2510) took 1762 ms. Trace:
I20260709 13:04:49.847261 15245 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.847741 15218 raft_consensus.cc:3055] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:49.851004 15175 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:49.850973 15218 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:49.851751 15175 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.852227 15218 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.853685 15218 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:49.854034  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2529) took 2125 ms. Trace:
I20260709 13:04:49.855309  8794 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:49.856658 15295 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:49.857901 15295 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:49.858469 15295 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.859874 15150 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:49.860711 15150 raft_consensus.cc:3055] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:49.861049 15150 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:04:49.861258 15291 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.429s	user 0.001s	sys 0.001s
I20260709 13:04:49.863531  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1259) took 1461 ms. Trace:
I20260709 13:04:49.864707 15295 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.865420 15276 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:49.863999  8755 rpcz_store.cc:276] 0709 13:04:48.402303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.402530 (+   227us) service_pool.cc:224] Handling call
0709 13:04:48.403096 (+   566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:48.639000 (+235904us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:48.639038 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:48.675076 (+ 36038us) log.cc:844] Serialized 26 byte log entry
0709 13:04:48.988433 (+313357us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:48.988474 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.988553 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:49.860425 (+871872us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:49.863154 (+  2729us) raft_consensus.cc:1685] finished
0709 13:04:49.863180 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.863458 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19764,"raft.queue_time_us":110736,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":2849,"spinlock_wait_cycles":24865664,"thread_start_us":425316,"threads_started":2,"wal-append.queue_time_us":610074}
I20260709 13:04:49.849855 15143 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.854660  8821 rpcz_store.cc:276] 0709 13:04:47.728357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.728470 (+   113us) service_pool.cc:224] Handling call
0709 13:04:49.023155 (+1294685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.028488 (+  5333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.853698 (+825210us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540009ced0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:49.853906 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2926618880}
I20260709 13:04:49.876181  8678 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:04:49.842063 15147 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.877017 15147 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:49.829923  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:49.879230  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:49.879858  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50055.5us
I20260709 13:04:49.880964  8750 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:49.881114  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:49.881359 15206 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:49.882721 15223 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:49.883270  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2618) took 1058 ms. Trace:
I20260709 13:04:49.883723  8750 rpcz_store.cc:276] 0709 13:04:48.824694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.183108 (+358414us) service_pool.cc:224] Handling call
0709 13:04:49.883074 (+699966us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b540034b054. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:49.883191 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1291999616}
W20260709 13:04:49.884032  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 86729.2us
I20260709 13:04:49.884984  8719 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:49.846282  8823 rpcz_store.cc:276] 0709 13:04:48.083372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.845052 (+1761680us) service_pool.cc:224] Handling call
0709 13:04:49.845341 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.845552 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.845579 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:49.845624 (+    45us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:49.845667 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.845724 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.845822 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:49.861867 15150 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.887609  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:49.890259  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:49.891510  8677 raft_consensus.cc:2410] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 13:04:49.899698  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:49.945245  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.903424  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:49.946650  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:49.904680  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:49.910207 14786 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:04:49.905668  8822 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:49.959802  8823 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:49.960207  8822 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:49.960860  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2524) took 1770 ms. Trace:
I20260709 13:04:49.961440  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:49.966187  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:49.961207  8822 rpcz_store.cc:276] 0709 13:04:48.190545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.959708 (+1769163us) service_pool.cc:224] Handling call
0709 13:04:49.960042 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.960616 (+   574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.960803 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:49.969022  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:49.969725  8749 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:49.974073 15245 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.979010 15216 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:49.980906 15216 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:49.981418 15216 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:49.939743 15174 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:49.982885 15149 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:49.983150  9383 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:04:49.983662 15174 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:49.984093  9383 raft_consensus.cc:606] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:49.984180  8827 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:49.991950  8752 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:04:49.997363  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:49.909670  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54470.5us
I20260709 13:04:49.911473  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:49.912266  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:49.911923 15107 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:50.001045  8675 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:50.001555  8675 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:50.002400  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:50.002815  8675 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:50.004117  8675 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:50.007565  8747 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:04:49.776144  8722 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:49.854385 15175 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.924450 14802 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:49.913323  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:50.010442 14802 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:49.929275  9384 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:49.934053  8830 raft_consensus.cc:2410] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:04:49.883200 15223 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:50.011976  8673 raft_consensus.cc:1217] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Deduplicated request from leader. Original: 9.6->[11.7-17.10]   Dedup: 13.8->[15.9-17.10]
I20260709 13:04:49.934880  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:49.936277  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 51419.6us
I20260709 13:04:50.012543  8673 pending_rounds.cc:85] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 8
I20260709 13:04:50.012836  8673 pending_rounds.cc:107] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:04:50.013070  8673 raft_consensus.cc:2889] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:49.929275  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:50.013824  8677 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:50.009119  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:50.012077 15223 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:50.014640  8747 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:50.014935  8825 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:04:50.015242  8719 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:49.943161  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:50.016662  8750 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:50.016959  8747 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:50.017498  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:50.009143  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 233064us
I20260709 13:04:49.944084  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:50.017983  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:49.948132  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:49.940928 15276 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:50.022742  8680 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:50.023027 15276 raft_consensus.cc:697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:49.904680  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:49.951906  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:50.024485  8754 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:49.948539 15150 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:49.953709  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:49.907752 15195 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:49.951575  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:50.012157  8674 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:50.029295  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:49.969245  8822 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:50.031138  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2525) took 1575 ms. Trace:
I20260709 13:04:50.031535  8822 rpcz_store.cc:276] 0709 13:04:48.455899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.968502 (+1512603us) service_pool.cc:224] Handling call
0709 13:04:49.968855 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.030838 (+ 61983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.031077 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:50.035940  8750 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:50.037355  8826 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:50.039614  8675 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 13:04:50.040202  8675 pending_rounds.cc:85] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 13
I20260709 13:04:50.001600 15107 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:49.912667  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:50.050448  8678 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:50.021828 15279 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:49.952785  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:49.900652  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 79490.6us
I20260709 13:04:50.026906  8747 raft_consensus.cc:2393] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:50.051651  8756 raft_consensus.cc:2410] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:04:49.949630 15172 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.040637  8675 pending_rounds.cc:107] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:04:50.056490  8822 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:04:50.056645  8675 raft_consensus.cc:2889] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:50.056973  8822 pending_rounds.cc:85] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 8
I20260709 13:04:50.057284  8822 pending_rounds.cc:107] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:04:50.057551  8822 raft_consensus.cc:2889] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:50.040709  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2523) took 1850 ms. Trace:
I20260709 13:04:50.057924  8823 rpcz_store.cc:276] 0709 13:04:48.190174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.886494 (+1696320us) service_pool.cc:224] Handling call
0709 13:04:49.959614 (+ 73120us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.960191 (+   577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.040468 (+ 80277us) spinlock_profiling.cc:243] Waited 72.7 ms on lock 0x7b540009c9d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:50.040623 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167221632}
I20260709 13:04:49.937180 15217 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:50.059463  8823 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:50.060281  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 60996us
I20260709 13:04:50.063370  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:50.064896  8676 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:50.040982  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2511) took 1949 ms. Trace:
I20260709 13:04:50.068960  8827 rpcz_store.cc:276] 0709 13:04:48.091997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.350545 (+1258548us) service_pool.cc:224] Handling call
0709 13:04:50.040745 (+690200us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b540032f354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:50.040900 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1372185856}
I20260709 13:04:50.069904  8794 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:49.999408  8796 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:50.043679  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:50.039939  8793 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:50.051829  8830 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2499) took 2167 ms (client timeout 2204 ms). Trace:
W20260709 13:04:50.075400  8830 rpcz_store.cc:269] 0709 13:04:47.883982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.635413 (+751431us) service_pool.cc:224] Handling call
0709 13:04:50.051588 (+1416175us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540009d8d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:50.051763 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2872375040}
I20260709 13:04:49.939152 15294 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:50.085884  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:50.029301  8719 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:49.911051  8824 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:04:50.055250  8750 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:49.958873 15133 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:50.090238  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 66115us
I20260709 13:04:50.091159  8719 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:50.080116  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:50.094190  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:50.094997  8754 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
W20260709 13:04:50.086493 15262 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:04:50.107719  8748 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:49.982352 15216 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:50.025017 15220 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:50.085487 15102 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:50.105314 15195 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:50.089466 15252 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:50.144594 15294 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:50.079054 15189 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:04:50.024147 15276 consensus_queue.cc:237] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:50.155324  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:50.078501 15295 consensus_queue.cc:1059] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 13:04:50.087656 15206 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:50.078922 15172 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:50.164760  8755 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:50.166993 15172 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:50.123832 15281 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:50.101222 15133 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:50.099815 15303 consensus_queue.cc:1059] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:04:50.134898 15256 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.085944 15279 raft_consensus.cc:697] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:50.176246 15220 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:50.177026  8672 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:50.137477 15217 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:50.099257 15107 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:50.177819 14771 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:50.098953 15168 raft_consensus.cc:2764] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:04:50.187453  8675 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:50.139510  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:50.050122 15175 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:04:50.198405 15192 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:04:50.189879  8722 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:50.194084  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 194756us
I20260709 13:04:50.240991  8826 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
W20260709 13:04:50.245503  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:50.196336  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 103220us
W20260709 13:04:50.251298  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 92681.6us
I20260709 13:04:50.251825  8796 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:50.198900 15195 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:50.199271 15213 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:50.261336  8794 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:04:50.265276  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 58380.8us
I20260709 13:04:50.270567  8749 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:04:50.271236 14382 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:50.259922  8722 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:50.280098  8826 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: [15.11-17.12]
I20260709 13:04:50.201620 15281 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:50.294345  8752 raft_consensus.cc:606] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:04:50.282560  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 84664.4us
I20260709 13:04:50.279840  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:50.285334  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.105551 15154 consensus_queue.cc:1059] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.159s
I20260709 13:04:50.288437 15280 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:50.287878 15195 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:50.240151 15107 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.292338  8796 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:50.293134 15252 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:50.298445  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.303962  9353 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:50.295001  8752 raft_consensus.cc:614] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:50.304751  8754 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:04:50.296880  8794 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:50.248522 15245 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.325841  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:50.327680  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:50.257254 15217 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:50.206727 15256 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:50.302736 14767 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:50.352847  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:50.322813 15213 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:50.353804  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:50.364387  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 263877us
I20260709 13:04:50.320724 15107 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:50.294834 15174 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:04:50.324542 15206 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:50.369223  9353 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:50.358086 15252 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:50.374872 15198 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:50.376217 15189 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:50.382997 15279 consensus_queue.cc:237] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:50.327056 15281 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:04:50.363269 15213 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.384910  8823 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:50.295156 15294 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:04:50.383023  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2509) took 2316 ms. Trace:
I20260709 13:04:50.345412 15245 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:50.389060  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.345973 15178 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:50.346171 15280 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:50.351809 14540 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:50.374979  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2964) took 1103 ms. Trace:
I20260709 13:04:50.359208 15157 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:04:50.328039 14786 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:04:50.410573  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:50.460685 15157 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:04:50.443816  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.431339 15223 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:50.407346  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:50.464483  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.464852  8822 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:50.409535 15234 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:04:50.465999 15223 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:50.427189 14771 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:50.463577 15157 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:50.407900  8754 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:50.468034 15157 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:50.406934 15154 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:50.469220  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:50.469560 15154 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.469697 15157 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:50.459983 15213 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:50.408787 15201 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:04:50.471892 15222 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:50.472263  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:50.472785  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.472365 15222 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:50.444691  8828 raft_consensus.cc:2410] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:04:50.474740 15294 consensus_queue.cc:1243] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:50.474927 14936 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:50.475587  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:50.475641 14936 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:50.478081 14936 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:50.409082 15256 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:50.479337  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:50.480072  8675 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:50.481643  8754 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:04:50.485242  8754 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:04:50.486140  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:50.486959  8755 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
I20260709 13:04:50.487773  8672 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:04:50.466876  8829 rpcz_store.cc:276] 0709 13:04:48.066280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.844324 (+1778044us) service_pool.cc:224] Handling call
0709 13:04:49.844733 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.845670 (+   937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.845710 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:49.846338 (+   628us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.253795 (+407457us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:50.253845 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.253927 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.382699 (+128772us) raft_consensus.cc:1685] finished
0709 13:04:50.382728 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.382952 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":128,"thread_start_us":407000,"threads_started":1,"wal-append.queue_time_us":531660}
I20260709 13:04:50.459666  8681 rpcz_store.cc:276] 0709 13:04:49.273522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.702558 (+429036us) service_pool.cc:224] Handling call
0709 13:04:49.702895 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.703349 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:49.703377 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:49.703906 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.067421 (+363515us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:50.067476 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.067560 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.374724 (+307164us) raft_consensus.cc:1685] finished
0709 13:04:50.374748 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.374911 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":132,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":96,"thread_start_us":301244,"threads_started":1,"wal-append.queue_time_us":639553}
I20260709 13:04:50.461907 15245 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:50.407244 15133 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:50.462280 15280 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:50.450820  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:50.463186 14540 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:50.463718 15233 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:50.531047  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:50.470074  8680 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:50.493472  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:50.494344  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:50.495607  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 210297us
I20260709 13:04:50.496821  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:50.533521  8754 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:50.534880 15172 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:50.537245  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:50.498178  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:50.505455  8645 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:50.510771 15189 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:50.523144  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:50.558182  8751 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 16: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:50.513444 15135 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.526377  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2526) took 2070 ms. Trace:
I20260709 13:04:50.470013 15154 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:50.518380  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:50.552177  8722 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:50.539738  8645 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:50.565351  8679 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:50.545418  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:50.566980  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:50.529742 14540 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:50.528034 15244 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:50.561422  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 449442us
I20260709 13:04:50.573406  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:50.575502  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:50.560468  8825 rpcz_store.cc:276] 0709 13:04:48.456272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.020981 (+1564709us) service_pool.cc:224] Handling call
0709 13:04:50.021291 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.059917 (+ 38626us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:50.091841 (+ 31924us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.525949 (+434108us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:50.526010 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:50.526072 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.526174 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.526316 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48014,"prepare.queue_time_us":31918,"raft.queue_time_us":38705,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":39409,"spinlock_wait_cycles":3814784,"thread_start_us":432635,"threads_started":1}
I20260709 13:04:50.579623  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:50.557418 15256 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:50.591832  8675 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:50.586380  8681 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:04:50.557994 15222 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:50.561302 15276 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:50.603061  8719 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:50.603061  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:50.614200  8676 raft_consensus.cc:3055] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:50.615496  8676 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:50.616185  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:50.616950  8825 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:50.520988 15149 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:04:50.564157 14767 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.627619 15222 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:50.627391 14767 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:50.628059 15222 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:50.629377 15222 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:50.630044 14767 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:50.630774 15222 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:50.605577  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2514) took 2418 ms. Trace:
I20260709 13:04:50.615443  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:50.637096 15149 consensus_queue.cc:1059] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:04:50.571570 15245 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:50.577833 15147 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:50.590682 14936 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:50.676368  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:50.599452 15150 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:50.609143 15256 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.688099 15256 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:50.581269 15265 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 13:04:50.693872 15154 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:50.697116 15154 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:50.569231 15135 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:50.686873  8825 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:50.708904  9384 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:50.711604  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 1346) took 1229 ms. Trace:
I20260709 13:04:50.758927  8828 rpcz_store.cc:276] 0709 13:04:49.481984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.672583 (+190599us) service_pool.cc:224] Handling call
0709 13:04:50.711266 (+1038683us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b540041d054. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:50.711506 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1769036032}
I20260709 13:04:50.760605  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:50.712736  8826 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:04:50.761703  8826 pending_rounds.cc:85] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 12
I20260709 13:04:50.762027  8826 pending_rounds.cc:107] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:04:50.762285  8826 raft_consensus.cc:2889] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:50.718166  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.681270  8676 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:50.748474  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:50.764875  8676 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:50.767405  8679 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:50.751752  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1348) took 1268 ms. Trace:
I20260709 13:04:50.770905  8822 rpcz_store.cc:276] 0709 13:04:49.483495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.058591 (+575096us) service_pool.cc:224] Handling call
0709 13:04:50.058942 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.465393 (+406451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.751406 (+286013us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54002292d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:50.751661 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":933008768}
W20260709 13:04:50.792312  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.661072 15294 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:50.873396 15294 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.873945 15294 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:50.876487 15326 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:50.880505 15262 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.881389 15262 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:50.883441 15320 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.200s	user 0.000s	sys 0.000s
I20260709 13:04:50.885016 15262 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:50.868002  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:50.868023  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 99722.5us
I20260709 13:04:50.772855  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:50.888234  8822 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:50.682775 15276 raft_consensus.cc:697] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:50.892822  8824 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:50.678385 15147 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.908558  8753 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:04:50.902189  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:50.903111  8794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.027s	sys 0.015s
W20260709 13:04:50.755981 15316 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
W20260709 13:04:50.919368  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:50.919826  8794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.027s	sys 0.016s
I20260709 13:04:50.919972  8824 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:50.864817 15149 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:50.894579 15220 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.929817  8676 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
W20260709 13:04:50.935458  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.910128 15147 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:50.941076 15195 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.005s	sys 0.022s
I20260709 13:04:50.680619 15245 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:50.928711  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 601072us
I20260709 13:04:50.990396  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:50.949893  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.992278  8753 raft_consensus.cc:606] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:50.950120  8822 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:50.941807  8824 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:04:50.940644 15269 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.005s	sys 0.019s
I20260709 13:04:50.924389 14936 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:50.988780 15195 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.005s	sys 0.022s
I20260709 13:04:50.951174  8676 pending_rounds.cc:85] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 9
I20260709 13:04:50.952805 15279 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.011934  8676 pending_rounds.cc:107] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:04:51.016746  8676 raft_consensus.cc:2889] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:50.951545 15143 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.000s	sys 0.026s
W20260709 13:04:51.019443 15143 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.000s	sys 0.028s
W20260709 13:04:51.014243  8748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.015s	sys 0.014s
W20260709 13:04:51.020951  8748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.015s	sys 0.014s
W20260709 13:04:51.009246  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 406243us
W20260709 13:04:51.022341  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 362426us
I20260709 13:04:51.025715  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:51.000618 15149 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.999723 15172 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:51.004375 15269 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.005s	sys 0.019s
W20260709 13:04:51.054313  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.030277  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:51.055675  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 516974us
I20260709 13:04:51.009902 15268 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:51.064623  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:51.060250  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:51.071815  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.060011  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:51.087379  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:50.987728 15220 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:51.034898 15168 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:04:51.118234  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:51.119637  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:51.060631 15328 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.380s	user 0.000s	sys 0.001s
W20260709 13:04:51.128969  8722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.014s	sys 0.023s
W20260709 13:04:51.128223  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 132081us
W20260709 13:04:51.134835 15324 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
W20260709 13:04:51.093546 15107 consensus_meta.cc:287] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.690s	user 0.008s	sys 0.004s
I20260709 13:04:51.128691 15222 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:51.133276 14540 consensus_meta.cc:287] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.002s
W20260709 13:04:51.135689 15317 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:04:51.175443  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.166267  8722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.018s	sys 0.034s
I20260709 13:04:51.084481 15149 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.113633 15268 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:51.178750  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.169828 15157 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.699s	user 0.005s	sys 0.009s
I20260709 13:04:51.160355 15326 raft_consensus.cc:697] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:51.177460 15065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.010s	sys 0.037s
I20260709 13:04:51.022521 15279 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.167249 15135 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:51.158218 15269 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.026108 15276 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.178673 15107 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.092464 15172 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:51.210685 15065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.011s	sys 0.038s
W20260709 13:04:51.242787  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46422.6us
I20260709 13:04:51.140079 14936 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.180526 15222 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.251217 15222 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:51.251343 15268 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:51.253279  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.254200  8751 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:51.257373  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.257958  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:51.257992  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.258773  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.228264 15135 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.251819 15222 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.279021  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2679) took 1343 ms. Trace:
I20260709 13:04:51.246047 15269 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.279533  8673 rpcz_store.cc:276] 0709 13:04:49.935302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.011379 (+ 76077us) service_pool.cc:224] Handling call
0709 13:04:50.011745 (+   366us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:04:50.051128 (+ 39383us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:50.558748 (+507620us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.014894 (+456146us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.015124 (+   230us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:51.015933 (+   809us) log.cc:844] Serialized 52 byte log entry
0709 13:04:51.048042 (+ 32109us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:51.048122 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.048274 (+   152us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.278720 (+230446us) raft_consensus.cc:1685] finished
0709 13:04:51.278746 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.278943 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37306,"prepare.queue_time_us":670819,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":50297,"raft.run_cpu_time_us":518,"raft.run_wall_time_us":50990,"spinlock_wait_cycles":8702592,"thread_start_us":878169,"threads_started":2,"wal-append.queue_time_us":703253}
I20260709 13:04:51.182263 14540 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:51.281436 15256 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.593s	user 0.007s	sys 0.003s
I20260709 13:04:51.211613 15349 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:51.269348  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.271833 15335 log.cc:927] Time spent T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:04:51.271370 15326 consensus_queue.cc:237] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:51.272727 15332 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:04:51.273373 14802 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.303798 14802 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:51.310881  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 44869.6us
I20260709 13:04:51.311527 14540 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:51.314805 15284 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.239388 15346 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:04:51.314807 15341 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:51.315996 15346 consensus_queue.cc:1243] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:51.275089  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.283597  8823 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:51.317247 15105 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:04:51.264194 15213 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.786s	user 0.011s	sys 0.002s
I20260709 13:04:51.298656  8749 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 17: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:51.315475 15284 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.318341 15213 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.321516 15284 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:51.321589 15213 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:51.317982 15105 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:51.292469  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 45490.7us
I20260709 13:04:51.279053 15222 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.317166 15359 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:51.282572 15294 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.299342 15349 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.256099 15157 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 13:04:51.306516 14802 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:51.409898 15361 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.410363 14802 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.410751 15361 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.411278 15360 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:51.411137 14802 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.413364 15349 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:51.298717 15245 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:51.325591 15105 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.415009 15354 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.415372 15294 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:51.415619 15354 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.416118 15157 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:51.416579 15157 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:51.281926 15256 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.293706 15135 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:50.631875  8821 rpcz_store.cc:276] 0709 13:04:48.186944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.867297 (+1680353us) service_pool.cc:224] Handling call
0709 13:04:49.867772 (+   475us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:49.868212 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.868246 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:49.868934 (+   688us) log.cc:844] Serialized 52 byte log entry
0709 13:04:50.257060 (+388126us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:50.580611 (+323551us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.604820 (+ 24209us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.605135 (+   315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.605254 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.605315 (+    61us) raft_consensus.cc:1685] finished
0709 13:04:50.605334 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.605506 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23573,"raft.queue_time_us":10235,"raft.run_cpu_time_us":493,"raft.run_wall_time_us":29295,"thread_start_us":620043,"threads_started":2,"wal-append.queue_time_us":498203}
I20260709 13:04:51.411870 15360 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.422562 14829 raft_consensus.cc:3055] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:51.424006 14829 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:51.425154 14829 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.431582 15251 raft_consensus.cc:3055] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:51.432276 15251 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:51.433408 15251 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:51.437592 15339 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.423s	user 0.000s	sys 0.001s
I20260709 13:04:51.438320 15279 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:51.443691  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:51.444571  8827 raft_consensus.cc:2376] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15. Re-sending same reply.
W20260709 13:04:51.445147  8827 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 1363) took 1377 ms (client timeout 1560 ms). Trace:
W20260709 13:04:51.445549  8827 rpcz_store.cc:269] 0709 13:04:50.067442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.070373 (+  2931us) service_pool.cc:224] Handling call
0709 13:04:51.443492 (+1373119us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:04:51.445072 (+  1580us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23041920}
I20260709 13:04:51.417534 15157 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.453524 15157 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:51.463416  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 847983us
I20260709 13:04:51.463637 14767 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:51.470541  8822 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:51.473510  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:51.477685  8673 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:51.478386  8672 raft_consensus.cc:2393] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:51.414177 15361 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:51.414281 14802 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:51.348344 15359 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.485939 14802 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:51.475329  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:51.479579  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:51.488125  8679 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:04:51.482125 15205 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.482439  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:51.483621 15223 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:51.475467 15150 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.484241  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2544) took 1750 ms. Trace:
W20260709 13:04:51.483629  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 493327us
I20260709 13:04:51.508201  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:04:51.509336  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.494876  8827 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: [15.11-17.12]
W20260709 13:04:51.497730  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.484019 15360 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:51.486558 14802 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.476328 15220 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:51.463449 15204 raft_consensus.cc:3055] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:04:51.513113  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.504848  8829 rpcz_store.cc:276] 0709 13:04:49.733440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.524356 (+790916us) service_pool.cc:224] Handling call
0709 13:04:50.558602 (+ 34246us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.578956 (+ 20354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.578991 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.591515 (+ 12524us) log.cc:844] Serialized 26 byte log entry
0709 13:04:51.020431 (+428916us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:51.118717 (+ 98286us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.483866 (+365149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.483955 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.484003 (+    48us) raft_consensus.cc:1685] finished
0709 13:04:51.484021 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.484171 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":392100,"prepare.queue_time_us":104215,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":4218,"raft.queue_time_us":396505,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":20721,"thread_start_us":312947,"threads_started":1,"wal-append.queue_time_us":548061}
I20260709 13:04:51.480388 15251 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:51.517076 15279 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:51.502882  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:51.504006 15147 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.509s	user 0.014s	sys 0.000s
I20260709 13:04:51.485975 15143 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:51.534566  8722 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:51.489835 15217 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:51.493114 14540 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:51.539546 14802 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:51.540405  8679 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:51.529529 15204 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:51.532469 15303 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:51.496080 15205 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.551402 15359 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:51.551921 15340 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:51.553058 15351 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:51.553958 15133 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:51.498833 14829 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:51.554514 15206 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:51.608441 15206 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:04:51.556711  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.559616  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1358) took 1897 ms. Trace:
I20260709 13:04:51.612263  8753 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:04:51.612182  8823 rpcz_store.cc:276] 0709 13:04:49.662362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.466142 (+803780us) service_pool.cc:224] Handling call
0709 13:04:50.466536 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.317232 (+850696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.559272 (+242040us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b54001946d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:51.559536 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1878535936}
W20260709 13:04:51.567210  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.566751 15217 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.616139 15217 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:51.609835 15206 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:51.619989 15323 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.921s	user 0.004s	sys 0.000s
W20260709 13:04:51.621521  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.624680 15269 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:51.567497 15231 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:04:51.569100  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 266052us
I20260709 13:04:51.569859  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:51.537221 14936 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:51.525934 15150 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.513698 15223 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.567677 15204 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.582692 15303 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:51.584642 15205 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:51.562337 15279 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.563583 15147 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.674870 15262 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:51.675853 15262 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.677085 15147 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:51.678404  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:51.565577 15143 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.587280  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:51.679834 15262 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:51.680256  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 93056.8us
I20260709 13:04:51.681701 15143 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:51.588073 15174 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:51.581434 15245 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:04:51.589764  8828 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:51.684845 15150 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:51.597282 15340 consensus_queue.cc:1243] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:51.601388 15351 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.605525 15133 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.687599 15351 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:51.688112 15133 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:51.551522  8794 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:51.631976  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 62219.6us
I20260709 13:04:51.688671  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:51.689369  8677 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
W20260709 13:04:51.689453  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 138037us
I20260709 13:04:51.635185  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 825) took 1043 ms. Trace:
I20260709 13:04:51.690295  8794 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:51.636868  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1312) took 1484 ms. Trace:
I20260709 13:04:51.634186  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:51.641368 15268 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:51.641481 15231 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:51.691677  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:51.692556 15231 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:51.690901  8678 rpcz_store.cc:276] 0709 13:04:50.152980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.153256 (+   276us) service_pool.cc:224] Handling call
0709 13:04:50.153715 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.154137 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.154176 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.154778 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.365203 (+210425us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:50.365287 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.365364 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.619244 (+1253880us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:51.627514 (+  8270us) raft_consensus.cc:1685] finished
0709 13:04:51.627541 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.636773 (+  9232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3166,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":110,"spinlock_wait_cycles":378302848,"thread_start_us":208887,"threads_started":1,"wal-append.queue_time_us":458673}
I20260709 13:04:51.694188 15256 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:51.695071  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:51.645620 15201 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:51.695936 15178 raft_consensus.cc:3055] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:51.671480 15223 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:51.645026 15303 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.696581 15178 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:51.696758  8822 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:51.562650 15345 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.091s
I20260709 13:04:51.687608  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:51.607461  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1357) took 2085 ms. Trace:
I20260709 13:04:51.698654 15133 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:51.698769  8826 rpcz_store.cc:276] 0709 13:04:49.521946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.282442 (+760496us) service_pool.cc:224] Handling call
0709 13:04:50.712183 (+429741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.762906 (+ 50723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.607097 (+844191us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b540041d050. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:51.607368 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":987252608}
I20260709 13:04:51.699716  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:51.697417  8793 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:51.700589  8673 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:04:51.701632 15150 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:51.702050 15150 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:51.695155  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:51.702431 15150 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:51.703085  8825 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:51.702797  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:51.703984  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:51.704272  9388 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:04:51.699649  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:51.691345  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:51.707688  8676 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 11.
I20260709 13:04:51.696045 15279 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:51.695950  8828 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:51.690448  9384 rpcz_store.cc:276] 0709 13:04:50.592096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.592322 (+   226us) service_pool.cc:224] Handling call
0709 13:04:50.613220 (+ 20898us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.758879 (+145659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.634796 (+875917us) spinlock_profiling.cc:243] Waited 95.4 ms on lock 0x7b54001a4854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:51.635089 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219499776}
I20260709 13:04:51.707603  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:51.714897  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 80860.1us
I20260709 13:04:51.691310 15268 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:51.725935  8673 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:51.726830  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.682440 15143 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:51.750308  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:51.750100  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:51.751543  9388 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:51.755239  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:51.721184  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:51.759723  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:51.760012  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:51.700614 15133 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:04:51.761952  8719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.016s	sys 0.032s
W20260709 13:04:51.762459  8719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.016s	sys 0.032s
I20260709 13:04:51.766258  8722 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:51.765177  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 506151us
W20260709 13:04:51.768136  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 47059.5us
I20260709 13:04:51.774638  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:51.744544 15268 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:51.801469  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:51.806329  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:51.805120  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:51.806491  8828 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:51.746331 15223 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:51.808974  8796 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:51.812206  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:51.813159  8826 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:51.807143  9382 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
W20260709 13:04:51.815130  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 48960.2us
W20260709 13:04:51.817628  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.819831  9384 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:04:51.725355 15281 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:04:51.821837  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:51.822479  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 53805.6us
W20260709 13:04:51.844744  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.773862 15234 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:51.861177  8676 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:04:51.805262 15245 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:51.880373  8722 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:51.866505  8676 pending_rounds.cc:85] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 10
I20260709 13:04:51.864197  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:51.896173  8676 pending_rounds.cc:107] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:04:51.896512  8676 raft_consensus.cc:2889] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:51.860381  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:51.860041  8680 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:04:51.873564  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.873840  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.875667  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.898821  8756 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:04:51.898955  8754 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:04:51.877023  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.877764  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:51.890511 15366 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:51.901710  8828 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:04:51.892367 15188 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:51.904290  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.816937 15344 raft_consensus.cc:3055] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:51.813879 15175 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.874266  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.896389  8826 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:51.860118  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:51.908109  8673 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:51.908641  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.908802  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.901213  8755 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:51.871896  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.909650  8750 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:04:51.814124 15223 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.900518  8751 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:51.764519 15143 raft_consensus.cc:697] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:51.903136  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:51.812739 15205 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:51.892795 15363 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:51.908985  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:51.762931 15217 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:04:51.851400 15195 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:51.929872  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.940296  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:51.932461  8673 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:51.859983 15349 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.895617 15245 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.954052  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:51.898263 15326 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.030s	sys 0.048s
I20260709 13:04:51.955798  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:51.957242  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:51.959532  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:51.774345 15178 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.935252  8828 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:51.944638  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:51.950397  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:51.940791  8751 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:51.904353 15174 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:51.962008  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1361) took 2142 ms. Trace:
I20260709 13:04:51.967144  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:51.966055  8830 rpcz_store.cc:276] 0709 13:04:49.819376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.548880 (+729504us) service_pool.cc:224] Handling call
0709 13:04:50.569572 (+ 20692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.569845 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:51.079746 (+509901us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.961620 (+881874us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:51.961690 (+    70us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:51.961753 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.961833 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.961949 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":271062,"prepare.queue_time_us":117844,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":304080,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":7390,"thread_start_us":1100994,"threads_started":3,"wal-append.queue_time_us":530149}
I20260709 13:04:51.909992 15344 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:51.926803 15223 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:51.944862 15327 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.933s	user 0.012s	sys 0.012s
I20260709 13:04:51.968638  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:51.970249  9389 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:04:51.991786  8830 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:51.975956  8750 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:51.969797  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:51.984440  8673 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
W20260709 13:04:51.939965 15105 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.036s	sys 0.049s
W20260709 13:04:51.987282  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.974740  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:51.965646 15326 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.030s	sys 0.048s
I20260709 13:04:52.000535  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:52.004629  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:51.972306  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:52.007057  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:51.945783 15363 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.963959 15245 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:52.005859  8673 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:52.000641  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.013141  8830 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:52.015074  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:52.016126  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.017387  8830 raft_consensus.cc:2410] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:04:52.009892  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:51.913795 15188 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:51.945000 15369 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:51.998401 15168 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.930547 15175 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:52.039078  8823 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:04:52.023037  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:52.002951 15105 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.036s	sys 0.049s
I20260709 13:04:52.016906  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:52.034085  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:51.991106 15327 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.013s	sys 0.013s
I20260709 13:04:51.945717 15205 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.047466  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:52.048893 14802 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:52.049469 14802 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.049779 15205 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:52.049961 14802 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.004586 15279 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:52.051990 15279 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.052608 15279 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:52.052626  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:52.011659 15220 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:52.058357  8673 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:52.037629  8826 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
I20260709 13:04:51.965480 15349 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:52.037981  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:52.011821 15178 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:52.029723 15369 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.031735 15365 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:52.037223 15252 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.032034 15133 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:52.069871 15369 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:52.110921 15133 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.113919 15133 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:52.072773 15363 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:52.078289 15279 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.117400 15279 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:52.084337 15188 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:52.094192  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.097505 15175 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:52.118420 14802 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:52.123862  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:52.124568  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.058454 15220 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.125473 15220 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:52.100608 15150 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:52.129182 15252 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.132007 15150 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:52.132304 15204 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:52.043748 15168 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:52.135344 15168 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:52.142467  9388 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:52.145221 15172 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:52.147207  8747 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:52.148598 15349 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:52.068944 15143 consensus_queue.cc:237] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:52.058679 15344 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.117612 15354 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:52.136101 15245 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:52.139957 15205 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:52.139698 15369 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.140486 14802 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:52.140700 15107 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:52.141342 15216 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:52.141559  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:52.105334 15365 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:52.151311  8827 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.012s	sys 0.034s
I20260709 13:04:52.153946  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:52.061591  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:52.159471  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 396703us
W20260709 13:04:52.156241  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 389862us
W20260709 13:04:52.163501  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 204086us
I20260709 13:04:52.190284  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:52.191560  8673 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 11.
I20260709 13:04:52.190270  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:52.139742 15364 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.516s	user 0.003s	sys 0.000s
I20260709 13:04:52.192744  8679 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:04:52.170269  8827 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.012s	sys 0.034s
I20260709 13:04:52.132053 15252 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:52.193436  8722 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:52.200052  8830 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:52.199579  8826 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:04:52.200106  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 242704us
I20260709 13:04:52.195053  8673 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:52.216569  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.216524  8722 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:52.218448  8673 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:52.166131 15147 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:52.239001 15363 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:04:52.239821 15356 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.004s	sys 0.014s
W20260709 13:04:52.240272 15356 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.004s	sys 0.015s
I20260709 13:04:52.238024  8719 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:52.245018  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:52.248665  8719 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:52.226701  8828 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:52.233402  8826 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:52.166759 15360 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:52.253162  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 205778us
I20260709 13:04:52.199884 15107 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:52.251953  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.170078 15205 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.260766 15105 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:52.261329 15327 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:52.223964 15349 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:52.266113  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:52.272547  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.273808  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:52.272308  8826 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:52.278326  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:52.289100  8673 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:52.256503 15172 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:52.246238 15279 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:04:52.294790  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:52.296671  8673 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:52.298424  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:52.274632 15327 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.275513 15349 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:52.323827  8755 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 23.
I20260709 13:04:52.324965  8645 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:52.325084  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.272257 15105 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:52.327092  8826 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:52.326217  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.311007 15349 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:52.327113  8645 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:52.328297  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.278576 15192 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:04:52.372381  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:52.331777  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 358443us
W20260709 13:04:52.375267  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.331349  8826 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:52.320217 15133 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:52.319506 15327 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:52.378679  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:52.320092 15365 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:52.337056 15105 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.388873  8826 raft_consensus.cc:2376] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19. Re-sending same reply.
I20260709 13:04:52.398353  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.336748 15262 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:52.370397 15280 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.663s	user 0.012s	sys 0.012s
I20260709 13:04:52.379792 15188 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.411504  8673 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
W20260709 13:04:52.390028 15342 log.cc:927] Time spent T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:04:52.419322  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:52.356824 15390 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:04:52.385156 15133 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.437541  8753 raft_consensus.cc:606] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:52.391019 15172 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:04:52.404125 15379 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:04:52.407248 15262 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:04:52.421643  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:52.466974  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:52.467454  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.388705 15256 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:52.442179  8793 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:52.427549  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:52.468873  8826 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:52.416308 15268 consensus_meta.cc:287] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.593s	user 0.013s	sys 0.000s
W20260709 13:04:52.414964 15280 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.708s	user 0.013s	sys 0.012s
I20260709 13:04:52.436419  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:52.470777  8673 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:52.384585 15391 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.081s
W20260709 13:04:52.472729  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 81138.7us
I20260709 13:04:52.436741 15392 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.438793 15354 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:52.440440 15133 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:52.459885 15279 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:52.470942 15262 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:52.470575 15245 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:52.443308 15365 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:52.522035 15365 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.524487 15365 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:52.472347 15256 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:52.525229 15256 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:52.525352  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:52.526510  8796 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:52.527870  8796 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:52.528589  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.529017 15295 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:04:52.529866 15380 log.cc:927] Time spent T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 13:04:52.530745 15295 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:52.513020 15386 log.cc:927] Time spent T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:04:52.532922  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:52.533274  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:52.533782  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:52.514930  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:52.514092 15394 log.cc:927] Time spent T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:04:52.515249 15387 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
W20260709 13:04:52.516299 15389 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:04:52.518989 15222 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:52.517602 15341 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:52.537213  8822 raft_consensus.cc:1217] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Deduplicated request from leader. Original: 16.10->[18.11-18.11]   Dedup: 18.11->[]
I20260709 13:04:52.537933  8822 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:04:52.537442 15341 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:52.539006  8647 leader_election.cc:336] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.575s (SENT)
I20260709 13:04:52.534014  8677 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 17: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:52.540127 15341 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.530580  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:52.537487  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.539613 15222 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:52.537129  8678 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:04:52.544755  8678 pending_rounds.cc:85] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 11
I20260709 13:04:52.545061  8678 pending_rounds.cc:107] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:04:52.545313  8678 raft_consensus.cc:2889] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:52.543017  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:52.531356 15213 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:52.500208 15381 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 13:04:52.496330 15392 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:52.549794  8647 leader_election.cc:336] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.588s (SENT)
I20260709 13:04:52.487637 15369 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:52.534245  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:52.580932  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:52.529328  8823 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:52.475057 15252 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:52.470170 15268 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.556875  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:52.597925  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:52.598573  8755 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:52.440116 15188 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.488179 15353 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.557814  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:52.563918 15400 log.cc:927] Time spent T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:04:52.565011 15205 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:52.567137  9385 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:52.568672  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:52.501466 14771 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.613081  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:52.501854 15354 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:52.551395  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:52.576818  8826 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:52.617239  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.583027  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:52.584316  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:52.584971  8754 pending_rounds.cc:85] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 9
I20260709 13:04:52.591285  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:52.503875 15279 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.555135  8673 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 12.
I20260709 13:04:52.632035  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:52.603556  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:52.611382  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 61628.5us
I20260709 13:04:52.607769 15135 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:52.634572  8754 pending_rounds.cc:107] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:04:52.636427  8823 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:52.639637  8754 raft_consensus.cc:2889] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:52.639760  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:52.630689  8755 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:04:52.636606  8822 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:04:52.636991  8647 leader_election.cc:336] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.614s (SENT)
W20260709 13:04:52.624464  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.640797  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:52.641875  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.643015  8749 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:52.568372 15175 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:04:52.642980  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:52.611806 15353 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:52.636287  8748 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:52.620786  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:52.647066  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.648976  9384 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.652245  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:52.652993  8749 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:52.653441  8647 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.644s (SENT)
I20260709 13:04:52.653930  8647 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:52.655788  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:52.656555  8647 leader_election.cc:336] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.712s (SENT)
I20260709 13:04:52.657092  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:52.608912 15252 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:52.618544 15222 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:52.658510  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:52.653755  8826 raft_consensus.cc:2393] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:52.657227  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:52.669754  8748 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:52.670558  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:52.666302  8823 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
W20260709 13:04:52.668612  8647 leader_election.cc:336] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.758s (SENT)
I20260709 13:04:52.671798  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:52.660977 15135 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:52.674206  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.676246  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.677168  8823 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 17: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:52.678931  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:52.679700  8647 leader_election.cc:336] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.776s (SENT)
I20260709 13:04:52.680192  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:52.680125  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.648648 15245 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:52.672145  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:52.654909 15213 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:52.684512  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.685678  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.695612  8823 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:52.696947  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.683687  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:52.697355  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:52.697803  8823 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:52.698383  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:52.698558  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.699805  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:52.700103  8796 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:52.635118 14771 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:52.667815 15362 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:52.702355  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:52.705940  8749 raft_consensus.cc:2410] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:04:52.600062 15172 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:52.712600  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.716449  8826 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
I20260709 13:04:52.717224  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:52.717679  8826 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:52.685254 15178 consensus_meta.cc:287] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.618s	user 0.013s	sys 0.000s
I20260709 13:04:52.719797  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:52.721386  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:52.722193  8673 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:04:52.713387  8645 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:52.724313  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:52.724972  8823 raft_consensus.cc:2376] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17. Re-sending same reply.
I20260709 13:04:52.737687  9389 raft_consensus.cc:2410] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:04:52.739569  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:52.740836  8826 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 23.
I20260709 13:04:52.770926  8749 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:04:52.771692  8748 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:52.701831 15135 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:52.774358  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.775027  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:52.776022  8748 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:52.775688  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.779803  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.780606  8826 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:52.712189 15372 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:52.781448  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.715178 15213 raft_consensus.cc:697] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:52.787406  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.802548 15411 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:52.795342  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 69108.1us
W20260709 13:04:52.795594  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 56120us
I20260709 13:04:52.815150  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.817191  8826 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:52.793622  8673 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:04:52.700238 15245 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:52.815920  8748 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:52.821811  8748 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:04:52.812433  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 99195.4us
I20260709 13:04:52.823031  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:52.822237  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.812071  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:52.835465  8722 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:52.836639  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:52.837646 15406 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.838930 15410 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.839087 15406 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:52.839511 15410 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.841248 15404 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:52.841969 15374 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:52.842310 15223 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.837s	user 0.007s	sys 0.003s
I20260709 13:04:52.811893 15135 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:52.831753 15220 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.706s	user 0.015s	sys 0.000s
I20260709 13:04:52.826606 15411 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:52.853353 15409 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:04:52.860796 15413 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:04:52.861016 15409 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:52.861557 15409 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:52.853911 15408 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:52.854727 15220 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.855428 15343 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:52.855701 15105 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:52.856669 15349 consensus_meta.cc:287] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.000s
I20260709 13:04:52.832780 15403 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:52.842562 15374 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.858927 15375 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:52.870313 15424 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:52.868736 15404 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:52.852317 15281 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:52.868224  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:52.796072 15252 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:52.879024  9386 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:52.878518  9385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 829) took 1176 ms. Trace:
I20260709 13:04:52.871361  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:52.869227  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:52.870246  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:52.870848 15405 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:52.894145  9385 rpcz_store.cc:276] 0709 13:04:51.701648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.703650 (+  2002us) service_pool.cc:224] Handling call
0709 13:04:52.878253 (+1174603us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b5400557154. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:52.878406 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1983291648}
I20260709 13:04:52.846881 15280 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:52.979569  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.985770  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:53.001928  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:53.002127 15105 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:52.895073  9382 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:53.033926  9381 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 17: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:53.035611 15403 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.036569 15403 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.904676  9384 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:52.850809 15135 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.039762 15280 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:53.039847  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:53.041213 15280 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.901304 15174 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:52.901434 15188 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:52.879837 15189 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:53.044281  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:53.044636 15189 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:53.045492  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.884155 15213 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:53.046571  9385 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:53.046914  8672 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
W20260709 13:04:53.049326  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.049091  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:52.910898 15245 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:52.849906 15223 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:53.051280  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:53.050868 15189 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.052399  8672 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:53.053108 15189 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:53.054015  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.917408 15256 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:52.901887 15281 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.903046 15279 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:52.922168 15353 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:52.900235 15349 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:53.058207 15256 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:52.923707 15374 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:53.100821  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2556) took 1550 ms. Trace:
I20260709 13:04:52.906314 15375 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:53.103986  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:53.104592 15375 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:53.106920  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:52.899782 15401 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:04:52.888012 15404 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.114094 15404 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:52.911648 15405 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:52.896356 15406 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:52.941917 15409 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:52.874430 15410 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:52.865536 15411 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:52.916003 15414 log.cc:927] Time spent T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:52.918793 15419 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.152040 14936 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:52.900156 15424 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:53.152900 14936 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:52.896840  8794 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:53.013547 15295 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:53.019026 15369 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:53.012207 15205 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:53.159377 15205 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.161566  8751 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:53.009049 15216 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.821s	user 0.011s	sys 0.005s
I20260709 13:04:53.010697  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:53.158473 15369 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.164268  8823 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:53.166008  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:53.019208  8752 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:04:53.017475 15390 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:53.166386 15369 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:53.004839  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:53.048224 15403 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:52.937481 15327 consensus_meta.cc:287] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.534s	user 0.012s	sys 0.000s
I20260709 13:04:53.060600  8674 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:53.059688  8828 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:53.058116 15281 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:53.081395  8754 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:53.083369 15135 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:53.060477  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:53.054347 15223 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:53.026513 15133 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.508s	user 0.009s	sys 0.003s
I20260709 13:04:53.061034  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:53.096524 15349 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:53.097939  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:53.187166  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 290260us
I20260709 13:04:53.101511  8824 rpcz_store.cc:276] 0709 13:04:51.550101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.558536 (+  8435us) service_pool.cc:224] Handling call
0709 13:04:51.558881 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.680355 (+121474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:51.680390 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:51.680441 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:51.680475 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.100101 (+1419626us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540016ba54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:53.100183 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.100534 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278822144}
I20260709 13:04:53.189574  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:53.105100  8825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2536) took 1411 ms (client timeout 1829 ms). Trace:
W20260709 13:04:53.190866  8825 rpcz_store.cc:269] 0709 13:04:51.694024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.103340 (+1409316us) service_pool.cc:224] Handling call
0709 13:04:53.105003 (+  1663us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.191553 15421 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:53.192572  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:53.193543  8825 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:53.112567  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:53.155025 14936 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.120520 15280 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:53.126930  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.139829  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2707) took 1220 ms. Trace:
I20260709 13:04:53.147198  8826 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:53.123101 15405 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.199440  8679 rpcz_store.cc:276] 0709 13:04:51.895714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.207983 (+312269us) service_pool.cc:224] Handling call
0709 13:04:52.208370 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:52.208749 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:52.208781 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:52.221172 (+ 12391us) log.cc:844] Serialized 26 byte log entry
0709 13:04:52.481785 (+260613us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:52.481828 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.481901 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:53.115245 (+633344us) raft_consensus.cc:1685] finished
0709 13:04:53.115302 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.115590 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6581,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":4602,"thread_start_us":208161,"threads_started":1,"wal-append.queue_time_us":361691}
I20260709 13:04:53.205621  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:53.155157 15424 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:53.125615  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 82045.7us
I20260709 13:04:53.173867 15409 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:53.174978 15205 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:53.183230 15220 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:53.212116 15360 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:53.212920 15360 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:53.215257  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.215703  8719 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:53.183480 15133 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:53.220218  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:53.220741 15392 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:53.222250  8822 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:53.182518  8676 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:04:53.098922  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:53.175868 15327 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:53.230373  8824 raft_consensus.cc:2410] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:04:53.114888  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:53.189072  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:53.167380 15390 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.187443  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:53.201931  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 389802us
I20260709 13:04:53.233624  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2558) took 1135 ms. Trace:
I20260709 13:04:53.241405  9382 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:04:53.237548  8823 rpcz_store.cc:276] 0709 13:04:52.094942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.165503 (+1070561us) service_pool.cc:224] Handling call
0709 13:04:53.229837 (+ 64334us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.225107 15133 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:53.152520 15419 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:53.248358  8821 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:53.249063  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:53.250349  9383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:58628 (request call id 11) took 1596 ms. Trace:
I20260709 13:04:53.250771 15354 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:53.251309  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:53.251559 15349 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:53.253618  8749 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:53.250808  9383 rpcz_store.cc:276] 0709 13:04:51.654128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.667972 (+ 13844us) service_pool.cc:224] Handling call
0709 13:04:53.249752 (+1581780us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54003a6454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24ff735 00007f32e24f79fa 00007f32e74bf4c9 00007f32e0a3bc54 00007f32e0a3bbe2 00007f32e0a3bb11 00007f32e0a3ba8c 00007f32e0a3ad42 00007f32e083448c 00007f32e08338c6
0709 13:04:53.250122 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":981809536}
I20260709 13:04:53.255434  8747 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:53.214082 15269 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
W20260709 13:04:53.257491  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:53.259347  8566 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7aff04f53d4845a6a4fa491fd2a06145 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:53.262629 15327 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:53.263383 15345 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.262897  8750 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 12: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:53.264169 15345 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.271214 15390 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:53.272130 15345 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:53.272895  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:53.274873  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:53.276170 15143 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:53.276459  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 87862.5us
I20260709 13:04:53.274997 15430 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:53.278136  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:53.252246 15349 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.285247 15392 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:53.285748 15143 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:53.282145 15419 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:53.158473 15295 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.301882 15143 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.302820 15295 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:53.286878 15430 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.309163  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45676 (request call id 25) took 1702 ms. Trace:
I20260709 13:04:53.310015  8672 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:53.294008 15349 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:53.315326  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:53.322368  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:53.324234 15149 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:53.345826 15149 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:53.332636  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2563) took 1561 ms. Trace:
I20260709 13:04:53.347270  8830 rpcz_store.cc:276] 0709 13:04:51.770684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.022650 (+251966us) service_pool.cc:224] Handling call
0709 13:04:53.332219 (+1309569us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b540009d8d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:53.332429 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313170304}
I20260709 13:04:53.347970  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:53.348475  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:53.349267  8830 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:53.346179  8647 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:53.324028  8755 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:53.313884  9386 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 931) took 1570 ms. Trace:
I20260709 13:04:53.312206  9389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 1165) took 1174 ms. Trace:
I20260709 13:04:53.371361  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:53.387183 14771 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:53.350709 15149 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:53.328650  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:53.326242 15147 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:53.329362  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:53.350247  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:53.394996  8826 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:53.330248  8672 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:53.394027  9389 rpcz_store.cc:276] 0709 13:04:52.137642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.200092 (+ 62450us) service_pool.cc:224] Handling call
0709 13:04:53.311758 (+1111666us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b5400018bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:53.311987 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1130176256}
W20260709 13:04:53.415886 15262 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.868s	user 0.008s	sys 0.020s
I20260709 13:04:53.441246  8821 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:53.441874  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2535) took 1743 ms. Trace:
I20260709 13:04:53.442245  8826 rpcz_store.cc:276] 0709 13:04:51.693774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.276469 (+1582695us) service_pool.cc:224] Handling call
0709 13:04:53.288737 (+ 12268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.436663 (+147926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.436865 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.393661  9386 rpcz_store.cc:276] 0709 13:04:51.743429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.743629 (+   200us) service_pool.cc:224] Handling call
0709 13:04:52.878449 (+1134820us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.893689 (+ 15240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.313458 (+419769us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400557150. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:53.313644 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2582507264}
I20260709 13:04:53.310585 15430 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:53.447870  8749 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:53.309729  8753 rpcz_store.cc:276] 0709 13:04:51.607075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.612073 (+  4998us) service_pool.cc:224] Handling call
0709 13:04:53.308745 (+1696672us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b54001581d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f3853 00007f32e74c160a 00007f32e0a477d4 00007f32e0a47762 00007f32e0a47691 00007f32e0a4760c 00007f32e0a468c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:53.308906 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1896301824}
I20260709 13:04:53.417354 15410 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:53.417812  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 89384.8us
I20260709 13:04:53.449820  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2564) took 1673 ms. Trace:
W20260709 13:04:53.453555  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 49633.4us
I20260709 13:04:53.455184  8823 rpcz_store.cc:276] 0709 13:04:51.771012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.296227 (+1525215us) service_pool.cc:224] Handling call
0709 13:04:53.444515 (+148288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.458927  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:53.463059 15431 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.290s	user 0.000s	sys 0.005s
I20260709 13:04:53.463478  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:53.464233  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:53.464428  8821 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
W20260709 13:04:53.465893  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 119875us
I20260709 13:04:53.458953  9387 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:04:53.467304  9355 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:53.467655  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:53.468375  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:53.469879  8821 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 17. Ops: []
I20260709 13:04:53.471045  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2538) took 1776 ms. Trace:
I20260709 13:04:53.471483  8751 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:53.471501  8821 rpcz_store.cc:276] 0709 13:04:51.694579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.468999 (+1774420us) service_pool.cc:224] Handling call
0709 13:04:53.469478 (+   479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.470724 (+  1246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.470938 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.473791  8821 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 17. Ops: []
I20260709 13:04:53.474411  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2539) took 1764 ms. Trace:
W20260709 13:04:53.477027  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 147942us
I20260709 13:04:53.459056  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:53.479902  8821 rpcz_store.cc:276] 0709 13:04:51.709855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.472946 (+1763091us) service_pool.cc:224] Handling call
0709 13:04:53.473393 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.474176 (+   783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.474356 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.460412  8719 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:53.470328  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:53.481698  8830 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:53.486078  8680 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:53.486775  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:53.489231  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:53.481762 15345 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:53.469942 15201 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.461807 15133 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:53.496047  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.493727  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:53.502151  8679 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:53.504518  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:53.500305  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 44056.6us
I20260709 13:04:53.452991 15262 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:53.519218  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.447181 15147 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:53.524868  9355 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:53.527973  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:53.507822 15280 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:53.495136 15268 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:53.426882 15404 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:53.530218  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:53.537954  8821 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:53.516625 15149 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:53.545581  8826 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:04:53.546008  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2540) took 1830 ms. Trace:
I20260709 13:04:53.548851  8821 rpcz_store.cc:276] 0709 13:04:51.710139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.483605 (+1773466us) service_pool.cc:224] Handling call
0709 13:04:53.490577 (+  6972us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.538499 (+ 47922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.540952 (+  2453us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.520273 15201 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.544399 15205 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:53.536326 15280 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:53.545006 15433 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.067s	user 0.003s	sys 0.000s
I20260709 13:04:53.569368  8676 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:53.563915  8823 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:53.539578 15430 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:53.573195  8680 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 12: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:53.575703  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:53.562968 15280 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:53.579387  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 109247us
W20260709 13:04:53.583168  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 63576us
I20260709 13:04:53.599984  8677 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:53.600471 15262 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:53.601049  8678 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:53.601349  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:53.602655  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 85801.2us
I20260709 13:04:53.604990  8826 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:04:53.605786  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2560) took 1836 ms. Trace:
I20260709 13:04:53.617616  8830 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 16. Ops: []
I20260709 13:04:53.606166  8826 rpcz_store.cc:276] 0709 13:04:51.769780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.566250 (+1796470us) service_pool.cc:224] Handling call
0709 13:04:53.604443 (+ 38193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.605457 (+  1014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.605710 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:53.621462  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 56304.7us
I20260709 13:04:53.621117  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:53.630784 15248 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:53.631374 15248 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.631764 15248 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:53.573165 15411 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.623641  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2541) took 1908 ms. Trace:
I20260709 13:04:53.637440  8830 rpcz_store.cc:276] 0709 13:04:51.710391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.523649 (+1813258us) service_pool.cc:224] Handling call
0709 13:04:53.571832 (+ 48183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.618248 (+ 46416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.618486 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.642416 15172 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:53.636703  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:53.650985  8672 raft_consensus.cc:1275] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:04:53.652949 15172 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:53.653460 15172 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:53.653890 15172 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:53.654680  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 161127us
I20260709 13:04:53.656113 14771 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:53.656597 14771 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:53.656967 14771 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:53.594010 15410 consensus_queue.cc:1059] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:04:53.659343 15410 raft_consensus.cc:3055] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:53.659806 15410 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:53.662357 15416 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.855s	user 0.000s	sys 0.002s
W20260709 13:04:53.665508  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.670171  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3048) took 1077 ms. Trace:
I20260709 13:04:53.685885  9385 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:53.691901 15411 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:53.695286 15404 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:53.698071 15437 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:04:53.700177  8830 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:53.700979  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2568) took 1927 ms. Trace:
I20260709 13:04:53.701314  8830 rpcz_store.cc:276] 0709 13:04:51.773100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.666910 (+1893810us) service_pool.cc:224] Handling call
0709 13:04:53.699704 (+ 32794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.700676 (+   972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.700918 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.703068 15065 consensus_queue.cc:1059] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:53.704018 15360 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:53.709403  8824 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:04:53.715982  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:53.661031 15410 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:53.737326  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 69503.5us
I20260709 13:04:53.738209 15268 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.739567  8748 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:53.743363 15265 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:53.743893  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 88261.7us
I20260709 13:04:53.746958  8823 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
W20260709 13:04:53.752754 15444 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:04:53.752735 15268 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.674451  8756 rpcz_store.cc:276] 0709 13:04:52.592408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.592571 (+   163us) service_pool.cc:224] Handling call
0709 13:04:52.593016 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:52.606484 (+ 13468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:52.606527 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:52.607184 (+   657us) log.cc:844] Serialized 26 byte log entry
0709 13:04:52.794314 (+187130us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:52.794381 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.794483 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:53.661422 (+866939us) spinlock_profiling.cc:243] Waited 8.86 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:53.664937 (+  3515us) raft_consensus.cc:1685] finished
0709 13:04:53.664960 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.669934 (+  4974us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11035,"raft.queue_time_us":15470,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":6982,"spinlock_wait_cycles":20382976,"thread_start_us":118873,"threads_started":1,"wal-append.queue_time_us":134098}
I20260709 13:04:53.757745  8794 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:53.758744 15143 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:53.758836 15265 raft_consensus.cc:697] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:53.756892 15268 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:53.744177 15222 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:53.704118 15065 consensus_queue.cc:1243] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:53.767596  8830 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:53.674091 15349 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:53.786160  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:53.786446  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2555) took 1771 ms. Trace:
I20260709 13:04:53.787039  8830 rpcz_store.cc:276] 0709 13:04:52.014704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.707508 (+1692804us) service_pool.cc:224] Handling call
0709 13:04:53.765514 (+ 58006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.786071 (+ 20557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.786365 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:53.793560  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 194312us
I20260709 13:04:53.794884  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:53.797191  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:53.797672 15430 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:53.802026  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 3053) took 1193 ms. Trace:
I20260709 13:04:53.802590  8750 rpcz_store.cc:276] 0709 13:04:52.608342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.612677 (+  4335us) service_pool.cc:224] Handling call
0709 13:04:53.801684 (+1189007us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b540034bf54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:53.801929 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1491132800}
W20260709 13:04:53.807114  8825 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.595s	user 0.016s	sys 0.000s
I20260709 13:04:53.808907  9385 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:04:53.809685  9385 pending_rounds.cc:85] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting all ops after (but not including) 11
I20260709 13:04:53.810420  8825 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:53.813730  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:53.821242  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2586) took 1300 ms. Trace:
I20260709 13:04:53.823068  8825 rpcz_store.cc:276] 0709 13:04:52.510754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.192066 (+681312us) service_pool.cc:224] Handling call
0709 13:04:53.811273 (+619207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.807543 15360 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:53.775282 15222 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:53.810032  9385 pending_rounds.cc:107] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:04:53.832178  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1369) took 1612 ms. Trace:
I20260709 13:04:53.843889  9385 raft_consensus.cc:2889] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:53.843922  8824 rpcz_store.cc:276] 0709 13:04:52.219568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.711327 (+1491759us) service_pool.cc:224] Handling call
0709 13:04:53.785286 (+ 73959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.818486 (+ 33200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.818691 (+   205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.818905 (+   214us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:53.829462 (+ 10557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.829731 (+   269us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.832106 (+  2375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.815783 15262 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:53.867290  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2570) took 1610 ms. Trace:
I20260709 13:04:53.853039 15222 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:53.872596  8823 rpcz_store.cc:276] 0709 13:04:52.256542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.768236 (+1511694us) service_pool.cc:224] Handling call
0709 13:04:53.808472 (+ 40236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.857838 (+ 49366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.857892 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.866912 (+  9020us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:53.866962 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.867028 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.867213 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24009}
W20260709 13:04:53.874300  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 116729us
I20260709 13:04:53.872121  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1283) took 2840 ms. Trace:
I20260709 13:04:53.879209  8824 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:04:53.880020  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1371) took 1336 ms. Trace:
I20260709 13:04:53.880378  8824 rpcz_store.cc:276] 0709 13:04:52.543880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.856366 (+1312486us) service_pool.cc:224] Handling call
0709 13:04:53.878922 (+ 22556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.879745 (+   823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.879963 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.878552  8752 rpcz_store.cc:276] 0709 13:04:51.032115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.033943 (+  1828us) service_pool.cc:224] Handling call
0709 13:04:51.034373 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.167048 (+2132675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.871681 (+704633us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b54001040d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:53.872008 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4562140928}
I20260709 13:04:53.892534 15340 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:53.893308 15445 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.068s	user 0.001s	sys 0.001s
I20260709 13:04:53.893424 15340 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.831666 15248 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.867586 15327 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:53.911158 15436 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.493s	user 0.000s	sys 0.003s
I20260709 13:04:53.911619 15341 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:53.873905 15383 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.905820 15248 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:53.879169 15295 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:53.902283  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 105224us
W20260709 13:04:53.920243  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 142991us
I20260709 13:04:53.831956 15265 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.928068  8794 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:53.935354 15397 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.455s	user 0.000s	sys 0.004s
I20260709 13:04:53.934440 15295 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.943281 15349 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:53.947146 15143 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.949469 15143 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.952162 15143 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:53.952312 15378 log.cc:927] Time spent T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 1.564s	user 0.006s	sys 0.000s
I20260709 13:04:53.947423  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2719) took 1235 ms. Trace:
I20260709 13:04:53.953264  8675 rpcz_store.cc:276] 0709 13:04:52.712356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.712639 (+   283us) service_pool.cc:224] Handling call
0709 13:04:52.772106 (+ 59467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.981901 (+209795us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:52.981972 (+    71us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:52.982052 (+    80us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:52.982113 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.946945 (+964832us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b54001fa254. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:53.946993 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.947181 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480469760}
I20260709 13:04:53.956713  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2554) took 2407 ms. Trace:
I20260709 13:04:53.957240  8827 rpcz_store.cc:276] 0709 13:04:51.549486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.559537 (+ 10051us) service_pool.cc:224] Handling call
0709 13:04:51.559943 (+   406us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:51.560605 (+   662us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:51.561019 (+   414us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.622347 (+ 61328us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b1800006220. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:52.207645 (+585298us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:52.208398 (+   753us) log.cc:844] Serialized 78 byte log entry
0709 13:04:52.208578 (+   180us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:52.208839 (+   261us) log.cc:844] Serialized 14 byte log entry
0709 13:04:52.221294 (+ 12455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.221384 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:53.951266 (+1729882us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:53.956233 (+  4967us) raft_consensus.cc:1685] finished
0709 13:04:53.956257 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.956478 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8642,"prepare.run_cpu_time_us":145,"prepare.run_wall_time_us":8877,"raft.queue_time_us":41458,"raft.run_cpu_time_us":476,"raft.run_wall_time_us":26155,"spinlock_wait_cycles":29435520,"thread_start_us":646111,"threads_started":1,"wal-append.queue_time_us":733388}
W20260709 13:04:53.967790  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 66649.8us
I20260709 13:04:53.970393 15390 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:53.970667  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:53.971069 15390 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:53.971670 15205 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:53.949551 15383 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.972177  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2579) took 1301 ms. Trace:
I20260709 13:04:53.972306 15390 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:53.972703  8823 rpcz_store.cc:276] 0709 13:04:52.671020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.899482 (+1228462us) service_pool.cc:224] Handling call
0709 13:04:53.971411 (+ 71929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.971781 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.971816 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.971886 (+    70us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:53.971917 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.971976 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.972107 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.973991 15390 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:53.977463  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1372) took 1335 ms. Trace:
I20260709 13:04:53.978370 15157 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:04:53.978883 15432 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:04:53.979054 15157 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:53.980396 15157 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:53.981907 15157 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:53.985671  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3001) took 1414 ms. Trace:
I20260709 13:04:53.987129 15284 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:53.988075 15376 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 1.734s	user 0.004s	sys 0.000s
I20260709 13:04:53.974391 15340 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:53.989817 15430 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:53.977545  9381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 940) took 1297 ms. Trace:
I20260709 13:04:53.990690  9381 rpcz_store.cc:276] 0709 13:04:52.679461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.871057 (+191596us) service_pool.cc:224] Handling call
0709 13:04:53.977029 (+1105972us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b54001a4854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:53.977274 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57907968}
I20260709 13:04:53.991760  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 2996) took 1704 ms. Trace:
I20260709 13:04:53.978117  8824 rpcz_store.cc:276] 0709 13:04:52.642490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.891432 (+1248942us) service_pool.cc:224] Handling call
0709 13:04:53.976380 (+ 84948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.976734 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.976771 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.976854 (+    83us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:53.976907 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.976988 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.977371 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.992374  8677 rpcz_store.cc:276] 0709 13:04:52.287844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.540905 (+253061us) service_pool.cc:224] Handling call
0709 13:04:52.541436 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.600668 (+1059232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.991435 (+390767us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b540033a9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:53.991693 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2300232064}
W20260709 13:04:53.993119  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.989984  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2553) took 2440 ms. Trace:
I20260709 13:04:53.994118  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:53.993803  8829 rpcz_store.cc:276] 0709 13:04:51.549134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.554645 (+  5511us) service_pool.cc:224] Handling call
0709 13:04:51.555060 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:51.555487 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:51.555521 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:51.556107 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 13:04:51.945960 (+389853us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:51.946603 (+   643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.946859 (+   256us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:53.987158 (+2040299us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:53.989456 (+  2298us) raft_consensus.cc:1685] finished
0709 13:04:53.989480 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.989773 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":446,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":5258496,"thread_start_us":389105,"threads_started":1,"wal-append.queue_time_us":633928}
I20260709 13:04:53.995965  8752 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:04:53.997049  8752 raft_consensus.cc:606] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:53.986229  8674 rpcz_store.cc:276] 0709 13:04:52.570880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.590427 (+ 19547us) service_pool.cc:224] Handling call
0709 13:04:52.590821 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.175617 (+584796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.985341 (+809724us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b5400339fd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:53.985551 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079053568}
I20260709 13:04:54.001055  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:54.001605  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:53.972594 15205 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:04:53.974246 15383 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:54.003548  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:53.938469  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:54.008263  8830 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260709 13:04:54.016216  9387 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 821) took 3987 ms (client timeout 1776 ms). Trace:
I20260709 13:04:54.019142  8821 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:54.029048  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.029436  8752 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:04:54.004077  8678 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:04:54.006062  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:54.018693  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2584) took 1508 ms (client timeout 1675 ms). Trace:
W20260709 13:04:54.043382  9387 rpcz_store.cc:269] 0709 13:04:50.028815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.028960 (+   145us) service_pool.cc:224] Handling call
0709 13:04:54.015928 (+3986968us) spinlock_profiling.cc:243] Waited 3.13 s on lock 0x7b54000762d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:54.016120 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7206575360}
I20260709 13:04:54.021420  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3054) took 1406 ms. Trace:
I20260709 13:04:54.028434  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2580) took 1353 ms. Trace:
I20260709 13:04:54.053493  8829 rpcz_store.cc:276] 0709 13:04:52.675167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.026841 (+1351674us) service_pool.cc:224] Handling call
0709 13:04:54.027432 (+   591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.027937 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.027987 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.028047 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:54.028106 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.028179 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.028347 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.020095  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2724) took 1224 ms. Trace:
W20260709 13:04:54.007889  8828 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2570) took 1695 ms (client timeout 1709 ms). Trace:
W20260709 13:04:54.032085 15450 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.084s	user 0.002s	sys 0.000s
I20260709 13:04:54.035887  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2582) took 1211 ms. Trace:
I20260709 13:04:54.007858  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:54.059538  8827 rpcz_store.cc:276] 0709 13:04:52.824353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.034352 (+1209999us) service_pool.cc:224] Handling call
0709 13:04:54.034962 (+   610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.035472 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.035504 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.035565 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:54.035608 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.035667 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.035817 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.029882  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:54.063524  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.050631  8822 rpcz_store.cc:269] 0709 13:04:52.510129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.641510 (+131381us) service_pool.cc:224] Handling call
0709 13:04:54.018340 (+1376830us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b5400218ed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:54.018587 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1331331456}
I20260709 13:04:54.056003  8676 rpcz_store.cc:276] 0709 13:04:52.796209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.263573 (+467364us) service_pool.cc:224] Handling call
0709 13:04:53.264133 (+   560us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.572403 (+308270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.019704 (+447301us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b5400175a54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:54.020001 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":686608128}
I20260709 13:04:54.062700  9381 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
W20260709 13:04:54.059029  8828 rpcz_store.cc:269] 0709 13:04:52.312922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.377318 (+ 64396us) service_pool.cc:224] Handling call
0709 13:04:54.007613 (+1630295us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b54002189d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:54.007825 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1513203968}
W20260709 13:04:54.022264 15439 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.321s	user 0.000s	sys 0.003s
I20260709 13:04:54.052724  8748 rpcz_store.cc:276] 0709 13:04:52.615003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.824991 (+209988us) service_pool.cc:224] Handling call
0709 13:04:52.825376 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.752233 (+926857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.020982 (+268749us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b54002fe4d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:54.021336 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1968442112}
I20260709 13:04:54.061522  8752 raft_consensus.cc:606] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:54.037123 15438 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:04:54.073498  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2730) took 1093 ms. Trace:
W20260709 13:04:54.083719  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.081331  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2581) took 1401 ms. Trace:
I20260709 13:04:54.085008  8823 rpcz_store.cc:276] 0709 13:04:52.679756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.038430 (+1358674us) service_pool.cc:224] Handling call
0709 13:04:54.080540 (+ 42110us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.080906 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.080942 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.081000 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:54.081062 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.081125 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.081265 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.082219  8679 rpcz_store.cc:276] 0709 13:04:52.981217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.520600 (+539383us) service_pool.cc:224] Handling call
0709 13:04:53.561024 (+ 40424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.596079 (+ 35055us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:53.596125 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:53.633075 (+ 36950us) log.cc:844] Serialized 26 byte log entry
0709 13:04:53.866827 (+233752us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:53.866904 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.867023 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.031658 (+164635us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b2400195ea8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df2929c4 00007f32df290acb 00007f32df44bdc9 00007f32df44ba55 00007f32df44b09d 00007f32e207d10e 00007f32e209488c 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221
0709 13:04:54.073131 (+ 41473us) raft_consensus.cc:1685] finished
0709 13:04:54.073154 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.073420 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5920,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":7995,"spinlock_wait_cycles":70579584,"thread_start_us":111989,"threads_started":1,"wal-append.queue_time_us":246178}
I20260709 13:04:54.002563 15222 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:54.091063  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 152736us
I20260709 13:04:54.092935 15222 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:54.093904  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:54.094275 15366 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.094759 15430 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:54.094853 15366 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:54.095783 15430 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:54.096982 15430 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:54.097465  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:54.098850  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2571) took 1839 ms. Trace:
W20260709 13:04:54.099872  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.100629 15390 consensus_queue.cc:1048] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:54.021185  8573 catalog_manager.cc:5697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 7 to 12, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 12 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:54.104777 15410 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:54.108857  8828 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:04:54.109344  8828 pending_rounds.cc:85] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 10
I20260709 13:04:54.109623  8828 pending_rounds.cc:107] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:04:54.109871  8828 raft_consensus.cc:2889] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:54.109946  8647 connection.cc:256] RPC call timeout handler was delayed by -1.4014939699999331s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
I20260709 13:04:54.110824  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2585) took 1275 ms. Trace:
I20260709 13:04:54.111171  8828 rpcz_store.cc:276] 0709 13:04:52.835594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.072422 (+1236828us) service_pool.cc:224] Handling call
0709 13:04:54.072902 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.110517 (+ 37615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.110770 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.112579 15143 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:54.112389 15262 consensus_queue.cc:1059] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:54.113384  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:54.113637 15262 raft_consensus.cc:3055] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:54.114107 15262 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:54.108857  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2562) took 2338 ms. Trace:
W20260709 13:04:54.104501  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.116552  8821 rpcz_store.cc:276] 0709 13:04:51.770354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.566469 (+1796115us) service_pool.cc:224] Handling call
0709 13:04:54.018870 (+452401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.050996 (+ 32126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.108609 (+ 57613us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b5400218ed0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:54.108765 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1002427776}
I20260709 13:04:54.097640 15366 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:54.005102 15201 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:54.099262  8830 rpcz_store.cc:276] 0709 13:04:52.259697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.788027 (+1528330us) service_pool.cc:224] Handling call
0709 13:04:54.007975 (+219948us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:54.051373 (+ 43398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.098602 (+ 47229us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54002189d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:54.098767 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":504646912}
I20260709 13:04:54.009567 15295 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:53.925841 15147 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:54.132113  9355 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:54.058126  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:54.112419  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:54.126883  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 123445us
I20260709 13:04:54.138644  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:54.139452  9387 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:53.944301 15280 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:54.180347  8829 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:54.142825 15360 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:54.181866 15360 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:54.182878 15280 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:54.183652 15280 consensus_queue.cc:1059] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:54.184634 15280 raft_consensus.cc:3055] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:54.184984 15280 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:54.185201 15360 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:54.185827  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:54.187532  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:54.188817  8796 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:54.189327  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:54.148239  8821 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:54.162470  8827 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:54.148850  8676 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:54.149617  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:54.191264  9387 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:04:54.150959  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:54.150815 15349 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:54.151499  8828 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:54.148859  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:54.153168 15295 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:54.155508  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:54.195171  8824 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:54.159550  8681 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:54.160923  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:54.161628  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:54.168283 15172 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.003s
I20260709 13:04:54.167837  8823 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:54.163864  8822 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:54.171868 15262 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:54.066020 15105 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:04:54.173615  8675 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:54.174965 15147 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:54.175587 15404 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:54.181026 14771 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.524s	user 0.013s	sys 0.002s
W20260709 13:04:54.139456  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 81508.4us
I20260709 13:04:54.189267  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:54.212284  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:54.140846 15201 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:54.196703  8679 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:04:54.158422  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:54.203864  8674 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:54.213582  8722 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:54.204320  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2583) took 1379 ms. Trace:
I20260709 13:04:54.185997 15280 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:54.214274  8822 rpcz_store.cc:276] 0709 13:04:52.824648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.067117 (+1242469us) service_pool.cc:224] Handling call
0709 13:04:54.145474 (+ 78357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.203970 (+ 58496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.204245 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.195242 15349 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:54.222550  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:54.236637  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:54.240348  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:54.243849  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 130585us
I20260709 13:04:54.245105  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:54.216277 15201 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.249951 15201 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:54.249977 15360 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:54.250437 15133 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:54.252360 15133 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:54.206308 15172 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:54.255559 14936 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:54.256501  8749 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:54.257599 15172 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:54.257961 15262 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:54.258383  8681 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:04:54.259889  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:54.261368  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:54.262383  8647 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:54.213492 15411 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.264128  9381 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:54.270864 15372 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:54.271365 15372 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.238690 15105 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:54.271690  8748 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:54.271982 15105 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:54.271716 15372 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:54.272979 15147 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:54.273433 15147 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:54.273851 15147 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:54.231202 15349 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:54.282888  8676 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
W20260709 13:04:54.267288  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 109172us
I20260709 13:04:54.287045  8678 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:54.302676 15411 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:54.305377 15411 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:54.306560 15280 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:54.255640 15268 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.052s
W20260709 13:04:54.309440  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 47206.7us
I20260709 13:04:54.311286  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:54.313645  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:54.317978  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:54.316738  8573 catalog_manager.cc:5697] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 12 to 15, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 15 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:54.319653  8645 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:54.320031  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:54.322543  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:54.324786  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:54.326407 15344 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:54.327178  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:54.329807  8749 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:04:54.331459  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.331606  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 82258.7us
I20260709 13:04:54.324939  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:54.339872  9381 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:04:54.263093 15390 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:54.220503 14771 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:54.334182  9352 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:54.346853  8676 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:04:54.350414 15410 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:54.350924 15410 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.351303 15410 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:54.347874 15366 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:04:54.356818 15383 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:54.357362 15383 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:54.361354  8824 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:54.372040  9355 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:54.370791  8755 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:54.376986 15172 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:54.379072 14771 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:54.383162  8681 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:54.386296  9352 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:54.404378  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:54.407171 15198 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:04:54.407841  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.360538 15105 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:54.409221 15105 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:54.402721  9386 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:04:54.380235 15216 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:54.422917 15198 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:54.424167 15198 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:54.428777  8748 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:54.379963 15133 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:54.434945  8749 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:04:54.440418  8824 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:04:54.441490  8573 catalog_manager.cc:5697] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:54.362733 15383 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:54.434643 15216 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.447423 14786 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.454900  8796 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:54.463490  8680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.023s	sys 0.012s
W20260709 13:04:54.463891  8680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.024s	sys 0.012s
W20260709 13:04:54.478760  8826 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.011s	sys 0.019s
W20260709 13:04:54.479542  8826 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.011s	sys 0.019s
W20260709 13:04:54.488813  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.486058  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.456915 15216 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:54.530248  8677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.007s	sys 0.010s
W20260709 13:04:54.531051  8677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.008s	sys 0.010s
I20260709 13:04:54.522684 15295 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:54.534399 15295 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.534952 15295 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:54.535009 15201 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:54.522655 15265 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.011s	sys 0.023s
W20260709 13:04:54.543292 15265 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.012s	sys 0.023s
I20260709 13:04:54.543437 15245 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.453581 15147 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:54.569429  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.461790 14786 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:54.544425 15133 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:54.586486  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.597350  8678 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:54.578276 15233 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.539549 15372 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:54.598536 15217 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.183s	user 0.019s	sys 0.021s
I20260709 13:04:54.596459 15383 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:54.622081  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 370454us
I20260709 13:04:54.544134 15245 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:54.638693  8830 raft_consensus.cc:2393] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 20: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:54.640012  8722 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:54.641770  9388 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:54.642513 15340 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.643119 15340 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:54.618193 15217 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.230s	user 0.020s	sys 0.021s
W20260709 13:04:54.635913 15281 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.005s	sys 0.028s
I20260709 13:04:54.635480 15430 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:54.687759  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:54.688750  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.694422  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 374885us
I20260709 13:04:54.618130 15133 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:54.635087 15233 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:54.678005 15281 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.005s	sys 0.029s
I20260709 13:04:54.674661 15404 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:54.677798 15143 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.671077 15201 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:54.715276  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 75475.1us
I20260709 13:04:54.691269 15430 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.734908 15327 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:04:54.771185  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55632us
W20260709 13:04:54.735428  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 424173us
I20260709 13:04:54.749269  8673 raft_consensus.cc:2410] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:04:54.752594  8672 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:04:54.777765  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:54.783241  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2744) took 1449 ms. Trace:
I20260709 13:04:54.774544  8676 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:54.784680  8722 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:54.792991  8647 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:54.758124 15430 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:54.789297 15408 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.020s	sys 0.009s
I20260709 13:04:54.790392 15222 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:54.782091 15147 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:54.788576 15262 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:54.786738 15327 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:54.758957 15143 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:54.783841  8680 rpcz_store.cc:276] 0709 13:04:53.333569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.656960 (+323391us) service_pool.cc:224] Handling call
0709 13:04:53.657388 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.657772 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:53.868169 (+210397us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.782744 (+914575us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.782812 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:54.782869 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.783008 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.783163 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10975,"prepare.queue_time_us":268580,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"thread_start_us":1069197,"threads_started":3,"wal-append.queue_time_us":806097}
W20260709 13:04:54.830914 15360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.014s	sys 0.007s
W20260709 13:04:54.867597  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 83100.5us
W20260709 13:04:54.842226 15408 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.021s	sys 0.009s
I20260709 13:04:54.856107  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:54.861624  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:54.869232  8681 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:04:54.870997  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:54.871958  8756 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:04:54.876343  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:54.877139  9381 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:54.842134 15280 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:54.867815  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:54.874125 15457 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:54.871320  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:54.888343  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:54.889156  9386 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:54.869731 15360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.015s	sys 0.007s
I20260709 13:04:54.847265 15262 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:54.896518  9352 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:54.834033 15410 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:54.857064 14786 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:54.762392  8573 catalog_manager.cc:5697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:54.902595  8647 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.675s (SENT) [suppressed 24 similar messages]
I20260709 13:04:54.860920 14771 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:04:54.882401 15245 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:54.901525 15217 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:54.886801 15372 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:54.890182 15457 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:54.899324 15360 raft_consensus.cc:697] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:54.899689 15201 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:54.925354 15459 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:04:54.926074 15233 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:54.928851 15470 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:54.924642 15473 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:04:54.930855 15133 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:54.919930 15467 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:54.912982 15345 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:54.931373 15340 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:55.018574  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 130759us
W20260709 13:04:54.942401  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.880802  8571 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43478 (request call id 258) took 29037 ms (client timeout 15000 ms). Trace:
I20260709 13:04:54.907423 15262 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:54.915783 15452 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:04:54.933789  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:54.986907  9381 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
W20260709 13:04:54.995625  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 92490.2us
I20260709 13:04:54.904285 15280 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:54.949940 15217 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.983757 15466 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:54.994024 15372 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.031744  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:55.050395  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 175788us
I20260709 13:04:55.037514  8679 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:55.005365 15345 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.040339  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:55.055362  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 71070.5us
I20260709 13:04:55.005488 15467 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:55.035887  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:55.036221 15349 consensus_meta.cc:287] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.760s	user 0.014s	sys 0.000s
I20260709 13:04:55.054142  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:55.063779  8672 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:55.059078  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:55.041622 15455 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.224s	user 0.001s	sys 0.006s
I20260709 13:04:55.066960  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:55.068071  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:55.031724 15245 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.074568  8756 raft_consensus.cc:2393] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:55.036710 15340 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.075935  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:55.077898  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.046839 15217 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:55.084632  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:55.085599  8823 raft_consensus.cc:2393] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:55.068902  8830 raft_consensus.cc:2376] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16. Re-sending same reply.
I20260709 13:04:55.080703  8749 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:55.076750  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:55.087720  8679 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:55.096221 15456 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.288s	user 0.000s	sys 0.007s
I20260709 13:04:55.096570  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3002) took 1097 ms. Trace:
I20260709 13:04:55.097038  8677 rpcz_store.cc:276] 0709 13:04:53.999361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.999519 (+   158us) service_pool.cc:224] Handling call
0709 13:04:53.999980 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:54.000406 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.000436 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:54.002944 (+  2508us) log.cc:844] Serialized 26 byte log entry
0709 13:04:54.533420 (+530476us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:54.533486 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.533591 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:55.096311 (+562720us) raft_consensus.cc:1685] finished
0709 13:04:55.096335 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.096493 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1920,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"thread_start_us":529980,"threads_started":1,"wal-append.queue_time_us":708082}
I20260709 13:04:55.018805 15327 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.081331 15372 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.027140 15133 consensus_queue.cc:1059] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:04:55.071530 15466 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:55.054284 15465 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:55.060978 15345 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:55.091614 15340 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.063419 15467 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:55.098511  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2572) took 2836 ms. Trace:
W20260709 13:04:55.098814 15453 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
W20260709 13:04:55.098927 15472 log.cc:927] Time spent T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.051s	user 0.003s	sys 0.004s
W20260709 13:04:55.100646 15454 log.cc:927] Time spent T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.297s	user 0.001s	sys 0.001s
I20260709 13:04:55.074224 15349 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.152715  8753 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:55.091207  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:55.159720  8750 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
W20260709 13:04:55.160549 15105 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.751s	user 0.017s	sys 0.000s
I20260709 13:04:55.165004  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:55.166360  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:55.166568  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:55.166994  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:55.167179  8822 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:55.167980  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:55.168318  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:55.168649  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:55.167259  8821 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:55.104990 15372 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:55.170053  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:55.110993 15466 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:55.174841  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.121834  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:55.131269  8672 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:55.142983  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:55.143402  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:55.144695  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:55.175680  8824 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:55.178720  8821 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:55.168319  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:55.175125  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:55.180853  8829 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:55.180557  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:55.145752  8825 rpcz_store.cc:276] 0709 13:04:52.262344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.869606 (+1607262us) service_pool.cc:224] Handling call
0709 13:04:53.916564 (+ 46958us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:54.010806 (+ 94242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.010845 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:54.011478 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 13:04:54.467154 (+455676us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:54.467213 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.467314 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:55.098152 (+630838us) raft_consensus.cc:1685] finished
0709 13:04:55.098186 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.098419 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":128,"thread_start_us":455250,"threads_started":1,"wal-append.queue_time_us":623895}
I20260709 13:04:55.183916  8821 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:55.125900  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2566) took 3353 ms. Trace:
W20260709 13:04:55.188374  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 100148us
I20260709 13:04:55.138396  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:55.200663  9386 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:04:55.202875  8796 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:55.186975 15269 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.241118  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:55.241351 15269 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.186489  8679 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
W20260709 13:04:55.246789  8647 leader_election.cc:336] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.249s (SENT)
I20260709 13:04:55.247586  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:55.023583  8571 rpcz_store.cc:269] 0709 13:04:25.649553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.649826 (+   273us) service_pool.cc:224] Handling call
0709 13:04:54.686926 (+29037100us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:42.775766 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:42.776827 (+  1061us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:42.776894 (+    67us) write_op.cc:435] Acquired schema lock
0709 13:04:42.776933 (+    39us) tablet.cc:662] Decoding operations
0709 13:04:42.856521 (+ 79588us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:42.868263 (+ 11742us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:42.875694 (+  7431us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:04:42.925076 (+ 49382us) tablet.cc:701] Row locks acquired
0709 13:04:42.925140 (+    64us) write_op.cc:260] PREPARE: finished
0709 13:04:42.925390 (+   250us) write_op.cc:270] Start()
0709 13:04:42.925581 (+   191us) write_op.cc:276] Timestamp: P: 1783602282925301 usec, L: 0
0709 13:04:42.925643 (+    62us) op_driver.cc:348] REPLICATION: starting
0709 13:04:43.120437 (+194794us) log.cc:844] Serialized 7866 byte log entry
0709 13:04:43.716172 (+595735us) spinlock_profiling.cc:243] Waited 5.22 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:04:43.978988 (+262816us) op_driver.cc:464] REPLICATION: finished
0709 13:04:43.999399 (+ 20411us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32df3ef0ac 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e5f2fef0
0709 13:04:44.259339 (+259940us) write_op.cc:301] APPLY: starting
0709 13:04:44.259467 (+   128us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:44.829475 (+570008us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:44.829526 (+    51us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:45.375568 (+546042us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b18002ee8e0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e22380ec 00007f32e223aaf7 00007f32e223bfb0 00007f32e5f501bb 00007f32e5f975d4 00007f32e5e438da 00007f32e5e463cb 00007f32e5e45fd3 00007f32e5f44e34
0709 13:04:45.683605 (+308037us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:45.684227 (+   622us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 13:04:45.684320 (+    93us) write_op.cc:312] APPLY: finished
0709 13:04:45.684957 (+   637us) log.cc:844] Serialized 180 byte log entry
0709 13:04:45.906093 (+221136us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:45.949258 (+ 43165us) write_op.cc:454] Released schema lock
0709 13:04:46.023989 (+ 74731us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:46.024573 (+   584us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:46.024611 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":464511,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":279976,"apply.run_cpu_time_us":58125,"apply.run_wall_time_us":1765209,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87647,"num_ops":20,"prepare.queue_time_us":228346,"prepare.run_cpu_time_us":37545,"prepare.run_wall_time_us":502939,"raft.queue_time_us":204919,"raft.run_cpu_time_us":37185,"raft.run_wall_time_us":278980,"replication_time_us":1053145,"spinlock_wait_cycles":84387840,"thread_start_us":810751,"threads_started":5,"wal-append.queue_time_us":488596,"wal-append.run_cpu_time_us":21182,"wal-append.run_wall_time_us":2618867}]]}
I20260709 13:04:55.145076  8573 catalog_manager.cc:5697] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 11 to 13, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 13 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:55.135471 15410 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:55.118407 15295 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.583s	user 0.007s	sys 0.007s
W20260709 13:04:55.102381 15216 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.002s
I20260709 13:04:55.250547  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:55.264675  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:55.271188 15458 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.300s	user 0.001s	sys 0.001s
I20260709 13:04:55.276952  8821 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:55.181819 15340 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:55.187897  8826 rpcz_store.cc:276] 0709 13:04:51.772418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.651970 (+1879552us) service_pool.cc:224] Handling call
0709 13:04:53.727775 (+ 75805us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.763158 (+ 35383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.763195 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:53.833374 (+ 70179us) log.cc:844] Serialized 26 byte log entry
0709 13:04:54.480162 (+646788us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:54.835639 (+355477us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.984701 (+149062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.984804 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:55.125651 (+140847us) raft_consensus.cc:1685] finished
0709 13:04:55.125673 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.125830 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":485814,"prepare.run_wall_time_us":58,"raft.queue_time_us":152797,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":8335,"spinlock_wait_cycles":30723328,"thread_start_us":1035234,"threads_started":3,"wal-append.queue_time_us":806919}
I20260709 13:04:55.273149  8829 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:04:55.113528 15280 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:55.123471 15465 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:04:55.271245 15471 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:04:55.281332  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:55.311748  8751 raft_consensus.cc:2410] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:04:55.251060 15269 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:55.317018  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:55.333132  8749 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:04:55.113782 15245 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.271502 15480 consensus_queue.cc:1059] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.204s
I20260709 13:04:55.276700 15468 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:55.351383 15477 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.227s
W20260709 13:04:55.373837  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 63527.9us
W20260709 13:04:55.348762  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 152814us
I20260709 13:04:55.361526  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:04:55.350765  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 240617us
I20260709 13:04:55.365412  8645 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:55.312948 15340 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:55.383503  9388 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 937) took 3124 ms (client timeout 1855 ms). Trace:
I20260709 13:04:55.384227 15222 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:55.384649  9383 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:55.385288 15245 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:55.118885 15457 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.275960 15295 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.336098 15485 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.104086 15360 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.349702 15483 raft_consensus.cc:993] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: : Instructing follower e814ed2013244fa6be42623ea5c0c7b9 to start an election
I20260709 13:04:55.280864 15216 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.412814  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:55.415791  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:55.416971  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:55.341400 15349 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:55.438522  8647 leader_election.cc:336] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 2.540s (SENT)
I20260709 13:04:55.439075  8647 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:55.439301  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 236603us
I20260709 13:04:55.420615  8752 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:04:55.410681 15468 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:55.425938  9355 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:55.429641  8751 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:55.414929  9388 rpcz_store.cc:269] 0709 13:04:52.259285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.542735 (+283450us) service_pool.cc:224] Handling call
0709 13:04:55.377299 (+2834564us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b5400520b54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:55.383400 (+  6101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4779606784}
I20260709 13:04:55.418224 15222 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:55.428540 15245 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:55.427435 15477 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:55.343926 15481 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.311s
I20260709 13:04:55.481365  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:55.482514 15216 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:55.483887 15245 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:55.484347 15345 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.485126 15245 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.476576 15473 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:55.375218  8573 catalog_manager.cc:5697] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 3 to 10. New cstate: current_term: 10 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:55.489166  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1309) took 1078 ms. Trace:
I20260709 13:04:55.489904 15490 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.490516 15490 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.493146 15490 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:55.442159 15340 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.495776 15481 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:55.495887 15295 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:55.496191 15340 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:55.496665  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.496758 15481 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:55.497838 15481 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:55.498265 15217 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.500705 15217 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:55.479192 14771 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:04:55.502341 15133 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:55.502902 15133 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:55.494500 15477 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.503966 15133 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.505007 14936 raft_consensus.cc:3055] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:55.505144 15477 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:55.505517 15133 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:55.505577 14936 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:55.507345 14936 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.508669 14936 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:55.509025 15430 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.650s	user 0.007s	sys 0.007s
I20260709 13:04:55.509326 15462 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:55.475700 15349 raft_consensus.cc:1081] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Signalling peer e814ed2013244fa6be42623ea5c0c7b9 to start an election
I20260709 13:04:55.509805 15462 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:55.510851 15462 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.511991 15462 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:55.512713 15262 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:55.516301 15383 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.880s	user 0.016s	sys 0.001s
I20260709 13:04:55.488426  8573 catalog_manager.cc:5697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:55.518918  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:55.519151  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:55.519454 15345 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:55.520495  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:55.523221 15495 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:55.524904  8680 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:55.525439  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47290.8us
I20260709 13:04:55.437417 15485 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:55.530840  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 105050us
I20260709 13:04:55.489655  8753 rpcz_store.cc:276] 0709 13:04:54.410553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.416426 (+  5873us) service_pool.cc:224] Handling call
0709 13:04:54.430316 (+ 13890us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.153167 (+722851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.488854 (+335687us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54002ac954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:55.489092 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1660772736}
I20260709 13:04:55.458086 15284 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.523221  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:55.542920  8678 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20. Ops: []
W20260709 13:04:55.544884  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.547104  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:55.548619  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:55.545969  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:55.552218  8752 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:04:55.557058  9355 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:55.554164  8672 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:55.560773  9355 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:55.559615  8674 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:04:55.571825  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:55.546082 15281 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:55.575757  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 149905us
I20260709 13:04:55.578178  8794 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:55.548074 15327 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.547564 15340 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:55.509608 15430 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.537168 15349 consensus_queue.cc:1059] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:04:55.576634  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:55.600402 14786 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.608392  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:55.609334  8824 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:55.543107 15468 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:04:55.541380  8573 catalog_manager.cc:5697] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:55.448768 15465 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.573081 15492 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:55.606637 14786 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.614014 15494 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.614872 15430 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:55.615684  8676 raft_consensus.cc:1813] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Handling vote request from an unknown peer e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:55.641233  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:55.642241  8829 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:55.617678  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:55.618660  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:55.564149 15493 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:55.542707 15284 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.613103 15485 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:55.649103 15485 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:55.649588 15497 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.649747 15485 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.650198 15497 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.628080  8673 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:55.652314 15497 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:55.652874 15497 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:55.653453  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:55.653353 15497 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.573516 15499 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:55.655072 15499 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:55.656113 15499 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.657408 15499 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:55.659823  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:55.659956  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:55.660634  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:55.661216  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:55.660781  8747 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:04:55.628894  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:55.643380  8672 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:04:55.645435  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:55.661372  8676 raft_consensus.cc:2368] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17.
W20260709 13:04:55.635169  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 57099us
I20260709 13:04:55.665230  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:55.665951  8673 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:55.644393  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:55.594986 15262 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:55.660738 15284 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:55.665088  8794 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:55.670287  8750 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 21: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:04:55.686759  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.621774 14771 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:04:55.672827  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:55.600811 15489 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:04:55.644194  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:55.679234  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:55.711369  8752 raft_consensus.cc:1813] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Handling vote request from an unknown peer e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:55.711478  8747 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:55.710364  8824 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:55.595465 15281 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.680819  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:55.617226 15483 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:04:55.638971 15481 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:55.652163 15485 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:55.696591  8829 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:55.640383 15410 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.787283 15410 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.700846  8794 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:55.705374  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 91645.6us
W20260709 13:04:55.702047 15198 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.040s	user 0.011s	sys 0.014s
I20260709 13:04:55.620541 15327 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.715922  8752 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:55.720546 15372 raft_consensus.cc:3055] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:04:55.721252  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 49568.7us
I20260709 13:04:55.721276 15493 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:55.722154  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:55.805903  8830 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:55.805791  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 105049us
I20260709 13:04:55.644378 15494 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.726086  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:55.632835 15466 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:04:55.728277  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:55.810511  9386 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:55.728305  8749 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:04:55.734412  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:55.735946  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55222.3us
W20260709 13:04:55.787890  8722 connection.cc:256] RPC call timeout handler was delayed by -1.6935563210000737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
I20260709 13:04:55.645022 15492 raft_consensus.cc:697] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:55.714098 15500 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:55.795755  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:55.799154  8747 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:55.807003  8794 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:55.810542  9381 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 21: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:04:55.812678  8722 connection.cc:256] RPC call timeout handler was delayed by -1.1743999069999518s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19920000000000002s for a response
I20260709 13:04:55.789058  9387 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:04:55.815143 15251 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:55.697187 15345 consensus_queue.cc:1059] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.101s
W20260709 13:04:55.792868 15198 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.144s	user 0.011s	sys 0.014s
I20260709 13:04:55.822203  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:55.822957  9388 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:55.825474  8722 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:55.838017  8722 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:55.785707 15476 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.828493  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:55.830996  8674 raft_consensus.cc:2393] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:55.832266  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:55.841773  8821 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:55.831817  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 122499us
I20260709 13:04:55.838552  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:55.805160 15372 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:55.811815 15295 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:55.734330 15491 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.218s
I20260709 13:04:55.847882  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:55.851128  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:55.853638  9380 raft_consensus.cc:2368] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17.
W20260709 13:04:55.855417 15467 consensus_meta.cc:287] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.710s	user 0.014s	sys 0.001s
I20260709 13:04:55.859704  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:55.833767 15251 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.847249 15481 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:55.858142  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:55.860824  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:55.864243  8827 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 23.
I20260709 13:04:55.869601  8821 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:04:55.842185 15143 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:55.883297  9385 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:04:55.884547  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:55.885440  8825 raft_consensus.cc:2368] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17.
I20260709 13:04:55.879249  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:55.887898  9380 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 21: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:55.855844 15216 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:55.872378 15251 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:55.910962 15513 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:55.897444  8823 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:55.898435 15494 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:55.907286 15493 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.907970 15281 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:55.879653 15500 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:55.849028  8573 catalog_manager.cc:5697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 9 to 11, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 11 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:55.862398 14786 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:55.908671 15372 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.898499 15512 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:55.910001 15133 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:04:55.910111  8647 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:55.910629 15216 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.908095 15492 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:55.916996  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 43635.6us
I20260709 13:04:55.917593 15327 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:55.871723 15467 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.920120  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:55.920938  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:04:55.925002  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.926251  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 88328.1us
I20260709 13:04:55.927842  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:55.933945  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.848886 15476 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.939498  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:55.940191 15295 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:55.941536  8680 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:04:55.942231 15372 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:55.913467 15497 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:55.919515 15502 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:04:55.948248  8827 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:04:55.946280  8673 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 21.
I20260709 13:04:55.954365  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:55.953258  8794 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:55.954944  8823 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [15.11-18.12]
I20260709 13:04:55.955138  8673 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:04:55.955996  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:55.955974  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:55.956911  8673 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 21: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:55.959954  8794 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:55.960124  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:55.917654 15481 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:04:55.961660  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:55.961911  8678 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:55.969552 15251 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:55.970928  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:55.972031 15474 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:55.973543  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:55.974684 15474 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:55.975205 15474 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:55.975693 15474 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:55.976325 15476 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:55.981999  8751 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.509s	user 0.005s	sys 0.006s
I20260709 13:04:55.983000  8821 raft_consensus.cc:1275] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:55.946879 15410 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:55.982503  8751 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:04:55.984979  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:55.985653  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.988049  8823 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:55.988011  8753 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:55.986589  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:55.977931 15222 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:55.958197 15262 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:55.964071 15500 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:55.984103 15143 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:55.998160  8647 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:55.994575  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:55.994562  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:55.993237 15499 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:55.986706 15476 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:55.961606 15497 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:04:56.003208 15360 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:04:56.136257  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:56.122898 15462 consensus_meta.cc:287] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.610s	user 0.011s	sys 0.003s
I20260709 13:04:56.001397  8823 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621"
dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:60438
I20260709 13:04:56.300927  8823 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:56.301451  8823 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:56.119213 15480 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.143630 15295 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:56.335235  8673 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:04:56.179765 15510 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:56.064987 15262 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:56.393605 15517 log.cc:927] Time spent T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.206s	user 0.000s	sys 0.000s
I20260709 13:04:56.394306 15468 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:56.423390 15462 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:56.424381 15462 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:56.429263 15222 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:56.002002  8573 catalog_manager.cc:5697] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 8 to 9, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 9 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:04:56.299237 15499 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:56.503648  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.502326  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:56.353818 15281 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:56.193115 15476 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:56.529255  8823 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:56.425957 15462 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:56.476716 15510 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:56.447225 15295 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:56.594367  8679 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:56.640213  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 642028us
W20260709 13:04:56.641235  8826 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.000s	sys 0.010s
W20260709 13:04:56.617772 14936 consensus_meta.cc:287] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.106s	user 0.009s	sys 0.009s
I20260709 13:04:56.356153 15480 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:56.441624  8573 catalog_manager.cc:5697] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 6 to 7, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 7 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:56.669695  8756 raft_consensus.cc:2410] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:04:56.658563  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:56.669816  8830 consensus_meta.cc:287] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.822s	user 0.011s	sys 0.003s
I20260709 13:04:56.661952 15295 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:56.664194 15510 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:56.644553  8823 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:56.657353  8826 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.000s	sys 0.013s
I20260709 13:04:56.605196 15281 raft_consensus.cc:697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:56.670980  8830 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:56.682567  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:56.683171  8830 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:56.683547  8747 raft_consensus.cc:2410] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
W20260709 13:04:56.673925  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 679398us
I20260709 13:04:56.686053  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:56.596906 15430 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:56.614717 15251 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:56.706307  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47904.2us
I20260709 13:04:56.709878  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:56.713122  8722 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:04:56.672526 15492 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.000s	sys 0.010s
I20260709 13:04:56.716807  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:56.717264  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:56.719704  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:56.723670  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:56.724859  8751 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 23.
I20260709 13:04:56.725503  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:56.731627  8755 raft_consensus.cc:2410] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260709 13:04:56.733132  8829 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.006s	sys 0.017s
I20260709 13:04:56.734882  8754 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:04:56.717559  8646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.015s	sys 0.011s
W20260709 13:04:56.736624  8646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.015s	sys 0.011s
W20260709 13:04:56.733969  8829 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.006s	sys 0.017s
I20260709 13:04:56.866118  8747 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:56.867002  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:56.866552  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 153532us
W20260709 13:04:56.865084  8827 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.858s	user 0.018s	sys 0.027s
W20260709 13:04:56.871210  8827 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.019s	sys 0.027s
I20260709 13:04:56.682250 15467 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:56.883570 15222 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:56.884334 15222 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:56.886467 15500 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:56.886898 15222 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:56.889353  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:56.886962 15500 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:56.891832 15500 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:56.893028  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 185531us
I20260709 13:04:56.893837 15500 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:56.894397  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:56.895756 15360 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:56.896541 15360 raft_consensus.cc:697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:56.896811 15363 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.176s
I20260709 13:04:56.896561  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:56.901333 15222 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:56.897850 15360 consensus_queue.cc:237] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:56.904552 15499 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:04:56.723336 15492 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.000s	sys 0.011s
I20260709 13:04:56.732547 14936 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:56.908344  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:56.910651 15222 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:56.912227 15480 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:56.915091  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:56.915884  9385 raft_consensus.cc:1813] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Handling vote request from an unknown peer e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:56.916435  9385 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:04:56.917130 15519 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.302s	user 0.002s	sys 0.000s
I20260709 13:04:56.917667  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:56.918373  9382 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:56.923236  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:04:56.928174 15372 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.974s	user 0.013s	sys 0.000s
I20260709 13:04:56.722975 15522 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:04:56.932668 15474 consensus_meta.cc:287] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.956s	user 0.015s	sys 0.000s
I20260709 13:04:56.933465 15474 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:56.938256  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:56.939563 15465 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:56.942189  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:56.945729  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:56.946415 15222 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:56.951251  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:56.952806 15201 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:56.956382  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 1.23253e+06us
I20260709 13:04:56.998862  9352 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:57.000752 14786 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:57.001394 14786 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.001485  9352 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:56.963248 15476 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:56.967730  9382 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:04:56.938294  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2605) took 1062 ms. Trace:
I20260709 13:04:56.959726 15262 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:56.991111  9387 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:57.002700  8821 rpcz_store.cc:276] 0709 13:04:55.875499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.875827 (+   328us) service_pool.cc:224] Handling call
0709 13:04:55.878039 (+  2212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.983478 (+105439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.937852 (+954374us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540041d054. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:56.938191 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231532928}
I20260709 13:04:56.975011  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3023) took 1455 ms. Trace:
W20260709 13:04:56.993868  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 55737.5us
I20260709 13:04:57.005172  8647 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:57.001914 14786 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:57.007968  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:57.011322  8749 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:04:57.018919  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:57.021608 15490 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:57.022092 15490 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:57.023655  8674 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:56.986317 15506 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:57.032799  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.034054  9355 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.035s	sys 0.103s
W20260709 13:04:57.034730  9355 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.036s	sys 0.103s
W20260709 13:04:57.030076  8645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.028s	sys 0.047s
W20260709 13:04:57.039954  8645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.028s	sys 0.048s
W20260709 13:04:57.041045  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 1.04649e+06us
I20260709 13:04:57.022598  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:57.046075 15524 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:57.048138  8722 leader_election.cc:400] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17.
I20260709 13:04:57.048647  8722 leader_election.cc:403] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 13:04:57.050588  8678 consensus_meta.cc:287] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.070s	user 0.013s	sys 0.000s
I20260709 13:04:57.051239  8678 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
I20260709 13:04:57.053411 15524 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17.
I20260709 13:04:57.004779 15467 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:57.004124  8677 rpcz_store.cc:276] 0709 13:04:55.519083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.520476 (+  1393us) service_pool.cc:224] Handling call
0709 13:04:55.520892 (+   416us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:55.521390 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:55.521420 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:55.522121 (+   701us) log.cc:844] Serialized 52 byte log entry
0709 13:04:55.616714 (+ 94593us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:56.945768 (+1329054us) spinlock_profiling.cc:243] Waited 5.36 ms on lock 0x7b1800352720. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e20a14bc 00007f32df3ef48b 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df409455 00007f32e5ef13c7 00007f32e25039a7 00007f32e25130bf
0709 13:04:56.946023 (+   255us) log.cc:844] Serialized 14 byte log entry
0709 13:04:56.946350 (+   327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.946434 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:56.946475 (+    41us) raft_consensus.cc:1685] finished
0709 13:04:56.946594 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.949311 (+  2717us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b1800352720. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:56.974902 (+ 25591us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":90867,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":78,"raft.queue_time_us":45026,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":15114,"spinlock_wait_cycles":24119936,"thread_start_us":161280,"threads_started":2,"wal-append.queue_time_us":176139}
I20260709 13:04:56.877468 15430 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.024379 15251 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:57.005963 15222 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.009912 15201 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.066351  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3075) took 1526 ms. Trace:
I20260709 13:04:57.076460  8753 rpcz_store.cc:276] 0709 13:04:55.539638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.541430 (+  1792us) service_pool.cc:224] Handling call
0709 13:04:55.987729 (+446299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.990958 (+  3229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.066091 (+1075133us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b54001022d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:57.066279 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1001142144}
I20260709 13:04:57.009506 15363 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.012915 15262 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:57.078042  8821 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:57.060045  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:57.033123 15516 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
W20260709 13:04:57.110496  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 50631.1us
I20260709 13:04:57.029549 15530 raft_consensus.cc:2764] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:04:57.117202  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:57.118773 15527 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:57.113440 15262 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:57.128852 15262 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.132401 15522 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:57.133505 15515 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:57.133371 15522 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.134168 15515 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.135383  9388 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:04:57.062376 15506 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.136564 15515 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:57.137573 15522 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:57.140688 15522 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:57.142808 15525 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:57.144196 15222 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:57.146062 15201 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:57.142995 15522 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.093219 15251 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:04:57.154063  8756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 1316) took 1497 ms (client timeout 1893 ms). Trace:
I20260709 13:04:57.153110  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 3021) took 1634 ms. Trace:
W20260709 13:04:57.155112  8756 rpcz_store.cc:269] 0709 13:04:55.656818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.660376 (+  3558us) service_pool.cc:224] Handling call
0709 13:04:57.153807 (+1493431us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001572d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:57.153982 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2317718784}
I20260709 13:04:57.155642  8676 rpcz_store.cc:276] 0709 13:04:55.518422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.518550 (+   128us) service_pool.cc:224] Handling call
0709 13:04:57.152788 (+1634238us) spinlock_profiling.cc:243] Waited 96 ms on lock 0x7b54002576d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:57.152990 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220776832}
I20260709 13:04:57.158828  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:57.171393 15526 log.cc:927] Time spent T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:04:57.150486 15284 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:57.175457 15284 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:57.176430 15284 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.177585 15284 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:57.138415 15506 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:57.154086 15523 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.179096 15251 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.179569 15523 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.160157 15527 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:57.183935 15527 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.185001  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:57.185878  8676 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 20: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:57.094113 15363 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.153335 15522 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:57.083168 15490 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.083073 15281 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.281001  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2781) took 1327 ms. Trace:
W20260709 13:04:57.282052  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 123368us
I20260709 13:04:57.189846 15523 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:57.285943  8722 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:57.200080 15463 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:04:57.339493  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 53658.9us
I20260709 13:04:57.164244 15516 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.117s
I20260709 13:04:57.340582  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:57.270522 15466 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:57.350087  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:57.348409  8722 leader_election.cc:400] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17.
I20260709 13:04:57.284226 15265 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:57.284178 15506 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:57.350965  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:57.356258  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:57.357548  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:57.338055 15522 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:57.362591  8749 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:57.342090 15515 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:57.365659 15198 raft_consensus.cc:3055] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:57.365476 15515 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.366173 15198 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:57.366822 15508 consensus_queue.cc:1059] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:57.367197 15198 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.367784 15508 raft_consensus.cc:3055] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:57.189473 15525 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.346747 15530 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:57.341372 14786 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.350857 15512 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:57.321653 15280 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:04:57.381909 15276 raft_consensus.cc:2764] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:04:57.385557 15198 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:57.388513  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.390415 15326 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:57.391669 15430 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:57.355266 15466 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.394560 15512 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.395229 15512 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:57.342689 15463 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.372783 15509 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:04:57.403170 15525 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:57.404256  8753 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:57.360890 15506 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.408149 15363 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:57.372426 15522 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:57.418962  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.418802 15522 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:57.422137 15474 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:57.394474 15466 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:57.422487 15506 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:57.367838 15515 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:57.425719 15516 raft_consensus.cc:3055] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:57.426508 15516 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:57.427897 15516 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.356621 15490 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:57.432484  9355 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:57.368263 15508 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:57.446376  9387 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:04:57.448590  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:57.449357  8750 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:04:57.450495  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:57.451853  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:57.449297  8754 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:57.418284 15463 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:57.447777 14936 consensus_meta.cc:287] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.004s
I20260709 13:04:57.484984  8754 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:04:57.487084  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 54701.4us
I20260709 13:04:57.489243  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:57.491531  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:57.472457 15516 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:57.479022 14936 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:57.495362  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:57.508483  8676 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:57.533345  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:57.493944 15201 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:04:57.548465  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 43670.1us
W20260709 13:04:57.548856  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 92777.2us
I20260709 13:04:57.547775  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:57.501446 15198 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:57.551367  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:04:57.556136  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.553287  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:57.494763 15532 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:57.555344  8755 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:57.557776  9383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 1174) took 3580 ms. Trace:
I20260709 13:04:57.560376  9383 rpcz_store.cc:276] 0709 13:04:53.969816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.971000 (+  1184us) service_pool.cc:224] Handling call
0709 13:04:55.384144 (+1413144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.414665 (+ 30521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.547533 (+2132868us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400520b50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:57.550586 (+  3053us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3248986496}
I20260709 13:04:57.505550 15383 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:04:57.565671  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:57.566174  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:57.568035  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:57.568960  9389 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:57.576601 14786 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:57.554834 15198 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.579123 15198 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:57.580688  8676 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:57.553974 14936 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:57.589897  8676 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:57.590919  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:57.595485  8755 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:04:57.591728  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:57.592819  8752 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:57.593339  8719 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:57.599666  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 141311us
I20260709 13:04:57.601162  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:57.620539  8755 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:04:57.621207  8755 pending_rounds.cc:85] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 10
I20260709 13:04:57.621532  8755 pending_rounds.cc:107] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:04:57.621826  8755 raft_consensus.cc:2889] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:57.636027  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:57.636869  8749 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:57.610112 15463 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:57.606850 15527 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:57.656388  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:57.657109  8676 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:57.499295 15508 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.659829 15201 raft_consensus.cc:697] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:57.615015 15105 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:04:57.603888 14936 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.588862 15532 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.667755  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3029) took 2052 ms. Trace:
I20260709 13:04:57.672076  8681 rpcz_store.cc:276] 0709 13:04:55.615316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.615954 (+   638us) service_pool.cc:224] Handling call
0709 13:04:55.616344 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:55.642724 (+ 26380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:55.642759 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:55.684759 (+ 42000us) log.cc:844] Serialized 26 byte log entry
0709 13:04:57.174757 (+1489998us) spinlock_profiling.cc:243] Waited 4.75 ms on lock 0x7b18001d36e0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:04:57.494189 (+319432us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:57.494267 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.494887 (+   620us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:57.663479 (+168592us) raft_consensus.cc:1685] finished
0709 13:04:57.663504 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.667680 (+  4176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":38709,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":12309,"spinlock_wait_cycles":10925312,"thread_start_us":319163,"threads_started":1,"wal-append.queue_time_us":1901331}
I20260709 13:04:57.676299  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:57.674084  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:57.694864 15522 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:04:57.674562 15506 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:04:57.706504 15198 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.720880 15198 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:57.707173 15465 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.710s	user 0.011s	sys 0.003s
I20260709 13:04:57.722262  8675 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:04:57.722678  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:57.723608  8822 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:57.723244 15512 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.724592  8753 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 13:04:57.725847 15512 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:57.675027 15515 consensus_queue.cc:1175] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" responder_term: 16 status { last_received { term: 16 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:57.736780 15485 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:04:57.747159  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 73112.5us
I20260709 13:04:57.736629  8822 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:57.695726 15524 raft_consensus.cc:3055] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:57.670110 15201 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.744486  8676 raft_consensus.cc:2410] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:04:57.658257 15527 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.750269  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:57.758548  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:57.665655 15530 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 13:04:57.749462 15524 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:57.753789 15147 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:57.755156 15508 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:57.747555 15485 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:57.763931  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:57.761222 15198 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:57.767572  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:57.774487  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.784511 15198 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:57.787720 15500 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.788256 15500 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.755535 15527 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:57.761524 15522 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:57.780948  8751 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:57.790827  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:57.790994  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.775355  9386 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 952) took 2079 ms. Trace:
I20260709 13:04:57.782100  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2612) took 1853 ms. Trace:
I20260709 13:04:57.773430  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2611) took 1852 ms. Trace:
I20260709 13:04:57.803913  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:57.792121  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2607) took 1873 ms. Trace:
I20260709 13:04:57.783918  8676 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:04:57.802330  9386 rpcz_store.cc:276] 0709 13:04:55.695827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.696079 (+   252us) service_pool.cc:224] Handling call
0709 13:04:55.696380 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.811077 (+114697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.774982 (+1963905us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b540061b554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:57.775271 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222852352}
I20260709 13:04:57.806206  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2601) took 1991 ms. Trace:
I20260709 13:04:57.809114  8829 rpcz_store.cc:276] 0709 13:04:55.815324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.827106 (+ 11782us) service_pool.cc:224] Handling call
0709 13:04:55.836263 (+  9157us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:55.850464 (+ 14201us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:55.862864 (+ 12400us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.770225 (+1907361us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b18000f0ca0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:57.778813 (+  8588us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:57.779486 (+   673us) log.cc:844] Serialized 26 byte log entry
0709 13:04:57.779659 (+   173us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:57.779895 (+   236us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.780089 (+   194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.780163 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:57.805955 (+ 25792us) raft_consensus.cc:1685] finished
0709 13:04:57.805982 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.806138 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1003464,"prepare.queue_time_us":26255,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":103,"raft.queue_time_us":975921,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":319,"spinlock_wait_cycles":43422464,"thread_start_us":940699,"threads_started":1,"wal-append.queue_time_us":1237353}
I20260709 13:04:57.802809  8827 rpcz_store.cc:276] 0709 13:04:55.928846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.959254 (+ 30408us) service_pool.cc:224] Handling call
0709 13:04:55.970989 (+ 11735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.971329 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:56.871827 (+900498us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.781721 (+909894us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:57.781785 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:57.781834 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.781901 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.782033 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":668589,"prepare.queue_time_us":432098,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":661815,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":82,"thread_start_us":1095939,"threads_started":2,"wal-append.queue_time_us":445206}
I20260709 13:04:57.766551 15147 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:57.768275 15465 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:57.801254 15430 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:57.814085  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:57.820746  8751 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:57.821105  8751 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:57.815133  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:57.790460 15494 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:57.774360 15481 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:57.763628 15506 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:04:57.821383  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:57.799942 15522 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:57.822783  8825 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:57.802909  8828 rpcz_store.cc:276] 0709 13:04:55.920690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.952022 (+ 31332us) service_pool.cc:224] Handling call
0709 13:04:55.952361 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.952583 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:56.179377 (+226794us) log.cc:844] Serialized 14 byte log entry
0709 13:04:56.693847 (+514470us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.772767 (+1078920us) spinlock_profiling.cc:243] Waited 7.66 ms on lock 0x7b1800389c80. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:04:57.772974 (+   207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:57.773043 (+    69us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:57.773098 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.773183 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.773353 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1048883,"prepare.queue_time_us":445134,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":1080055,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":10164,"spinlock_wait_cycles":333529984,"thread_start_us":579460,"threads_started":2,"wal-append.queue_time_us":865577}
I20260709 13:04:57.823836  8796 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:57.824214  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:57.828088  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:57.824491  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:57.829654  8829 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:57.829906  8827 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:57.804659  8826 rpcz_store.cc:276] 0709 13:04:55.919197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.919369 (+   172us) service_pool.cc:224] Handling call
0709 13:04:55.919856 (+   487us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:55.920356 (+   500us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:56.693905 (+773549us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.782749 (+1088844us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.783094 (+   345us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.783251 (+   157us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:57.783908 (+   657us) log.cc:844] Serialized 52 byte log entry
0709 13:04:57.784055 (+   147us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:57.784256 (+   201us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.784547 (+   291us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.784755 (+   208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.784841 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:57.791937 (+  7096us) raft_consensus.cc:1685] finished
0709 13:04:57.791955 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.792070 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":591927,"prepare.queue_time_us":469521,"prepare.run_cpu_time_us":155,"prepare.run_wall_time_us":153,"raft.queue_time_us":599083,"raft.run_cpu_time_us":492,"raft.run_wall_time_us":5664,"thread_start_us":1030674,"threads_started":2,"wal-append.queue_time_us":485330}
I20260709 13:04:57.825006  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:57.805107 15133 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.831524  8823 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:04:57.806874 15466 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:57.831774  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:57.835668  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:57.835944  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:57.829147  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:57.836290  8830 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:57.836656  8828 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 21: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:04:57.813265 15500 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:57.839043  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:57.839262  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:57.821354 15147 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:57.819006 15490 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.818794 15465 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.834168  8751 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.865010  8751 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:57.869347  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:57.833068 15295 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:57.874014 15295 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:04:57.875169 15295 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.801520 15525 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:04:57.838379  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:57.809769 15462 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:04:57.820549 15430 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:57.841372  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:57.862216  8823 raft_consensus.cc:2368] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16.
I20260709 13:04:57.862205  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:57.887706  8829 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:57.841701  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:57.822791 15477 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:57.790046 15198 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.882718  9386 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:57.891053  8824 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:04:57.840731  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:57.893057  8830 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:04:57.837850 15133 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.898075  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:57.927829 15522 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.929158 15133 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:57.929512 15522 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:57.901662  8826 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:04:57.861869 15269 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:04:57.937366  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.865878 15245 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:57.963845  8751 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
W20260709 13:04:57.968133 15563 log.cc:927] Time spent T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 13:04:57.969960 15562 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:04:57.974169 15530 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:04:57.930912 15477 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.835120 15481 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:57.981545 15477 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:57.982498 15481 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:57.982767 14936 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:57.983585 15245 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:57.881809 15465 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:57.986757  8823 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:57.987337  8829 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:57.988214  8822 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
W20260709 13:04:57.990417  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.991084  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.895562 15430 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:57.991791  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.992556  8748 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:04:57.993109  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.997988 15343 consensus_queue.cc:1048] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:04:58.000512 15216 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:57.884479 15490 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.006546  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:58.007269  8826 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:04:58.007373  8823 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
W20260709 13:04:58.008387  8680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2780) took 2089 ms (client timeout 2137 ms). Trace:
I20260709 13:04:58.008570  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:04:58.008852  8680 rpcz_store.cc:269] 0709 13:04:55.919073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.955615 (+ 36542us) service_pool.cc:224] Handling call
0709 13:04:58.008191 (+2052576us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54002409d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:58.008329 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3042582912}
I20260709 13:04:58.009860  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:57.837327 15524 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.012863 15524 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:58.009217  8824 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:58.016678  8824 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:04:58.010465  8828 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
W20260709 13:04:58.022387  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.839634 15485 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.031420  8828 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:58.033541  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:58.038563  8824 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:04:57.959043 15269 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:58.041162  8830 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:04:58.047142  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:58.049678  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:58.050734  8754 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:04:58.053323  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:58.013937 15512 consensus_queue.cc:1175] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e814ed2013244fa6be42623ea5c0c7b9" responder_term: 15 status { last_received { term: 15 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 15 index: 8 } }
I20260709 13:04:58.056495  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:58.046489 15269 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:58.079893  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:58.083292  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:58.084515  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:58.095057  8676 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:58.093986  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:58.096307  8823 raft_consensus.cc:2376] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18. Re-sending same reply.
I20260709 13:04:58.098440  8793 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:58.103693  8824 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 13:04:58.055356 15462 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.116626  8793 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:58.113950  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:58.119194  8827 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:58.118530  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:58.123358  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:58.078387 15490 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:58.052906 15474 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
W20260709 13:04:58.141805  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 109158us
I20260709 13:04:58.144150  8794 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:58.137049 15477 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:58.132139 15462 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.146212 15477 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:58.146749 15477 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:58.147755  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:58.148015  8794 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:58.148753  8676 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:04:58.150825  8750 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:58.136229 15372 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:58.138970 15480 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:58.174451  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:58.181914 15549 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.604s	user 0.004s	sys 0.000s
I20260709 13:04:58.177418 15525 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:58.200433  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:58.202073  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:58.194465 15525 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.204196  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:58.204943  8680 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:58.205771 15525 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:04:58.207450  8678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 1365) took 2614 ms (client timeout 2013 ms). Trace:
W20260709 13:04:58.207907  8678 rpcz_store.cc:269] 0709 13:04:55.592819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.594199 (+  1380us) service_pool.cc:224] Handling call
0709 13:04:58.207242 (+2613043us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b54002562d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:58.207371 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":731954048}
W20260709 13:04:58.184155  8673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 3033) took 2226 ms (client timeout 1954 ms). Trace:
I20260709 13:04:58.209281  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:58.177412  8676 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
W20260709 13:04:58.209364  8673 rpcz_store.cc:269] 0709 13:04:55.957888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.961410 (+  3522us) service_pool.cc:224] Handling call
0709 13:04:58.183795 (+2222385us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54001ff4d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:58.184004 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":850969472}
I20260709 13:04:58.210011  8678 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:04:58.212719  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:04:58.219683  9353 connection.cc:256] RPC call timeout handler was delayed by -1.550043776999928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18930000000000002s for a response
W20260709 13:04:58.220641  9353 connection.cc:256] RPC call timeout handler was delayed by -1.4942437769999515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
W20260709 13:04:58.221175  9353 connection.cc:256] RPC call timeout handler was delayed by -1.1530924449999702s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
I20260709 13:04:58.214274  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:58.230315  8673 raft_consensus.cc:2410] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:04:58.167469 15372 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.252246  9353 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:58.255271  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:58.256285  9353 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:58.256496  8676 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:04:58.225431 15457 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.615s	user 0.020s	sys 0.033s
I20260709 13:04:58.211551 15295 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:58.261260  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 128080us
I20260709 13:04:58.276893  8793 leader_election.cc:400] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17.
I20260709 13:04:58.221107  8827 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 24.
I20260709 13:04:58.267652  8680 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:04:58.286700  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:58.288338  9353 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:58.255009 15372 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:58.321677  8678 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:58.276151 15457 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.658s	user 0.021s	sys 0.034s
I20260709 13:04:58.322739  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:58.313403 15154 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:58.340260 15154 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:58.340786 15568 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
W20260709 13:04:58.342644 15572 log.cc:927] Time spent T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:04:58.343557  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:58.344641  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:58.346212  8755 raft_consensus.cc:1275] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:04:58.346572  8755 pending_rounds.cc:85] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 11
I20260709 13:04:58.346815  8755 pending_rounds.cc:107] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:04:58.347054  8755 raft_consensus.cc:2889] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:58.340740 15154 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:58.361413 15154 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.364058 15154 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:58.377339 15269 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.288187 15345 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:58.421458  8756 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:58.288771 15494 raft_consensus.cc:2764] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:04:58.433338  8678 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:58.310154 15065 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:58.436482  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:58.440476  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:58.401270 15154 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:58.444427  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.294689  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:04:58.459952 15522 consensus_meta.cc:287] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.004s
I20260709 13:04:58.435693 15462 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:58.458626 15527 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.003s
I20260709 13:04:58.468720 15405 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:58.473934 15154 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:58.477361  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 329447us
I20260709 13:04:58.462168 15345 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:58.475456 15147 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.003s
W20260709 13:04:58.492127  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 197589us
I20260709 13:04:58.511063  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:58.466894 15465 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.526518  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:58.530357  9383 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:58.524428 15147 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.533041  9384 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:04:58.534284  8676 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:58.517879 15345 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:58.512917 15390 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:58.484421 15405 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:58.534554 15147 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:58.558812  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:58.557183 15524 consensus_meta.cc:287] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.544s	user 0.015s	sys 0.000s
I20260709 13:04:58.561290 15583 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.563397 15480 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:58.511734 15527 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.540171 15201 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:58.571171  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:58.574422  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:58.560622 15476 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:58.556277 15390 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.604172  8754 raft_consensus.cc:2368] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17.
I20260709 13:04:58.606930  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:58.607496 15465 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:57.284592  8679 rpcz_store.cc:276] 0709 13:04:55.953793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.691335 (+737542us) service_pool.cc:224] Handling call
0709 13:04:56.691792 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.692092 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:56.996108 (+304016us) log.cc:844] Serialized 14 byte log entry
0709 13:04:57.280486 (+284378us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:57.280568 (+    82us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:57.280632 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.280723 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.280918 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":257567,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":127,"thread_start_us":587427,"threads_started":2}
I20260709 13:04:58.613370  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2782) took 2145 ms. Trace:
I20260709 13:04:58.580878 15340 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.613938  8674 rpcz_store.cc:276] 0709 13:04:56.467711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.961538 (+493827us) service_pool.cc:224] Handling call
0709 13:04:56.961870 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.035477 (+ 73607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.613039 (+1577562us) spinlock_profiling.cc:243] Waited 51.9 ms on lock 0x7b54004303d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:58.613296 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119318912}
I20260709 13:04:58.614953 15340 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.615607 15269 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:58.581514 15583 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.617384 15340 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:58.624620 15527 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:58.624969  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:58.625744  8828 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:04:58.632653 15390 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:58.603394 15405 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.640501 15405 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:58.651187 15551 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.056s	user 0.002s	sys 0.003s
I20260709 13:04:58.653258 15477 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.660804  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:58.661093  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:58.661898  9385 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:58.664410 15585 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.665127 15585 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.666064 15465 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.666805 15465 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.668056 15585 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:58.669576 15465 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:58.622249 15345 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.676929  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:58.681825 15245 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:58.685714  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.691727 15506 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:58.692385 15506 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.693439  9387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 960) took 1329 ms. Trace:
I20260709 13:04:58.693898  9387 rpcz_store.cc:276] 0709 13:04:57.364534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.364697 (+   163us) service_pool.cc:224] Handling call
0709 13:04:57.376977 (+ 12280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:57.447047 (+ 70070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.693156 (+1246109us) spinlock_profiling.cc:243] Waited 69.1 ms on lock 0x7b54005ea1d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:58.693371 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158876032}
W20260709 13:04:58.708343  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.677229 15345 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:58.661026 15390 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:58.716138  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:04:58.716481  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.719020  8676 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: []
W20260709 13:04:58.676980 15430 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.651s	user 0.009s	sys 0.003s
I20260709 13:04:58.719868  8722 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:58.695039 15506 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:58.706319 15467 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:58.722566  8677 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:58.714468 15262 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:58.723978  9383 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:58.731707  9382 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:58.737035  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:58.737798  8752 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:58.710035 15585 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:58.723235  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:58.740281  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:58.736719 15372 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.742210 15326 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:58.743418  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:58.721128 15513 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.745190 15359 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
W20260709 13:04:58.755254  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 237919us
I20260709 13:04:58.744621  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:58.768555  8749 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:58.761020  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:58.769654  8751 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:04:58.761929  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:58.770680  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:58.771189  8755 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:04:58.771364  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2686) took 1427 ms. Trace:
I20260709 13:04:58.764909  8677 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:04:58.714847 15269 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:04:58.771883  8756 rpcz_store.cc:276] 0709 13:04:57.343932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.344105 (+   173us) service_pool.cc:224] Handling call
0709 13:04:57.354770 (+ 10665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.466271 (+1111501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.771043 (+304772us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400156dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:58.771297 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2438627200}
I20260709 13:04:58.773434  8752 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:58.760357 15326 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.740191 15465 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:58.787283  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:58.788105  8676 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 12.
I20260709 13:04:58.746694 15506 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:58.754086 15360 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.789528  8749 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:58.722447 15390 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.764539 15295 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:58.794909  8722 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:58.798482 15484 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:58.798585 15268 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:58.798849  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:58.814074  8677 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:58.763141 15522 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:04:58.784611 15583 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:58.763042 15467 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:58.803010  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 82328.8us
I20260709 13:04:58.778981 15216 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:58.787168 15326 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:58.787103 14786 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.792457 15474 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:04:58.739521 15430 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.796036 15359 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:04:58.806177  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:58.807507 15530 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:04:58.812537  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 301572us
I20260709 13:04:58.748255 15262 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.799588  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:58.821167  9387 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:04:58.778904 15513 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.828217  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:58.832399  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:58.765952 15585 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.835320 15216 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.853019 15530 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.853705 15530 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.854984 15585 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:58.856014 15530 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:58.810022 15484 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:58.858822 15262 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:58.859557 15484 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.860829 15532 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:58.861440 15484 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:58.837677  9389 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:58.862915 15372 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:58.864327 15216 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:58.872575 15467 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:58.876946  8677 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:04:58.835237 15268 consensus_queue.cc:1059] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:04:58.813366 15360 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.881049 15360 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:58.881178 15390 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:58.881657 15360 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.846408  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:58.799542 15465 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.885955 15465 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:58.797888 15489 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:04:58.846201  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:58.822901 15457 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:58.878651  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:58.889521  8677 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:04:58.797881 15506 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.893985  8673 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:58.893332  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:58.895148  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:04:58.896001  8681 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:58.897764  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:58.847990 15359 consensus_queue.cc:1243] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:58.835572 15201 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 13:04:58.889561 15405 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:04:58.832993  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 390980us
I20260709 13:04:58.888844 15484 raft_consensus.cc:3055] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:58.916153  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:58.920763  8677 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 18: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:58.917209  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:58.882296 15360 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:58.924060  9383 raft_consensus.cc:2393] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 21: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:58.908705 15477 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:58.847611 14786 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:58.952790  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 171579us
I20260709 13:04:58.893383 15457 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:58.921675 15405 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.982678  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:58.922211 15484 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:58.984478  8645 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:58.998044  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:58.999549  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:04:59.001940  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 188950us
I20260709 13:04:59.002480  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:04:59.003616  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:59.006213  8681 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:04:59.013229  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2784) took 1830 ms. Trace:
I20260709 13:04:58.996284 15405 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:59.013870  8672 rpcz_store.cc:276] 0709 13:04:57.182831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.396464 (+213633us) service_pool.cc:224] Handling call
0709 13:04:58.185128 (+788664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.478627 (+293499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:58.478680 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:58.478745 (+    65us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:58.509851 (+ 31106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.509989 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.012979 (+502990us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b54001ff4d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:59.013144 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10265,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"spinlock_wait_cycles":1812777344,"thread_start_us":292654,"threads_started":1}
I20260709 13:04:59.023874  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:04:59.026176  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 202272us
I20260709 13:04:59.027843  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:59.029018  8677 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 18: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:59.036075  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:59.043414  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 248562us
I20260709 13:04:59.048852  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:59.052276  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:59.048890  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:59.053791  8753 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:59.053992  8722 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:59.066165  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:04:59.036375 15359 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.067028  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:59.068128  9384 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:59.092736  9383 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:59.093941  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:59.096820  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:59.097623  9389 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:04:59.097867 14786 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:59.071614 15359 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.101279 15359 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:59.092378 15474 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:59.103260 15474 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.103804 15474 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:59.111593 15360 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:59.112568 15326 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.114030  8750 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:04:59.115146  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2798) took 1155 ms. Trace:
I20260709 13:04:59.039736 15484 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.115610  8680 rpcz_store.cc:276] 0709 13:04:57.960351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.287239 (+326888us) service_pool.cc:224] Handling call
0709 13:04:58.314115 (+ 26876us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:58.314711 (+   596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:58.314758 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:58.315560 (+   802us) log.cc:844] Serialized 52 byte log entry
0709 13:04:58.899989 (+584429us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:59.072092 (+172103us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.088824 (+ 16732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.088920 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.114856 (+ 25936us) raft_consensus.cc:1685] finished
0709 13:04:59.114883 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.115073 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16427,"raft.queue_time_us":1200,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":42962,"thread_start_us":612054,"threads_started":2,"wal-append.queue_time_us":606396}
I20260709 13:04:59.116977 15430 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:59.117883 15607 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:59.118155 15154 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:59.119104  8680 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:59.112954 15457 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:59.065738 15390 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:59.161552  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:59.048564 15532 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:04:59.066960 15606 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:59.204522  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:04:59.205613  8680 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:59.087487 15605 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:59.211014  9389 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:59.216892  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:59.226310  9353 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:59.127923 15603 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:59.178401 15390 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:59.238269  9353 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:59.202540 15606 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.199522 15360 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:59.207643 15506 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:59.232077 15462 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:59.253242  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:59.265167  8749 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:04:59.213502 15489 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.255288  9384 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:04:59.212821 15604 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:59.233897 15605 raft_consensus.cc:697] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:04:59.248330  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 154449us
W20260709 13:04:59.271634  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 244389us
I20260709 13:04:59.275368  8722 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:59.237432 15607 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:59.244994 15340 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:59.245390 15390 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:59.276062  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:59.239358 15603 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:04:59.257638 15216 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:59.256011 15606 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:59.299319  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:59.332345  8647 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:59.322028 15513 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:59.285341 15603 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:59.270300 15360 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.299172 15604 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.278339 15489 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.331663 15269 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:59.354648  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 70928us
W20260709 13:04:59.350101 15620 log.cc:927] Time spent T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.096s	user 0.001s	sys 0.001s
I20260709 13:04:59.359377  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:04:59.364243 15269 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:59.365629 15269 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.374606 15467 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:59.375248 15467 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:59.376303 15467 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.377835 15467 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:59.360371 15627 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:59.336810 14936 consensus_queue.cc:1059] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:04:59.381750 14936 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:59.382277 14936 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:59.383455 14936 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.387282 14936 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:59.384431 15527 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:59.384085 15607 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.347620 15621 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:59.368443 15622 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:04:59.357795 15605 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.393673 15522 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:59.361601 15604 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:04:59.404541  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 72316.6us
I20260709 13:04:59.423871  8647 leader_election.cc:400] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16.
I20260709 13:04:59.424330  8647 leader_election.cc:403] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:04:59.405336  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46099.6us
I20260709 13:04:59.356112 15457 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:04:59.421105  8675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2777) took 3702 ms (client timeout 2582 ms). Trace:
W20260709 13:04:59.425860  8675 rpcz_store.cc:269] 0709 13:04:55.718976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.859423 (+140447us) service_pool.cc:224] Handling call
0709 13:04:59.420881 (+3561458us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b54000da7d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:04:59.421028 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4281710464}
I20260709 13:04:59.433956  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:59.435267  9384 raft_consensus.cc:2393] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:04:59.437995  9384 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:04:59.443825  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:59.409471 15527 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:59.420892 15489 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:59.417737 15633 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:59.451157  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:59.456522  9383 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
W20260709 13:04:59.455191  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 192472us
I20260709 13:04:59.457470  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:59.458973  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:59.460599  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:59.465693  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:59.488986  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:59.499847  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1382) took 2010 ms. Trace:
I20260709 13:04:59.500540  8673 rpcz_store.cc:276] 0709 13:04:57.489953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.241246 (+751293us) service_pool.cc:224] Handling call
0709 13:04:58.241812 (+   566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.894880 (+653068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.499374 (+604494us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b54001f9854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:04:59.499720 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1499187456}
I20260709 13:04:59.441092 15457 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:59.514715 15588 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:04:59.540913  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:59.547180  9382 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
W20260709 13:04:59.547807  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.534505 15262 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:59.459434 15631 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16.
W20260709 13:04:59.511216 15622 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:04:59.490465 15621 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:59.513108 15602 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:59.510546 15340 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:59.510774 15632 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:59.458092 15405 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.583037 15615 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.513954 15592 consensus_queue.cc:1048] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:04:59.606441 15621 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:59.607311 15602 raft_consensus.cc:697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:59.562427 15360 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:59.610157  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 161893us
I20260709 13:04:59.547555 15349 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.587118 15262 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:59.517946 15625 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 13:04:59.604517 15632 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:59.547192 15217 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 13:04:59.549059 15527 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.621647 15615 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.654222 15633 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:59.670727 15602 consensus_queue.cc:237] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.672688 15621 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.640983 15349 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.644402 15632 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:59.680245 15474 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.574s	user 0.017s	sys 0.000s
I20260709 13:04:59.692162  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2631) took 1604 ms. Trace:
W20260709 13:04:59.692544  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 106914us
I20260709 13:04:59.694118  8830 rpcz_store.cc:276] 0709 13:04:58.087533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.106233 (+ 18700us) service_pool.cc:224] Handling call
0709 13:04:58.135358 (+ 29125us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:58.145118 (+  9760us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:58.145169 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:58.158672 (+ 13503us) log.cc:844] Serialized 26 byte log entry
0709 13:04:58.551894 (+393222us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:58.572564 (+ 20670us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.582038 (+  9474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.582135 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.382032 (+799897us) spinlock_profiling.cc:243] Waited 9.58 ms on lock 0x7b18000eae20. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:59.689540 (+307508us) raft_consensus.cc:1685] finished
0709 13:04:59.691922 (+  2382us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.692087 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22402,"prepare.queue_time_us":20378,"prepare.run_wall_time_us":62,"raft.queue_time_us":31874,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":6604,"spinlock_wait_cycles":86290816,"thread_start_us":307608,"threads_started":1,"wal-append.queue_time_us":422729}
I20260709 13:04:59.687207 15643 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.681115 15642 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:04:59.700268  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.704088 15615 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:59.699129 15527 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:59.708974 15410 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:59.711781 15262 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.714375 15633 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.734009  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:59.755671  8748 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:59.735618  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:59.737581  8754 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:59.738778  8755 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:59.756654  8752 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 18: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:04:59.699963 15360 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:59.753245  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:04:59.716061 15640 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:04:59.760964 15638 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:04:59.761925  8752 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:59.760964  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:59.766969  8749 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:04:59.758229  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:04:59.767851  8751 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 20: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:04:59.757850  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:04:59.716570 15268 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:59.781740 15268 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.755085 15603 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.784199  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:59.784766 15268 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:59.786231 15603 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:59.769939  8647 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:04:59.753173 15604 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:59.791062 15625 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:59.771803 15349 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:59.731196 15643 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.731108 15217 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:59.776015  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:04:59.768698 15615 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:59.749569 15642 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.798015  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:04:59.779888 15466 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.780936 15405 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:59.794800  9382 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:04:59.698390 15474 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.750422 15410 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:04:59.809587  8748 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:04:59.795495 15457 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:59.796590 15217 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:04:59.800431 15645 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.800982 15466 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:59.799906 15489 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:04:59.832578 15467 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:59.832542 15489 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.833343 15467 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:04:59.834853  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.836398 15606 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.007s
I20260709 13:04:59.836829 15606 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.839949  8793 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:59.849076  9389 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 13:04:59.847220  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:59.813613  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:04:59.817423 15642 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:59.819280 15410 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.819948 15484 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:59.859359 15649 log.cc:927] Time spent T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:04:59.820302 15462 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.541s	user 0.014s	sys 0.003s
I20260709 13:04:59.821666 15474 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:59.823395 15217 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.825906 15522 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:59.826624 15645 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.827378 15326 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:04:59.847504 15245 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:59.852780 15643 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:59.853084 15360 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:59.855115 15606 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:59.849967  8793 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:04:59.864177  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.865096  9382 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:04:59.806557 15621 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:59.868335 15198 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:59.867942 15295 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.867686 15583 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:59.942139  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 1.09601e+06us
I20260709 13:04:59.841346 15605 consensus_queue.cc:1059] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:04:59.877285 15245 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.943759 15372 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.943853 15595 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:59.954900 15372 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.955073 15595 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.956522 15245 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:59.957425 15372 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:59.957703 15605 raft_consensus.cc:3055] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:59.957746 15595 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:59.944065 15583 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.958263 15605 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:04:59.960484 15583 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:59.944912 15593 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:59.945076 15295 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.964155 15295 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:59.964820 15295 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.965452 15295 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.942684 15217 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:59.944579 15477 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.967214  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:04:59.967133 15477 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.968089  8828 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:59.969498 15645 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:59.971369  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:59.972162  8827 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:04:59.863014 15481 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:59.871371 15462 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:59.977641  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:59.978709 15500 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:59.979187 15500 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:59.979270 15586 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.980036 15494 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:04:59.973016  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:04:59.980159 15586 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:04:59.980566 15494 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.984334  9355 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:59.987596  8824 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:04:59.988173  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.959944 15605 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:04:59.969671 15477 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:04:59.977329 15499 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:59.936527 15596 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:04:59.979610 15500 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:59.974155  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:00.005045  8794 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:59.970886 15489 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:04:59.975723  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 125839us
I20260709 13:04:59.966316  8823 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:59.939083 15198 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.938019 15467 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:04:59.940400 15251 consensus_queue.cc:1059] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:04:59.868197 15607 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.093s
W20260709 13:04:59.976500  8647 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.949s (SENT) [suppressed 11 similar messages]
I20260709 13:04:59.970479 15594 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:59.988086 15340 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:00.007926 15133 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:59.992991 15481 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:00.008767 15586 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:00.009119 15596 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:00.010800 15390 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.716s	user 0.017s	sys 0.003s
I20260709 13:05:00.019104  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:00.021152 15245 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:59.934850 15360 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:00.035820  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 50334.7us
I20260709 13:05:00.036717  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:00.047600  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:00.038136  8828 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:05:00.043552 15580 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 13:05:00.002599 15595 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:04:59.872412 15492 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.162s
W20260709 13:05:00.045112  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 68632us
I20260709 13:05:00.028625 15494 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:00.032084 15198 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:00.033069 15605 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:00.052156  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:04:59.996439 15583 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
W20260709 13:05:00.090696  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 152684us
I20260709 13:05:00.043740  8794 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:04:59.995991 15632 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.035867 15477 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:00.041481 15340 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:00.086841  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.042044 15594 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:00.054483 15586 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:00.082542 15582 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:00.098291  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54660.9us
I20260709 13:05:00.101291  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:00.060695 15360 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:00.039986 15499 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
W20260709 13:05:00.115386  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 60789.1us
I20260709 13:05:00.096916 15658 raft_consensus.cc:3055] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:00.101104 15340 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:00.126713  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.055238 15390 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.102167 15580 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:00.155376 15580 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:00.156626 15580 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.103559 15494 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:00.107499 15586 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.103298 15582 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:00.164271 15582 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:00.164526 15390 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:00.103189 15245 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 13:05:00.100944 15594 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:00.141953 15657 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:00.147439 15372 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:00.117326 15295 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:00.149369 15632 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:00.151517 15655 log.cc:927] Time spent T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:05:00.151888 15596 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:00.124490 15492 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:00.170990 15492 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:00.172029 15492 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:00.173368 15492 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:00.136009 15466 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:00.176710 15466 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:00.163218 15494 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.177561 15494 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.163781 15586 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:00.120987 15658 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:00.189563 15658 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:00.194329  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2630) took 2107 ms. Trace:
I20260709 13:05:00.195048  8826 rpcz_store.cc:276] 0709 13:04:58.086977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.091421 (+  4444us) service_pool.cc:224] Handling call
0709 13:04:58.092071 (+   650us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:58.092791 (+   720us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:58.093265 (+   474us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.198456 (+105191us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b1800209320. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c557 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:04:58.839023 (+640567us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.880336 (+1041313us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.975592 (+ 95256us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:59.993419 (+ 17827us) log.cc:844] Serialized 52 byte log entry
0709 13:04:59.993610 (+   191us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:59.993931 (+   321us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.998979 (+  5048us) log.cc:844] Serialized 14 byte log entry
0709 13:05:00.002000 (+  3021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.002113 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:00.185604 (+183491us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b1800209320. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:05:00.194034 (+  8430us) raft_consensus.cc:1685] finished
0709 13:05:00.194065 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:00.194247 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1190167,"prepare.queue_time_us":53783,"prepare.run_cpu_time_us":248,"prepare.run_wall_time_us":233,"raft.queue_time_us":1328081,"raft.run_cpu_time_us":853,"raft.run_wall_time_us":23945,"spinlock_wait_cycles":8159744,"thread_start_us":669873,"threads_started":3,"wal-append.queue_time_us":399466}
I20260709 13:05:00.211485 15494 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:00.228107 15630 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.881s	user 0.002s	sys 0.002s
I20260709 13:05:00.231333 14786 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.232154 14786 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:00.251825  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.252698  8681 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:00.253563  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.254340  8676 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:00.254581  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.255463  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.261168  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:00.267226 14786 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:00.272507 15524 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:00.273286 15524 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:00.274430  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:00.275151  8679 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:00.277443  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:00.278321  9382 raft_consensus.cc:2393] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:00.275796 15524 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:00.287197  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.288650  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:00.289423  9382 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:00.299743  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:00.280613  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:00.302911  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:00.305016  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.281198  8719 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:05:00.307482  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 189580us
W20260709 13:05:00.311403  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 211490us
I20260709 13:05:00.319849  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.319156  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:00.314059  8823 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:00.316618  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:00.312582  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.322450  8824 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:05:00.323773  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.324486  8827 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:00.326568  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.327268  8827 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:00.328898  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:00.331142  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.335954  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.339104  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.341604  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:00.342499  9383 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:00.342775  8822 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:00.344368  8679 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:00.347234  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.348714  8677 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:00.346134  8678 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 12. (term mismatch)
I20260709 13:05:00.352450  8678 pending_rounds.cc:85] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 11
I20260709 13:05:00.352837  8678 pending_rounds.cc:107] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 23.12
I20260709 13:05:00.353259  8678 raft_consensus.cc:2889] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:00.341487  8645 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:00.356168  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.356909  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:00.357463  8672 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:00.357968  8680 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 20: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:00.358590  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.359331  8830 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:00.367177  8673 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 21. Current term is 22. Ops: []
W20260709 13:05:00.369203  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 88126.1us
I20260709 13:05:00.311573 15524 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.386513  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:00.370420  8719 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:00.355891 15201 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:00.374521 15532 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:00.380178 15457 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:00.381234 15626 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.383241  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.385066  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.386437  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.318840 15493 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:05:00.391570  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:00.384915  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.412199  8822 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:00.412490  8679 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:05:00.425451  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 324218us
I20260709 13:05:00.393920  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:00.351742 15658 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:00.405614  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.266592 15586 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.403478 15512 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:00.409499  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:00.424234  8829 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:00.395552  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:00.426443  8722 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:00.430121  8794 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:00.457252  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 63517.3us
I20260709 13:05:00.459749  8794 leader_election.cc:400] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17.
I20260709 13:05:00.436918  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:00.439874  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.408303 15532 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.407918 15359 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:00.444165 15489 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:00.436434 15390 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:00.464250 15390 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.444888  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.444427  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:00.467216  8827 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:00.467361  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.468125  8679 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:05:00.410382 15457 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:00.470909 15457 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:00.471091  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.450026  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.453226 15485 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:00.473995  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2714) took 1738 ms. Trace:
I20260709 13:05:00.446233 15198 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:00.460175  8794 leader_election.cc:403] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:00.474509  8750 rpcz_store.cc:276] 0709 13:04:58.735455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.735616 (+   161us) service_pool.cc:224] Handling call
0709 13:04:58.744225 (+  8609us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:59.114494 (+370269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.473676 (+1359182us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54001572d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:00.473921 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":849774976}
I20260709 13:05:00.461843 15633 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:00.479490 15633 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:00.479809 15359 raft_consensus.cc:697] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:00.425138 15626 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.481088 15359 consensus_queue.cc:237] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.479553  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:00.479856  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.483369  8747 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:05:00.483667  8681 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:05:00.484441  8825 raft_consensus.cc:2368] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 20. Current term is 21.
I20260709 13:05:00.479553  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.408484 15201 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:00.463374 15532 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:00.460803 15512 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:00.460100 15499 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:00.472011  9389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 970) took 1032 ms. Trace:
I20260709 13:05:00.404529 15585 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.404453 15524 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.490120  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:00.479972 15633 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:00.497962  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.497962  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:00.500067  8672 raft_consensus.cc:2410] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:05:00.500489  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.464813 15390 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:00.464151  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.524482  8676 raft_consensus.cc:2410] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:05:00.507642  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.509315  9389 rpcz_store.cc:276] 0709 13:04:59.439350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.439989 (+   639us) service_pool.cc:224] Handling call
0709 13:04:59.440465 (+   476us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:59.849558 (+409093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.471678 (+622120us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b540061b554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:00.471933 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":924563840}
I20260709 13:05:00.525408  8675 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:05:00.504429  9383 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:00.497071  8824 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:00.514874  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:00.502419  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:00.532757  8827 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:00.509541 15512 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:00.535388  8829 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 17: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:00.534044  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.546731  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.547709  8681 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:05:00.546196  8827 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:05:00.548981  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.550987  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:00.551960  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 49687.4us
I20260709 13:05:00.553958 15664 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.554601 15664 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.551963  8672 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 24.
I20260709 13:05:00.522794 15647 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.562796  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:00.529266 15626 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:00.524199 15610 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:00.529703 15583 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17.
I20260709 13:05:00.557185 15477 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:00.550997 15499 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.556950 15664 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:00.523269 15585 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:00.700397  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:00.728386  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 269401us
I20260709 13:05:00.600524 15477 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:00.645561 15667 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.583503 15582 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.579054 15647 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:00.758335  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 292967us
W20260709 13:05:00.759172  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.760183  8722 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:05:00.762348  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 199713us
I20260709 13:05:00.750257 15477 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:00.764916  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:00.769824  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.770865  8824 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:00.770177  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:00.833817  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 63774.7us
I20260709 13:05:00.800331  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.789927  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:00.795362  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 308850us
I20260709 13:05:00.805315 15632 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.827641  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:00.753499 15667 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:00.770020 15500 consensus_meta.cc:287] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.734s	user 0.013s	sys 0.000s
I20260709 13:05:00.841648  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:05:00.795732 15340 consensus_meta.cc:287] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.003s
I20260709 13:05:00.790136 15489 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:00.771688 15133 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:00.757717 15499 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:00.806097 15580 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:00.832224 15626 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:00.857209  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.830569 15624 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:00.804397 15201 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:00.855480 15586 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:00.872467 15489 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:00.873621 15594 consensus_meta.cc:287] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.704s	user 0.013s	sys 0.004s
I20260709 13:05:00.874130 15594 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.874595  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.875969  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.757351 15610 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.876814 15667 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:00.877161  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.877836  8826 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:00.878649 15610 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:00.879483 15582 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:00.847103 15500 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.857791  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:00.878517 15198 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:00.881397  8823 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:00.800769 15666 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.883508  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.854506  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.888254 15594 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:00.889425 15633 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:00.891873 15675 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:00.892241 15532 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.881953 15476 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.898723  9384 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:05:00.918493  9355 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:00.899106 15457 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.899449 15647 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:00.901183  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:00.901568 15217 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.932s	user 0.015s	sys 0.000s
I20260709 13:05:00.922817  8824 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:00.921798 15476 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:00.935294  8679 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:00.936333  8821 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:05:00.847115 15632 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.903209  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:00.903733 15500 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:00.904637  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.853700 15133 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:05:00.905686 15360 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.788s	user 0.014s	sys 0.004s
I20260709 13:05:00.906638 15610 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:00.907222 15466 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:00.853994 15340 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:00.867991 15624 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:00.910322  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.914075 15410 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:00.935863 15499 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:00.912137 15585 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:00.967083  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:00.916512  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.968791  8826 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:00.948505 15133 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:00.854185 15580 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.867054 15524 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:00.947608  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:00.936137 15198 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:00.954010  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.955039  8672 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:00.975893  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:00.964311  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:00.964648  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:00.964829  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:00.965728  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:00.966274  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:00.914981 15676 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:00.970727  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.967953  8753 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:00.973801  8678 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:00.917544 15666 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:00.985759  8747 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:00.987741  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:00.993769  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:00.980361  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:00.994693  8751 raft_consensus.cc:2410] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:05:00.995939  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.997534  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:00.998319  8756 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:05:00.999650  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:01.000001  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:00.996760  8826 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:05:01.000640  8749 raft_consensus.cc:2410] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:05:01.000671  8752 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:00.950680 15360 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.002816  8754 raft_consensus.cc:2410] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:05:00.973275 15580 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:00.953970 15610 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:00.984891 15524 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.015883  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:01.015918  8753 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:05:00.983557 15676 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:01.020183  8678 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:05:01.006949  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:00.991003 15198 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:01.021577  8828 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
W20260709 13:05:01.003124  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 168125us
I20260709 13:05:01.030146  8722 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:01.023209  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:01.007452 15633 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:00.970553 15390 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.033116  8678 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:01.050910  8676 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 13:05:01.044708  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:01.045049  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:01.047916  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:01.062367  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 164089us
I20260709 13:05:01.062646  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:01.062984  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:01.064802  8828 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:01.067948  8678 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:01.069303  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:01.070226  8678 raft_consensus.cc:2410] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:05:01.071065  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:01.071907  8678 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:05:01.074314  8678 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:01.028894 15676 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:01.079324  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:01.047998 15489 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:01.091795  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 252077us
I20260709 13:05:01.095269  8826 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:01.096810  8826 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:01.077495 15626 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:05:01.094112  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 50202us
I20260709 13:05:01.107208  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:01.020844 15524 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.088152 15633 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:01.057750 15647 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:05:01.121485 15147 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:01.138173  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:01.139318  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3045) took 2410 ms. Trace:
I20260709 13:05:01.091756 15489 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.142652  8822 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:01.143699  8722 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:01.157471  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:01.115046 15359 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:01.150869  8672 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:01.093356 15532 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:01.155699 15489 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:01.111799 15681 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:01.181918  8822 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:01.142048 15457 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:01.168131 15683 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:05:01.139755  8674 rpcz_store.cc:276] 0709 13:04:58.711083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.711495 (+   412us) service_pool.cc:224] Handling call
0709 13:04:58.711893 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.712171 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:59.001137 (+288966us) log.cc:844] Serialized 14 byte log entry
0709 13:05:01.120159 (+2119022us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b18004644a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e20a14bc 00007f32df3ef48b 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a
0709 13:05:01.120470 (+   311us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:01.120551 (+    81us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:01.120632 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.120766 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.120969 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14079,"prepare.queue_time_us":374929,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":37700,"raft.queue_time_us":2304,"spinlock_wait_cycles":7251200,"thread_start_us":374672,"threads_started":2,"wal-append.queue_time_us":368071,"wal-append.run_cpu_time_us":1096,"wal-append.run_wall_time_us":1001123}
I20260709 13:05:01.148025  8828 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:05:01.171066  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:01.151562 15633 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.197409  8828 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:05:01.166326 15217 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:01.208309 15639 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:01.212688 15481 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:01.223026 15477 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:01.194392 15632 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:01.191761 15681 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:01.238484  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:01.207527 15633 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:01.240629  8752 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:01.238168  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:01.141777 15198 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.217989 15217 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.257028  8749 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:05:01.258728  8749 pending_rounds.cc:85] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 12
I20260709 13:05:01.259030  8749 pending_rounds.cc:107] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:05:01.259261  8749 raft_consensus.cc:2889] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:01.219470 15624 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:01.226125 15477 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:01.271971  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 149768us
W20260709 13:05:01.276739  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.249779 15681 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:01.247649 15680 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:01.262749 15217 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:01.309181  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:01.261126 15639 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:01.190181 15626 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.264953 15360 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:01.316751  8719 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:01.278331 15201 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.306074 15680 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.274025 15340 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:01.279503 15666 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:01.334038 15680 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:01.337026 15686 log.cc:927] Time spent T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:05:01.340468  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:01.340332 15201 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.311683 15390 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:01.347662  9353 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:01.346586 15512 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.783s	user 0.017s	sys 0.000s
I20260709 13:05:01.351336 15610 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:01.358736 15065 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:01.357573 15524 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:01.370137 15684 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:05:01.371366 15635 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:01.360755 15216 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:05:01.360518 15340 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:05:01.368067  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:01.407562  9353 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:01.411643  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 71242.6us
W20260709 13:05:01.402513 15687 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:05:01.416244  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:01.417279  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:01.417346  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:01.416091  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:01.358783 15512 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:01.423171  9387 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:01.424095  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:01.425094  9385 raft_consensus.cc:2376] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17. Re-sending same reply.
I20260709 13:05:01.423521  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:01.426304  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:01.426494  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:01.427531  8719 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:01.427582  9381 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: [16.12-17.13]
I20260709 13:05:01.428478  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:01.429054  9382 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:01.427742  8749 raft_consensus.cc:2410] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:05:01.407965 15065 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:01.409379 15390 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:01.432745  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:01.432943  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:01.434098  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:01.436978  8751 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:01.432718  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:01.429454 15690 log.cc:927] Time spent T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:05:01.439213  8750 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:05:01.421885 15493 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:05:01.377878 15639 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:01.421797 15410 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:01.430402 15610 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:01.433658 15692 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:01.449529 15493 consensus_queue.cc:1243] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:01.433848 15513 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:01.437112 15360 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:01.466949  9382 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
I20260709 13:05:01.463229 15410 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:01.433692 15635 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:05:01.449673 15602 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:01.475895  8749 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:01.467972 15692 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:01.476213 15694 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.143s	user 0.002s	sys 0.000s
W20260709 13:05:01.487488 15691 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.149s	user 0.002s	sys 0.000s
I20260709 13:05:01.489025  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1438) took 1036 ms. Trace:
I20260709 13:05:01.489499  8829 rpcz_store.cc:276] 0709 13:05:00.453113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.087961 (+634848us) service_pool.cc:224] Handling call
0709 13:05:01.094902 (+  6941us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.095326 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:01.095362 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:01.095905 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 13:05:01.176566 (+ 80661us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:01.176614 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.176692 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:01.488757 (+312065us) raft_consensus.cc:1685] finished
0709 13:05:01.488780 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.488944 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1339,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":99,"spinlock_wait_cycles":42240,"thread_start_us":80206,"threads_started":1,"wal-append.queue_time_us":164991}
I20260709 13:05:01.465808 15632 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:05:01.492365 15682 raft_consensus.cc:3055] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:01.492503 15692 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:01.492959 15682 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:01.494107 15682 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.495447 15593 raft_consensus.cc:3055] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:01.495733 15682 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:01.495961 15593 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:01.497054 15593 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.498206 15593 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:01.499305 15680 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:01.501971 15680 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:01.503156 15201 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:01.503911 15632 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:01.504633 15632 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.505002  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:01.506031 15681 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:01.507284 15632 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:01.468077 15610 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.488237  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:01.463630 15624 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:05:01.512377  8751 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:01.495258 15410 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.525389  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:01.526451  8674 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:01.528743  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 364845us
I20260709 13:05:01.531262  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:01.532527  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:01.533304  8747 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:01.513389 15602 raft_consensus.cc:697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:01.536321  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:01.509133 15681 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:01.538748  8828 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:05:01.540830  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:01.543040  9382 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
W20260709 13:05:01.530206 15695 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.157s	user 0.001s	sys 0.001s
I20260709 13:05:01.540869  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:01.541934  8722 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:01.540994  8566 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7aff04f53d4845a6a4fa491fd2a06145 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:01.546061  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:01.551083  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:01.555003  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:05:01.560250 15671 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.685s	user 0.003s	sys 0.000s
I20260709 13:05:01.566231  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:01.550192 15697 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:01.571394 15697 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.572178 15697 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:01.576401 15602 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.580334 15512 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:01.585500 15692 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:01.587361  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2650) took 1206 ms. Trace:
I20260709 13:05:01.588896 15692 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:01.582981 15065 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:01.571151  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:01.601811  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:01.603122  9381 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:01.609241 15633 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:01.611673 15676 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.501s	user 0.019s	sys 0.005s
I20260709 13:05:01.613373  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:01.621394  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:01.622349 15624 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:01.622815 15624 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.622922 15360 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:01.623208 15624 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:01.623476 15360 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:01.624011 15360 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:01.616605 15610 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:01.628654 15580 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.608s	user 0.012s	sys 0.003s
I20260709 13:05:01.630107  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:01.619068 15594 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:01.636514 15594 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:01.637017 15594 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:01.588102  8830 rpcz_store.cc:276] 0709 13:05:00.381341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.381539 (+   198us) service_pool.cc:224] Handling call
0709 13:05:00.381968 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:00.382249 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:00.382274 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:00.382880 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 13:05:00.723681 (+340801us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:00.723750 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.723859 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:01.559583 (+835724us) spinlock_profiling.cc:243] Waited 41.8 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:01.586730 (+ 27147us) raft_consensus.cc:1685] finished
0709 13:05:01.586783 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.587216 (+   433us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96094080,"thread_start_us":340336,"threads_started":1,"wal-append.queue_time_us":463890}
I20260709 13:05:01.615221  8755 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:05:01.619721  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 53541us
I20260709 13:05:01.633455  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:01.634472  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:01.641698  8681 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:01.635052  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:01.633380 15624 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:01.640836  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:01.641649  8678 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:01.629421 15580 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:01.687845  8828 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:05:01.708547  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:01.657567  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:01.694653  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:01.737109  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 79657.9us
I20260709 13:05:01.739064 15675 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:01.739549 15675 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.739955 15675 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:01.673043  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.741395  8747 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
I20260709 13:05:01.742537  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:01.646019  8719 connection.cc:256] RPC call timeout handler was delayed by -1.1345120810000253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 13:05:01.744144  8828 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:01.742388  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:01.747299  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:01.748574  9389 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:01.659204  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:01.673485  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:01.756685  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:01.643465  8829 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:01.700577  9384 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 979) took 1402 ms (client timeout 1795 ms). Trace:
I20260709 13:05:01.660322  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:05:01.735227  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 88849.8us
I20260709 13:05:01.759208  8674 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:05:01.643628  8793 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:01.640919  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:01.760790  8830 raft_consensus.cc:2376] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17. Re-sending same reply.
I20260709 13:05:01.765663  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:01.756937  9382 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:01.783423  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:01.762048  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 118509us
I20260709 13:05:01.755635  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:01.769950  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:01.786687  8722 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:01.787941  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:01.788539  8828 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:01.789099  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:01.775362  9381 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:01.790148  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:01.790822 15701 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:01.791326 15701 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.791671 15701 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:01.792227 15699 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:01.792790  8756 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:01.792841 15699 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.700668 15633 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:01.797351  8826 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 24. Current term is 25. Ops: []
I20260709 13:05:01.771440  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:01.659775 15632 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:01.775152  8794 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:01.800774  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:01.775300  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:01.802362  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:01.758227  9384 rpcz_store.cc:269] 0709 13:05:00.298484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.318788 (+ 20304us) service_pool.cc:224] Handling call
0709 13:05:01.700248 (+1381460us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b540046a5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:01.700472 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1325657472}
I20260709 13:05:01.776633  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:01.804261  8830 raft_consensus.cc:2410] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:05:01.777741  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:01.769480 15610 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:01.785537  8793 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:01.784541  8681 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:01.816905  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:01.772002  8677 raft_consensus.cc:2368] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16.
I20260709 13:05:01.798125  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:01.799105  8680 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:01.776480  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:01.822284  8674 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:01.810199  8678 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:01.628803 15681 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.173s
I20260709 13:05:01.805218  8722 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:01.657922 15647 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.138s
I20260709 13:05:01.658154 15692 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.140s
I20260709 13:05:01.774673 15481 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:01.704123 15697 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:01.835315  8830 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:01.781275 15477 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:01.840255  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:01.551710 15133 consensus_meta.cc:287] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.567s	user 0.013s	sys 0.002s
I20260709 13:05:01.701829 15217 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.846202  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:01.773900 15682 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:05:01.771015 15676 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:01.870170  8751 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:05:01.854516  8829 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:01.863300  8681 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:01.845470 15512 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:01.856308 15477 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:01.878094 15477 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:01.885839 15147 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:01.887744 15670 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 1.007s	user 0.005s	sys 0.000s
I20260709 13:05:01.889093 15676 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:01.889428 15583 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:01.890443 15583 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:01.891037 15583 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:05:01.891739 15489 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.685s	user 0.015s	sys 0.000s
I20260709 13:05:01.892220 15489 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:01.894520 15580 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 25 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:01.894914  8748 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:01.896117 15594 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:01.898483  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:01.902750 15217 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:01.905938  8756 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:01.913079 15699 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:01.898466 15594 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:01.913123 15624 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:01.916283 15624 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:01.918955  8751 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:01.919963  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:01.906826 15698 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.923969  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:01.925239  8747 raft_consensus.cc:2410] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:05:01.885600 15512 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:01.920729  8751 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:01.811555 15632 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.921504  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:01.929020  8751 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:01.930814  8681 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:01.940161  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 154736us
I20260709 13:05:01.947726  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:01.922730 15489 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:01.864967 15133 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:01.929997 15512 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:01.968468  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:01.971037  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:01.971997  8829 raft_consensus.cc:2368] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16.
I20260709 13:05:01.990983  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:01.929555 15698 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:02.015018  8829 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:02.017006  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:02.019160  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:02.023980  8722 leader_election.cc:400] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16.
I20260709 13:05:02.024426  8722 leader_election.cc:403] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:05:02.025630  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:02.026328  8830 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:02.027405  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 261854us
I20260709 13:05:02.032137  8756 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:05:02.032296  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 257194us
I20260709 13:05:02.033469  8794 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:01.999024 15489 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:05:02.044386  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 288864us
I20260709 13:05:02.046370  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:02.031507 15635 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:01.972606 15580 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:05:02.051107  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2661) took 1604 ms. Trace:
I20260709 13:05:02.053977  8794 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:02.056885  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:02.041178  8681 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:02.043457  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:02.052491  8827 rpcz_store.cc:276] 0709 13:05:00.446685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.580394 (+133709us) service_pool.cc:224] Handling call
0709 13:05:00.580815 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:00.581087 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:00.631776 (+ 50689us) log.cc:844] Serialized 14 byte log entry
0709 13:05:01.711685 (+1079909us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b1800055120. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:05:02.050672 (+338987us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:02.050746 (+    74us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:02.050798 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.050887 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:02.051038 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":658,"prepare.queue_time_us":39299,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":3765,"raft.queue_time_us":18395,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"spinlock_wait_cycles":3304320,"thread_start_us":338512,"threads_started":2,"wal-append.queue_time_us":1232431}
I20260709 13:05:02.066217  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:02.037484 15693 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:02.082441  8794 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:02.086020  8830 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:02.067394  8827 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:02.089112  8829 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:02.089588  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:02.087412  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 290833us
I20260709 13:05:01.962177 15583 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.100631  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:02.105077  8825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2658) took 1701 ms (client timeout 1990 ms). Trace:
W20260709 13:05:02.105603  8825 rpcz_store.cc:269] 0709 13:05:00.403911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.436571 (+ 32660us) service_pool.cc:224] Handling call
0709 13:05:02.104828 (+1668257us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b54003dd6d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:02.105002 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54719232}
I20260709 13:05:02.108212  8825 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:02.109555  8825 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:05:02.117630  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2652) took 1735 ms. Trace:
I20260709 13:05:02.118180  8823 rpcz_store.cc:276] 0709 13:05:00.382090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.385632 (+  3542us) service_pool.cc:224] Handling call
0709 13:05:00.385948 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:00.882207 (+496259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.117290 (+1235083us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b540002b7d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:02.117520 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1138921216}
I20260709 13:05:02.120414  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 3068) took 1329 ms. Trace:
I20260709 13:05:02.121088  8672 rpcz_store.cc:276] 0709 13:05:00.791052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.840689 (+ 49637us) service_pool.cc:224] Handling call
0709 13:05:02.120106 (+1279417us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b540042fed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:02.120320 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167954816}
I20260709 13:05:02.039892 15513 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:05:02.126372  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:02.051551 15489 raft_consensus.cc:697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:05:02.174005  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47728.7us
I20260709 13:05:02.175490  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:02.046289 15147 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:05:02.098757 15481 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:02.096670 15635 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:02.099702 15524 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:02.140085 15697 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:02.183624  8722 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:02.187506  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:02.144495 15594 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:02.141623 15712 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:02.141053 15711 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:02.049940 15360 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:02.181581  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 138091us
I20260709 13:05:02.144451 15593 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:02.191998  8719 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:02.141304 15602 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:02.193028  8673 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:02.194519  8677 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:02.187985 15147 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:02.191911 15720 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:02.190783 15681 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:02.188354 15682 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:05:02.186753  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:02.250866  8678 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:02.183735  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 101329us
W20260709 13:05:02.198755  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.252583  8794 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:02.139951 15639 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:02.240923  8829 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:02.203532 15706 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:05:02.200199 15707 log.cc:927] Time spent T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 13:05:02.237614 15593 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:02.194774 15513 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:02.194963 15524 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:02.250174  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 56862.2us
I20260709 13:05:02.196586 15133 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:02.196421 15697 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.203886 15727 raft_consensus.cc:2764] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:05:02.257303 15726 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:02.206750 15715 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:02.269541  8673 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:02.238433 15698 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:02.273972  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 82073us
I20260709 13:05:02.272392  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:02.296444  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:02.247505  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:02.248714 15632 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:02.311564 15632 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:05:02.312402 15701 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.520s	user 0.015s	sys 0.000s
I20260709 13:05:02.288132  8678 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:02.319754  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:02.320703  8672 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:02.249624 15147 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.275511 15712 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:02.309326  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:02.310855 15716 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.195381 15709 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 13:05:02.348044 15709 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.348688 15709 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:02.297228 15710 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
W20260709 13:05:02.353121 15675 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.613s	user 0.014s	sys 0.000s
I20260709 13:05:02.353681 15675 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:02.313169 15632 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:02.355034 15724 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:02.355568 15724 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:05:02.356132 15717 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:05:02.356506 15724 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.358110 15724 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:02.365067  9386 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 1195) took 1960 ms. Trace:
I20260709 13:05:02.365608  9386 rpcz_store.cc:276] 0709 13:05:00.404953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.405113 (+   160us) service_pool.cc:224] Handling call
0709 13:05:00.405525 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:00.405963 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:00.405996 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:00.406593 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 13:05:02.203074 (+1796481us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b180036ca60. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:05:02.269426 (+ 66352us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:02.269500 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.269636 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:02.364702 (+ 95066us) raft_consensus.cc:1685] finished
0709 13:05:02.364728 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:02.364976 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9142,"raft.queue_time_us":2529,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":12435,"spinlock_wait_cycles":25254144,"thread_start_us":65980,"threads_started":1,"wal-append.queue_time_us":1937413}
W20260709 13:05:02.324957 15719 log.cc:927] Time spent T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 13:05:02.371479 15672 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 1.494s	user 0.004s	sys 0.000s
I20260709 13:05:02.292984 15729 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:05:02.316190  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 43939.4us
I20260709 13:05:02.298466  8677 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
I20260709 13:05:02.374976  8794 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:02.375248  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:02.376453  8794 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:02.276963 15593 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:02.327076  8673 raft_consensus.cc:2410] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:05:02.255985 15720 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.314097 15697 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.318763 15513 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:02.345135 15711 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:02.345759 15716 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:02.325639 15715 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:02.258054 15489 consensus_queue.cc:237] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.382269  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:02.298748  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:02.319468 15726 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:02.400223  8672 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
W20260709 13:05:02.421335  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 189094us
I20260709 13:05:02.419723  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:02.451573  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 146015us
I20260709 13:05:02.365067 15360 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:02.385403 15712 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:02.454257  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:02.381409 15727 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:02.407016 15602 raft_consensus.cc:697] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:02.472506 15602 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.381011 15729 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:02.475589 15729 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.477025 15729 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:02.477785 15476 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:02.405571 15681 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:02.377885 15701 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:02.468708 15726 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:02.484093  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 185399us
I20260709 13:05:02.511247  8719 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:02.293602 15635 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:02.505175  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 223440us
I20260709 13:05:02.314051 15481 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.478556 15476 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:05:02.479810  8679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 3057) took 2009 ms (client timeout 2213 ms). Trace:
I20260709 13:05:02.464823 15712 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:02.403851 15711 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:02.453860 15583 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:02.536525  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 77882.1us
I20260709 13:05:02.538504  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:02.539572  8825 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:02.525612 15477 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.647s	user 0.013s	sys 0.001s
I20260709 13:05:02.516176 15736 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:05:02.548543  8823 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:05:02.549974 15681 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.557804 15697 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:02.559758 15701 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:02.521878 15739 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:02.561273 15675 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:02.544098  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.540418 15476 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:02.522689 15734 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:05:02.546211  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1445) took 1024 ms. Trace:
I20260709 13:05:02.571153  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.578281  8825 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
W20260709 13:05:02.522034  8679 rpcz_store.cc:269] 0709 13:05:00.470068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.476067 (+  5999us) service_pool.cc:224] Handling call
0709 13:05:02.479553 (+2003486us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b5400339fd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:02.479723 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":954448256}
I20260709 13:05:02.409754 15698 consensus_queue.cc:1059] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.113s
I20260709 13:05:02.539335 15709 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:02.567574  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 56433.6us
I20260709 13:05:02.541577 15715 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:02.614001  8750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 3108) took 1650 ms (client timeout 1948 ms). Trace:
W20260709 13:05:02.680024  8750 rpcz_store.cc:269] 0709 13:05:00.963192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.964553 (+  1361us) service_pool.cc:224] Handling call
0709 13:05:02.613651 (+1649098us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b54002ace54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:02.613915 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062043008}
I20260709 13:05:02.679198  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.683547  8825 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
W20260709 13:05:02.622330  8675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2824) took 2336 ms (client timeout 1531 ms). Trace:
I20260709 13:05:02.684792  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:02.684890  8675 rpcz_store.cc:269] 0709 13:05:00.285896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.286798 (+   902us) service_pool.cc:224] Handling call
0709 13:05:02.622056 (+2335258us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b5400174b54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:02.622229 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":545423104}
I20260709 13:05:02.686112  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.621544  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.681558  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:02.691785  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.543421 15712 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.623242  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2730) took 1176 ms. Trace:
W20260709 13:05:02.647902  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.682051  8828 rpcz_store.cc:276] 0709 13:05:01.521288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.791827 (+270539us) service_pool.cc:224] Handling call
0709 13:05:01.794431 (+  2604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.926861 (+132430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:01.926901 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:01.963185 (+ 36284us) log.cc:844] Serialized 26 byte log entry
0709 13:05:02.134335 (+171150us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:02.134393 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.134534 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:02.545937 (+411403us) raft_consensus.cc:1685] finished
0709 13:05:02.545964 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:02.546134 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94273,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":130,"thread_start_us":236662,"threads_started":2,"wal-append.queue_time_us":391119}
I20260709 13:05:02.579931 15711 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:02.673391 15512 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.681s	user 0.013s	sys 0.000s
I20260709 13:05:02.583132 15477 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:02.703718  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.607623  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.699227  8749 rpcz_store.cc:276] 0709 13:05:01.450340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.451789 (+  1449us) service_pool.cc:224] Handling call
0709 13:05:01.465174 (+ 13385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:01.476507 (+ 11333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.622907 (+1146400us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b54001581d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:02.623156 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23913856}
I20260709 13:05:02.596529 15697 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:02.710655  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.746224 15489 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:05:02.700961 15583 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:02.746685 15720 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:02.745718  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.779579  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:02.791365  8827 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:05:02.791729  8830 raft_consensus.cc:2368] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19.
I20260709 13:05:02.792335  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.745612 15685 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.652874 15476 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.714476 15512 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:02.743793 15477 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:02.744518  8824 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:02.795709  8827 raft_consensus.cc:2376] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16. Re-sending same reply.
W20260709 13:05:02.702157 15693 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.602s	user 0.014s	sys 0.000s
I20260709 13:05:02.713148 15736 raft_consensus.cc:3055] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:02.676488 15524 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:02.720911  8823 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:02.785092  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.745141  8719 leader_election.cc:400] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16.
I20260709 13:05:02.772509 15147 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:02.794438  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.737404 15217 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:05:02.721472  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.828336  8829 raft_consensus.cc:2410] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:05:02.828830  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.800391  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:02.777601 15739 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:05:02.796707 15746 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:02.793054 15685 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:02.808578 15602 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.793915 15583 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:02.811333 15390 raft_consensus.cc:3055] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:02.813055  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3064) took 2303 ms. Trace:
I20260709 13:05:02.835863  8676 rpcz_store.cc:276] 0709 13:05:00.509513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.554124 (+ 44611us) service_pool.cc:224] Handling call
0709 13:05:00.554565 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.058037 (+503472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.812697 (+1754660us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b54000dacd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:02.812971 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1140495232}
I20260709 13:05:02.836472  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.836282  8827 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:05:02.835619 15390 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:02.838354 15602 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:02.839977 15685 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:02.839788 15583 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.840875 15602 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:02.819070  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:02.817608  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 72622us
I20260709 13:05:02.830138  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:02.813948 15711 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.845997  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:02.848418  8647 leader_election.cc:336] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.949s (SENT)
I20260709 13:05:02.851344  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:02.815013 15736 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:02.706946 15712 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.851500  8827 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:02.815490 15693 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16.
I20260709 13:05:02.855080  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:02.880584 15693 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:02.881865 15693 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:02.857483  8825 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:02.834851 15746 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:02.852664 15602 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:02.867974 15736 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.870085 15512 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:02.870643 15715 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:02.872217  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:02.889220  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:02.875051  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.876855 15712 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:02.877714  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.897542  8827 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:02.882473 15477 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.882820 15527 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:02.906565 15527 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:02.883164 15147 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:02.906735 15602 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.856657  8823 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:02.910210  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:02.911027  8829 raft_consensus.cc:2376] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20. Re-sending same reply.
I20260709 13:05:02.912032  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.886758  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:02.912982  8748 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 21: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:02.913893  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:02.886390  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:02.886302  8719 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:02.907042 15527 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:02.918853 15512 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:02.919051 15217 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:02.919502 15512 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:02.919929 15217 consensus_queue.cc:1243] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:02.919711 15527 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.920454 15512 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.921103 15527 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:02.922513 15701 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:02.922983 15701 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:02.923906 15701 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.924957 15746 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:02.925259 15701 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:02.926735  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:02.928701  8675 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:05:02.929441  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:02.931480  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:02.932138  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:02.932308  8678 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:02.887549  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:02.933380  8751 raft_consensus.cc:2410] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 18 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:02.893908  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:02.934100  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:02.935113  8647 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:02.898337  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:02.905658 15593 consensus_meta.cc:287] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.006s
I20260709 13:05:02.905370 15477 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.936192 15465 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:02.936331 15593 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:02.938009 15477 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:02.938100  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:02.938687 15593 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:02.939059 15709 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:02.939936 15709 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:02.941262  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:02.939949  8680 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:02.889705  8830 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:02.942250 15465 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:02.942742 15465 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:02.943908  8647 leader_election.cc:400] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19.
I20260709 13:05:02.944357  8647 leader_election.cc:403] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:05:02.945113 15709 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19.
I20260709 13:05:02.946779  8823 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:02.943152 15465 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:02.896133  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.896839 15724 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:02.954164  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:02.956295  8677 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:05:02.946295  8647 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:02.948326  8678 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:02.958164  8647 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:02.958849  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:02.959156  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:02.960918  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:02.963019  8747 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:05:02.954876  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 166972us
I20260709 13:05:02.891705 15681 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:05:02.968640  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:02.968956  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.966583  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:02.969441  8829 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:02.970254  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:02.971305  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:02.973562  8823 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:02.976540  8679 pending_rounds.cc:85] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 12
I20260709 13:05:02.976943  8679 pending_rounds.cc:107] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:05:02.977226  8679 raft_consensus.cc:2889] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:02.980902  8722 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:02.966131  8825 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:02.984203  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.985500  8825 raft_consensus.cc:2410] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:05:02.960390 15675 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.988847 15675 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:02.956034 15724 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:02.990935 15724 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:02.993662  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:02.994942  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:02.995375  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:02.996165  8749 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:02.996348  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:02.997242  8749 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:02.997697  8747 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:02.967154 15217 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.973766  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:02.998420 15217 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.001147 15217 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:02.967638 15736 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:03.009917  8823 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:03.011591 15633 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:03.012135 15633 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:03.018141 15747 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:03.013214 15633 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:02.962777 15726 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.973802  8647 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:02.962649 15715 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:02.967054 15709 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:03.031790  8828 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:03.029215  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:03.039829  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 1423) took 3068 ms (client timeout 1536 ms). Trace:
W20260709 13:05:03.045007  8821 rpcz_store.cc:269] 0709 13:04:59.964746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.972826 (+  8080us) service_pool.cc:224] Handling call
0709 13:05:03.032446 (+3059620us) spinlock_profiling.cc:243] Waited 955 ms on lock 0x7b540032ee54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:03.032569 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2196267392}
I20260709 13:05:02.978791 15725 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:03.048658  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 67901.4us
I20260709 13:05:03.049721  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:03.051537 15489 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:05:03.075171  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 46067.2us
I20260709 13:05:03.076133  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:03.052011 15489 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:03.095160  8829 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:03.052001 15767 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:03.128203  8751 raft_consensus.cc:2410] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:05:03.081825 15747 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:02.975137 15390 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:03.081773 15715 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:03.131897  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:03.141881  8747 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:03.084578 15729 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.607s	user 0.017s	sys 0.000s
W20260709 13:05:03.134932  8755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35702 (request call id 1350) took 1896 ms (client timeout 1921 ms). Trace:
I20260709 13:05:03.087013 15489 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:03.084347 15681 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:05:03.137449 15645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.630s	user 0.008s	sys 0.036s
W20260709 13:05:03.137998  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 58278.6us
W20260709 13:05:03.162032  8755 rpcz_store.cc:269] 0709 13:05:01.238041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.238249 (+   208us) service_pool.cc:224] Handling call
0709 13:05:03.134676 (+1896427us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b54002ac954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:03.134858 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854052736}
I20260709 13:05:03.085172 15711 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:03.042007 15726 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.136986 15675 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:03.143090  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 169354us
I20260709 13:05:02.972523 15693 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:03.160557 15685 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:05:03.167613  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:03.139353 15747 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:03.169754  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:03.166220 15645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.663s	user 0.008s	sys 0.037s
I20260709 13:05:03.201262  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:03.184252 15766 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:03.205807  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:03.208987  8749 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:05:03.204715 15147 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:05:03.204484  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:03.209568  8747 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:03.177145 15747 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:03.208241 15715 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:03.212412  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:03.178763 15756 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:05:03.179008 15685 consensus_queue.cc:1243] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:03.215039  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:03.214932  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:03.177464 15729 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:03.191979 15716 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:03.216140 15372 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.224365 15729 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:03.224354 15372 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.177647 15681 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:03.237638  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2668) took 2682 ms. Trace:
I20260709 13:05:03.230086 15602 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:03.243460 15726 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:03.243697 15747 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.245765 15731 raft_consensus.cc:3055] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:03.245925 15465 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.246793 15676 raft_consensus.cc:3055] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:03.247227 15676 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:03.248214 15676 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:03.249989 15681 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.238540  8826 rpcz_store.cc:276] 0709 13:05:00.555390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.097556 (+542166us) service_pool.cc:224] Handling call
0709 13:05:01.097964 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:01.801171 (+703207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.237090 (+1435919us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b540041c154. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:03.237524 (+   434us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1607903488}
W20260709 13:05:03.259258  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 1.39662e+06us
I20260709 13:05:03.259498 15065 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:03.246378 15731 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:03.261389 15731 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:03.263141 15731 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:03.265094 15410 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:03.266089  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.268667  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.269423  9385 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:05:03.270057  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.270993  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:03.271363  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:03.271804  9383 raft_consensus.cc:2393] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 21: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:03.271942  9388 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:03.272691  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:03.273294  9383 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:03.273407 15696 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.275106  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.275331 15372 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:03.276711 15699 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.274243 15696 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.277354 15699 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.279197 15696 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:03.279945 15696 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:03.277444 15657 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:05:03.280607 15390 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:03.281315 15657 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:03.270490  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.281716 15657 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:03.282315  9385 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:03.282766 15657 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.283578  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:03.282430 15624 raft_consensus.cc:3055] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:03.269340  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.284643 15624 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:03.285902  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:03.281831 15410 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.285902  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:03.276584  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.285676 15624 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.289990  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:03.273480  9352 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:03.296211  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:03.296656  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1449) took 1527 ms. Trace:
I20260709 13:05:03.297171  8824 rpcz_store.cc:276] 0709 13:05:01.769387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.844946 (+ 75559us) service_pool.cc:224] Handling call
0709 13:05:01.864656 (+ 19710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.798844 (+934188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.296366 (+497522us) spinlock_profiling.cc:243] Waited 880 ms on lock 0x7b5400126454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:03.296581 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2022946560}
I20260709 13:05:03.270370  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:03.293808  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:03.279971 15699 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:03.290307  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.284224  9385 raft_consensus.cc:2376] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22. Re-sending same reply.
I20260709 13:05:03.248515 15465 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:03.250502 15676 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:03.311259  9382 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:03.225243 15729 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:03.313031  9387 raft_consensus.cc:2368] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19.
I20260709 13:05:03.314034  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.225160 15709 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:03.314253  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.298425  9352 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:03.316888 15481 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.316970 15198 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:03.316378  9385 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:03.318001 15198 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:03.317678 15481 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.320137 15594 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:03.320696 15594 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:03.303794 15724 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.318748 15720 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:05:03.318897  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.322935 15720 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.323791 15457 raft_consensus.cc:3055] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:03.324208 15457 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:03.325419 15720 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:03.325170 15457 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:03.315495  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.328812 15724 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:03.328933  9387 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:05:03.316392 15699 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:03.333736  9387 raft_consensus.cc:606] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:03.331378  8796 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:03.326486  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.325210  9383 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:03.320137 15198 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:03.352408 15198 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:03.352831  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.354228  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.312584  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.335183  9381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 982) took 1787 ms. Trace:
I20260709 13:05:03.339876 15766 raft_consensus.cc:3055] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:03.246385 15747 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:03.348390  9387 raft_consensus.cc:614] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:03.308882  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2740) took 1679 ms. Trace:
I20260709 13:05:03.351294 15753 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:03.395953 15753 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:03.312393  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.290464 15624 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:03.354934  9386 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:03.397670  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.355841  9385 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:03.356261 15633 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:03.357179  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:03.364907  9381 rpcz_store.cc:276] 0709 13:05:01.547623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.547798 (+   175us) service_pool.cc:224] Handling call
0709 13:05:03.334929 (+1787131us) spinlock_profiling.cc:243] Waited 47.2 ms on lock 0x7b54006e1754. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:03.335100 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108605312}
I20260709 13:05:03.284214 15657 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:03.322300 15481 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:03.388708  9382 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:05:03.394237  9383 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 13:05:03.394577  8756 rpcz_store.cc:276] 0709 13:05:01.629501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.909373 (+279872us) service_pool.cc:224] Handling call
0709 13:05:01.909739 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.032628 (+122889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.308532 (+1275904us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400102a54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:03.308785 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280575872}
I20260709 13:05:03.397109  9389 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 21: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:03.354790  9380 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:05:03.342109 15489 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:03.386968 15766 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:03.290027 15410 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:03.396533 15753 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:03.224599 15715 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:03.408965 15457 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:03.446204  9388 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:03.435240 15481 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.015s
I20260709 13:05:03.462544 15481 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:05:03.552088  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.209606  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:03.564580  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 354905us
W20260709 13:05:03.575820  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.567240  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:03.567008  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:05:03.580480  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.581348  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:03.575323  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:03.569504  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:03.583861  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:03.564795  8672 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:03.584187  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:03.585235  8675 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:05:03.589238  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:03.594074  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:03.596349  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2853) took 1405 ms. Trace:
I20260709 13:05:03.598302  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:03.598599  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:03.602026  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:03.608656  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1345) took 2373 ms. Trace:
I20260709 13:05:03.610932  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:03.613286  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:03.629362  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:03.635703  8573 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40096 (request call id 142) took 65344 ms (client timeout 15000 ms). Trace:
I20260709 13:05:03.636525  8749 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:05:03.641316  8822 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:03.653263  8752 raft_consensus.cc:2410] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:05:03.660472  8750 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:03.661824  8756 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:03.667425 15602 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:03.668529 15711 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:03.661072  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:03.671139  8752 raft_consensus.cc:2410] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:05:03.669200  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:03.688114 15465 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.689101 15476 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.689601 15465 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:03.690989 15701 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:03.690753 15476 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:03.692597 15701 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:03.473752 15481 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:03.688360 15133 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.697360 15697 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:03.696895 15481 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.697705 15133 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.698719 15697 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.687031  8826 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:03.669945 15711 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.705749 15133 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:03.736922  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:03.737916  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.758860 15065 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:03.759244 15489 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:03.759852  8754 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:05:03.759918 15065 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.761090  8754 raft_consensus.cc:606] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:03.761737  8754 raft_consensus.cc:614] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:03.762856  8829 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:03.766826 15065 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:03.719159 15605 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:03.768270 15605 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:03.769227  9352 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:03.749322  8751 raft_consensus.cc:2376] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21. Re-sending same reply.
I20260709 13:05:03.770380 15605 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:03.773319  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.773751  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.775218 15774 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:03.775897 15774 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.776111  8677 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:05:03.776401 15774 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:03.777798  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:03.644127  8753 rpcz_store.cc:276] 0709 13:05:01.235600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.236398 (+   798us) service_pool.cc:224] Handling call
0709 13:05:01.253723 (+ 17325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.263656 (+  9933us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:01.263693 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:01.268333 (+  4640us) log.cc:844] Serialized 26 byte log entry
0709 13:05:01.901432 (+633099us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b48001e03d8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:05:03.237046 (+1335614us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b18001bcac0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:05:03.607712 (+370666us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:03.607829 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.608006 (+   177us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.608082 (+    76us) raft_consensus.cc:1685] finished
0709 13:05:03.608097 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.608364 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9482,"raft.queue_time_us":50175,"raft.run_cpu_time_us":3480,"raft.run_wall_time_us":593275,"spinlock_wait_cycles":29627648,"thread_start_us":370088,"threads_started":1,"wal-append.queue_time_us":2198735}
I20260709 13:05:03.778220  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:03.781797  8750 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:05:03.783288  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.784902  8794 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:03.785377  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.786640  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:03.490549 15729 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:03.643705  8681 rpcz_store.cc:276] 0709 13:05:02.190504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.190769 (+   265us) service_pool.cc:224] Handling call
0709 13:05:02.191163 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.385252 (+194089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.385292 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:02.385331 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:02.385361 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.595486 (+1210125us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54003a2ad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:03.595576 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.596062 (+   486us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":445860608}
I20260709 13:05:03.807467 15481 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:03.808255 15133 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:03.809542 15720 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:03.811470  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:03.811757 15720 raft_consensus.cc:3055] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:03.812479 15720 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:03.812889  8681 raft_consensus.cc:2410] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 20 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:05:03.817356 15729 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:03.819628 15476 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:03.825364  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 236375us
I20260709 13:05:03.673374 15766 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:03.827380 15753 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:03.848016  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:03.849066  8822 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:03.847815  8574 catalog_manager.cc:5697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:03.856014  8829 raft_consensus.cc:1240] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:03.858027  8824 raft_consensus.cc:2393] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 19: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:03.860191  8673 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:03.814519 15720 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:03.872464  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:03.789783 15465 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:03.874915 15476 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:03.875015  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 416456us
I20260709 13:05:03.875779 15701 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:03.884362  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:03.885989  9380 raft_consensus.cc:2393] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:03.889001  8678 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:03.889431 15676 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.016s
I20260709 13:05:03.891502 15711 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:03.893977  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:03.896873  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:03.862433 15753 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:03.901052  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 59254.4us
I20260709 13:05:03.901346 15147 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:03.889755  8681 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:05:03.920209  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:03.921324  9380 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:03.910216  8678 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:05:03.885919  8574 catalog_manager.cc:5697] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:03.954252  8676 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:05:03.951099  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:03.953069  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 45605.2us
I20260709 13:05:03.960979  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:03.991377  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:04.014494  8747 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:05:03.649745  8573 rpcz_store.cc:269] 0709 13:03:58.290578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.291025 (+   447us) service_pool.cc:224] Handling call
0709 13:05:03.635159 (+65344134us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:57.642779 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:57.643587 (+   808us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:57.643678 (+    91us) write_op.cc:435] Acquired schema lock
0709 13:04:57.643698 (+    20us) tablet.cc:662] Decoding operations
0709 13:04:57.644959 (+  1261us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:57.645072 (+   113us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:57.645103 (+    31us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:04:57.646284 (+  1181us) tablet.cc:701] Row locks acquired
0709 13:04:57.646343 (+    59us) write_op.cc:260] PREPARE: finished
0709 13:04:57.646562 (+   219us) write_op.cc:270] Start()
0709 13:04:57.666459 (+ 19897us) write_op.cc:276] Timestamp: P: 1783602297646477 usec, L: 0
0709 13:04:57.675590 (+  9131us) op_driver.cc:348] REPLICATION: starting
0709 13:04:57.733825 (+ 58235us) log.cc:844] Serialized 4334 byte log entry
0709 13:04:57.872897 (+139072us) op_driver.cc:464] REPLICATION: finished
0709 13:04:58.145938 (+273041us) write_op.cc:301] APPLY: starting
0709 13:04:58.146067 (+   129us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:58.535473 (+389406us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:58.535651 (+   178us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:58.745248 (+209597us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:58.745688 (+   440us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 13:04:58.745776 (+    88us) write_op.cc:312] APPLY: finished
0709 13:04:58.746249 (+   473us) log.cc:844] Serialized 105 byte log entry
0709 13:04:58.770264 (+ 24015us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:58.790518 (+ 20254us) write_op.cc:454] Released schema lock
0709 13:04:58.808656 (+ 18138us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:58.844665 (+ 36009us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:58.844718 (+    53us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4038927,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":261741,"apply.run_cpu_time_us":24168,"apply.run_wall_time_us":687786,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50063,"num_ops":11,"prepare.queue_time_us":501603,"prepare.run_cpu_time_us":11738,"prepare.run_wall_time_us":107953,"raft.queue_time_us":93210,"raft.run_cpu_time_us":53027,"raft.run_wall_time_us":180911,"replication_time_us":176802,"thread_start_us":583273,"threads_started":4,"wal-append.queue_time_us":16456,"wal-append.run_cpu_time_us":4648,"wal-append.run_wall_time_us":2080983}]]}
I20260709 13:05:04.014629 15693 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:05:04.016304  8826 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:05:04.022727  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 121797us
I20260709 13:05:04.025758 15465 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:04.038260 15465 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:04.030241  8821 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
W20260709 13:05:04.039088 15657 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.623s	user 0.006s	sys 0.016s
I20260709 13:05:04.039633 15715 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:04.040297 15715 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:04.040807  8673 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:05:04.034665  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.041869  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.036314 15766 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:04.044232  8681 raft_consensus.cc:2410] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:05:04.028523 15720 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:04.041499 15715 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.045917  8681 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:05:04.046314 15633 consensus_meta.cc:287] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.634s	user 0.006s	sys 0.008s
I20260709 13:05:04.035637  8674 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:05:04.042316  8673 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:04.061693  8678 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
W20260709 13:05:04.063295  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.064402  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:04.067560  9388 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:05:04.068526  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:04.069875  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:04.073354  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:04.075191  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:04.049845  8677 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:05:04.077458  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:04.108228  8681 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:05:04.115053  9384 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:05:04.115679  9384 pending_rounds.cc:85] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting all ops after (but not including) 15
I20260709 13:05:04.115980  9384 pending_rounds.cc:107] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:05:04.116305  9384 raft_consensus.cc:2889] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:04.136089  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:04.090777 15711 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:05:04.148507  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 123076us
W20260709 13:05:04.156845  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 288273us
I20260709 13:05:04.165104 15465 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.167142 15465 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:04.170938  8677 raft_consensus.cc:1240] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:04.175146 15697 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:04.175798  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.175163 15390 consensus_meta.cc:287] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.826s	user 0.006s	sys 0.018s
I20260709 13:05:04.173292  8574 catalog_manager.cc:5697] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 13 to 16, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 16 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:04.179105  8748 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:05:04.179293  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.180491 15711 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:04.181228 15147 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.182015 15147 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:04.183926 15594 consensus_meta.cc:287] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.863s	user 0.006s	sys 0.016s
W20260709 13:05:04.184031  8566 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7aff04f53d4845a6a4fa491fd2a06145 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:04.178855  8574 catalog_manager.cc:5697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:04.184664 15594 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:04.178400  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 213129us
I20260709 13:05:04.190392  8574 catalog_manager.cc:5697] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 8 to 11, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 11 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:04.192171  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:04.184831 15147 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:04.194360  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.196031  8754 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:04.184068  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:04.197821  8749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:05:04.198794 15605 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:04.199908 15605 consensus_queue.cc:1059] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:04.200812 15605 consensus_queue.cc:1243] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:05:04.201333 15729 raft_consensus.cc:3055] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:04.201570 15476 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:04.201766 15729 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:04.152346 15693 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.202896 15693 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.203088 15729 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:04.204712 15729 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:04.205581 15693 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:04.203193  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 47180.1us
W20260709 13:05:04.206585  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.207417  8749 raft_consensus.cc:606] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:04.207903  8749 raft_consensus.cc:614] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:04.210289  8645 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:04.210891  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 52628.1us
I20260709 13:05:04.211944 15605 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:04.213312 15340 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:04.214105 15605 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:04.215945  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3144) took 1085 ms. Trace:
I20260709 13:05:04.216470  8748 rpcz_store.cc:276] 0709 13:05:03.130909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.131886 (+   977us) service_pool.cc:224] Handling call
0709 13:05:03.132203 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.179961 (+1047758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.215585 (+ 35624us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400138154. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:04.215837 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2402452864}
I20260709 13:05:04.217047  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:04.194024  8574 catalog_manager.cc:5697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:04.195200  8747 raft_consensus.cc:1240] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:05:04.227114 15481 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:04.227586 15633 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.214011 15744 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:04.232789 15198 consensus_meta.cc:287] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.880s	user 0.016s	sys 0.007s
I20260709 13:05:04.231050 15390 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:05:04.233384 15198 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:04.218139 15133 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:04.235060  9381 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:04.235981  8750 pending_rounds.cc:85] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 11
I20260709 13:05:04.236426  8750 pending_rounds.cc:107] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:05:04.232941  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:04.236753  8750 raft_consensus.cc:2889] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:04.237259  8722 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:04.238970 15681 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:04.239537 15457 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:04.241659 15457 raft_consensus.cc:697] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:04.241708  8794 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:04.243299 15693 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:04.243011 15457 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:04.243741 15693 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.244154 15693 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:04.246163  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:04.231434 15744 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.228346 15633 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:04.239656 15681 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.178997 15580 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:04.251003  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:04.255029  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:04.254838 15133 raft_consensus.cc:2764] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:05:04.259256  8681 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:04.271746  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:04.272358 15787 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.107s	user 0.003s	sys 0.000s
I20260709 13:05:04.224043 15489 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:05:04.280978 15481 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:04.281200 15697 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:04.283744 15476 consensus_queue.cc:1059] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:04.280834  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:04.298105  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.296264  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.294544  8748 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:05:04.295907  8755 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
W20260709 13:05:04.283857 15624 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.864s	user 0.012s	sys 0.007s
I20260709 13:05:04.300666  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.287657 15744 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:04.251111 15580 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.287621 15698 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.305972 15133 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.308926  8674 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 21.
I20260709 13:05:04.314453  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.316910  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:04.318014  8753 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 13:05:04.306226 15711 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:04.321190  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:04.317569  8754 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:05:04.318421  8753 pending_rounds.cc:85] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 13
I20260709 13:05:04.324172  8753 pending_rounds.cc:107] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:05:04.324558  8753 raft_consensus.cc:2889] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:04.321868 15390 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:04.327283  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:04.328223  8752 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:04.329222  8823 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:05:04.329012 15133 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.326323  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:04.332005 15133 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:04.339202 15635 raft_consensus.cc:3055] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:04.339664  9353 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:04.339922 15635 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:04.341012 15594 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:04.341168 15635 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.343327  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:04.343866  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:04.344748  8828 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:05:04.326247  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.350090  8830 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:05:04.350278  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:04.360389  8824 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:04.361853  8829 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:04.370321  8751 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:04.377341  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:04.347868 15633 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:04.393023 15635 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:04.376629  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:04.408437  8794 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:04.434216  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.436589  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:04.442576  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.446955  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:04.444922 15676 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.450898  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:04.452898  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 172255us
I20260709 13:05:04.366804 15698 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:04.454586  8756 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:04.455382  8829 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:04.413101 15198 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:04.464507  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.467749  8753 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:05:04.468708  8753 raft_consensus.cc:606] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:04.469096  8753 raft_consensus.cc:614] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:04.466712  8756 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
W20260709 13:05:04.458412 15774 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.681s	user 0.009s	sys 0.009s
I20260709 13:05:04.496825  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:04.497128 15693 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:04.498268 15720 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:04.496570 15774 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:04.513754  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.517755 15198 raft_consensus.cc:3055] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:04.518268 15198 raft_consensus.cc:740] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:05:04.518646  8755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 3146) took 1328 ms (client timeout 1537 ms). Trace:
W20260709 13:05:04.519084  8755 rpcz_store.cc:269] 0709 13:05:03.189980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.204155 (+ 14175us) service_pool.cc:224] Handling call
0709 13:05:04.518325 (+1314170us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b5400139a54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:04.518512 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1994545920}
I20260709 13:05:04.519369 15198 consensus_queue.cc:260] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:04.526240 15711 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.527150 15711 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.531620  8748 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 21: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:04.475204 15133 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:04.530076 15711 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:04.543514 15711 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.544190 15711 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.498536  8749 raft_consensus.cc:2393] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:04.498646  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:04.498664  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:04.512104  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:04.447909 15676 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:04.546271  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 146578us
I20260709 13:05:04.505952 15693 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:04.552183 15791 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:05:04.510413 15681 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:04.546835 15790 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.154s	user 0.003s	sys 0.000s
I20260709 13:05:04.624203  8748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
W20260709 13:05:04.624728  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 126210us
I20260709 13:05:04.625185  8754 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:04.549343 15580 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:04.627728  8748 raft_consensus.cc:606] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:04.628096  8793 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:04.625313 15633 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:04.640658 15720 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:04.658768  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 191471us
I20260709 13:05:04.651650 15753 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:04.642138 15580 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:04.664084 15774 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:04.651701 15633 raft_consensus.cc:697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:05:04.669449  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 53013us
I20260709 13:05:04.669696  8748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:05:04.670691  8748 raft_consensus.cc:606] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:04.668597  8675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.012s	sys 0.037s
I20260709 13:05:04.671181  8748 raft_consensus.cc:614] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:05:04.671324  8675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.013s	sys 0.037s
I20260709 13:05:04.682407  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2873) took 1039 ms. Trace:
I20260709 13:05:04.684749  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:04.685242  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:05:04.687493  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 475759us
I20260709 13:05:04.691604  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:04.719359  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:04.723606  8672 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:04.702884 15720 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:04.726418  8647 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.037s	sys 0.033s
I20260709 13:05:04.703682 15676 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:04.730177  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 46814.1us
W20260709 13:05:04.731031  8647 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.037s	sys 0.033s
W20260709 13:05:04.724663 15657 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.017s	sys 0.011s
W20260709 13:05:04.733081  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 647295us
W20260709 13:05:04.737366  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.726944 15711 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:04.736263 15657 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.017s	sys 0.012s
I20260709 13:05:04.724179 15698 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:04.732215 15720 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:04.739689 15635 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:04.741415 15676 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.765180  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:04.768812  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 97264.9us
W20260709 13:05:04.771284  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.741429 15481 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:04.745182 15729 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:04.776027  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.777376  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 45760.6us
W20260709 13:05:04.782861  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.778635  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.778815  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.760974 15720 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:04.762648 15711 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:04.788198  8794 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:04.764853 15465 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.787184  8751 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:05:04.791132  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 521386us
I20260709 13:05:04.792793  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.776736 15635 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:04.781268 15624 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.773734 15676 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.802829  8748 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:05:04.760372 15633 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:04.816314  8754 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 20. Current term is 21. Ops: [19.12-20.13]
I20260709 13:05:04.819188 15635 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:04.787634 15711 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.841941  8722 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:04.849671  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:04.880776  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 541243us
I20260709 13:05:04.801133 15465 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.883564  8749 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:05:04.884605 15481 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.885093 15465 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:04.876127 15753 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:05:04.888748 15476 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:04.889842 15796 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.890410  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.890360 15796 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:04.894873  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.897193  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 96437.7us
I20260709 13:05:04.821065 15766 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.904904 15799 consensus_queue.cc:1059] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:04.802453 15729 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.791213 15698 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 13:05:04.892782 15711 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:04.899266  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 107237us
I20260709 13:05:04.682971  8675 rpcz_store.cc:276] 0709 13:05:03.657190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.657357 (+   167us) service_pool.cc:224] Handling call
0709 13:05:03.746930 (+ 89573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.748069 (+  1139us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:04.053477 (+305408us) log.cc:844] Serialized 14 byte log entry
0709 13:05:04.681954 (+628477us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:04.682035 (+    81us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:04.682095 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.682182 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:04.682330 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28285,"prepare.queue_time_us":480285,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":331,"raft.queue_time_us":8561,"thread_start_us":866432,"threads_started":2}
W20260709 13:05:04.913507  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 222956us
I20260709 13:05:04.893180 15796 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:04.874982 15657 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:04.931335  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 81776.3us
I20260709 13:05:04.935815  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:04.910300 15465 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:04.940840 15729 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:04.931209 15476 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:05:04.944700  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:04.923319 15390 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:04.902520 15766 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.857950 15624 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:04.956972  8672 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:04.953163  8647 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:04.937668 15133 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:04.966135 15802 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.970554  8756 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:05:04.970963  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 168658us
I20260709 13:05:04.972512  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.976365  8747 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:05:04.970957  8751 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:05:04.978132  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.975147  8756 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:04.973538  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.970113  8748 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:04.983300  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:04.987185  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 69302.8us
I20260709 13:05:04.960752 15799 raft_consensus.cc:3055] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:04.967763 15624 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:04.987669 15729 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:04.995507  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:04.996496  9353 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:04.967469 15797 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:05:04.998667  8681 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:05:05.002509  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 234762us
I20260709 13:05:05.005170  8749 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:05.004771  8752 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
W20260709 13:05:04.987550 15792 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:05:04.966482 15390 raft_consensus.cc:697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:05.013080  8794 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:05.015578  8677 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:05.010949  8676 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:05.014089 15814 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.017853  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:05.018689  8680 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:05.018536 15814 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.021276 15814 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:05.023216 15624 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.023931 15624 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.027637  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:05.016619  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:05.027072  8756 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.008522 15729 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.035347 15729 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:05.043272  8675 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:05:05.043922  9355 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:05.043134 15819 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:04.966399 15806 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.156s
I20260709 13:05:05.045173 15390 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.046293 15806 consensus_queue.cc:1243] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:05:05.033929 15624 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:05.047016  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1451) took 1193 ms. Trace:
I20260709 13:05:05.047315 15817 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:04.965094 15476 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:05:05.047621  8822 rpcz_store.cc:276] 0709 13:05:03.853814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.909290 (+ 55476us) service_pool.cc:224] Handling call
0709 13:05:03.935891 (+ 26601us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:03.997074 (+ 61183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:03.997126 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:04.106647 (+109521us) log.cc:844] Serialized 52 byte log entry
0709 13:05:04.649059 (+542412us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:04.649124 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.649289 (+   165us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:05.046719 (+397430us) raft_consensus.cc:1685] finished
0709 13:05:05.046745 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.046927 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25281,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":90445,"thread_start_us":471989,"threads_started":1,"wal-append.queue_time_us":640604}
I20260709 13:05:05.048946 15133 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:05.049728 15766 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:05.047065 15806 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:04.982409 15802 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.050836 15806 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:05.052357 15806 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:05.052515 15802 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:04.999334 15799 raft_consensus.cc:740] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:05.053973  8678 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:05:05.054841 15676 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:05.055604 15809 consensus_queue.cc:1059] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 13:05:05.057192  8756 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:05.056000  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 56938.7us
I20260709 13:05:05.059612  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:05.049453 15476 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:04.992316 15796 consensus_queue.cc:1059] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
W20260709 13:05:05.063378  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 80016.9us
I20260709 13:05:05.064759  8722 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:05.066680 15624 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:05.069211  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:05.069684  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:05.070993  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:05.049743 15133 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:05.074534 15133 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.075089  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:05.075932  8672 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:05.076112 15133 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:05.066711 15804 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.077579 15698 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:05.075807 15676 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:05.078552 15804 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.078478 15457 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:05:05.080416 15457 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:05.081449 15457 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.083385 15457 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:05.089608 15801 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:05:05.081553  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:05.110400  8750 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:05.113731  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:05.117084  8678 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
W20260709 13:05:05.117287  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 97926.1us
I20260709 13:05:05.117702  8678 pending_rounds.cc:85] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 8
I20260709 13:05:05.118104  8678 pending_rounds.cc:107] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:05:05.118412  8678 raft_consensus.cc:2889] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:05.081416 15804 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:05.116465  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 103948us
I20260709 13:05:05.121354  8756 raft_consensus.cc:1275] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:05:05.138904  8756 pending_rounds.cc:85] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 11
I20260709 13:05:05.138752  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:05.139251  8756 pending_rounds.cc:107] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:05:05.054029 15799 consensus_queue.cc:260] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.139562  8756 raft_consensus.cc:2889] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:05.109231 15766 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:05.146675  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 98190.3us
W20260709 13:05:05.146682  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 102879us
I20260709 13:05:05.148555  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:05.149688  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 133210us
W20260709 13:05:05.152194  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 44047.8us
I20260709 13:05:05.153141  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:05.109540 15635 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:05.137614 15814 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:05.155251  8794 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:05.119246 15624 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.156576  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:05.145016 15693 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:05.119441 15465 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:05:05.162130  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 92583.1us
I20260709 13:05:05.140738 15594 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:05.167882  8719 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:05.173417  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:05.149612 15676 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.176340  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:05.176388  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:05.180608  8647 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:05.161787 15476 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:05.163645 15809 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:05.183547 15828 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:05.171449 15693 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.185199 15795 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.162995 15635 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:05.158110 15657 raft_consensus.cc:3055] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:05.163486 15804 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.139283 15823 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:05:05.178496 15698 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:05.179971 15624 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.174511 15481 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:05:05.205173  9355 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:05.175046 15065 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:05:05.192076 15809 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:05.178042 15594 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.182633 15676 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.195214 15766 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:05.199225 15799 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:05.195514 15693 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:05.204645 15681 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:05.228466  9353 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:05.204557 15818 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:05.179930 15711 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.204346 15603 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.197270 15635 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:05.229609  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:05.196645 15657 raft_consensus.cc:740] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:05.197217 15147 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:05:05.238013  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.211903 15698 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.190776 15465 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.216451 15476 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:05.216882 15796 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:05.234920 15807 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:05.234882 15830 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.221201 15828 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:05.223588 15809 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:05.248871  8679 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:05.217921 15795 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:05.250161  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 69697us
W20260709 13:05:05.251950  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46947.1us
I20260709 13:05:05.234728 15808 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:05.256348 15724 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:05:05.256814 15825 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:05.296641  8827 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:05.256819 15657 consensus_queue.cc:260] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.298852  8821 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:05:05.299306  8821 pending_rounds.cc:85] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 13
I20260709 13:05:05.299618 15657 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:05.205392 15814 consensus_queue.cc:1059] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:05:05.257679 15594 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:05.241698 15681 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:05.301198 15808 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.302440 15681 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.307415 15681 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:05.308085 15465 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:05.308851 15465 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:05.309401 15465 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.313896 15198 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:05.316002 15676 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:05.315685 15693 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.291153 15476 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.317680 15476 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.318032 15693 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:05.318502  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 99032.9us
W20260709 13:05:05.249619  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 77460us
I20260709 13:05:05.266222 15603 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.273962 15834 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:05:05.251157 15729 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.287076 15744 raft_consensus.cc:3055] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:05.236263 15804 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.253113 15698 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:05.257941 15816 log.cc:927] Time spent T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
W20260709 13:05:05.295516 15820 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:05:05.299593  8821 pending_rounds.cc:107] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:05:05.302290 15814 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:05.325145  8680 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
W20260709 13:05:05.331148  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.340619  8821 raft_consensus.cc:2889] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:05.319962 15793 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.207s	user 0.001s	sys 0.001s
I20260709 13:05:05.267642 15715 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 13:05:05.363515  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:05.371368  8722 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:05.303781 15808 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:05.377038  8647 leader_election.cc:400] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19.
I20260709 13:05:05.253446 15818 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.271335 15720 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.332891 15744 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:05.269224 15819 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.104s
I20260709 13:05:05.362118 15476 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:05.387974  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:05.401446  8827 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 21.
W20260709 13:05:05.387320  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 204848us
W20260709 13:05:05.392576  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:05.403582  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:05.419806  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 48550.4us
I20260709 13:05:05.377646 15147 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:05.289721 15830 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.388032 15711 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:05.436561  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:05.438895  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:05.447996  9381 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:05.418145 15766 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:05.440662 15830 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:05.474278  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 266857us
I20260709 13:05:05.440948  9384 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:05:05.442281  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:05.443739  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:05:05.446764  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.452592 15832 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:05.448196  9387 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:05.453928 15693 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:05.455683 15729 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:05.475668  9383 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:05.460474 15198 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:05.462596  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:05.462879 15147 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.403621 15840 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.467113  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:05.475078  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 253112us
I20260709 13:05:05.484505  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:05.506955  8794 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:05.526758  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:05.528297  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:05.529228  8829 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:05.533241  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:05.533977  8680 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:05.535305  8827 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 20: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:05.537886  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:05.539439  8823 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:05.474738 15796 raft_consensus.cc:3055] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:05:05.543759  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 122764us
I20260709 13:05:05.541222  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:05.543244  8821 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 13:05:05.545691  8821 pending_rounds.cc:85] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 12
I20260709 13:05:05.546005  8821 pending_rounds.cc:107] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:05:05.546444  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:05.546819  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:05.540550  8826 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:05.546267  8821 raft_consensus.cc:2889] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:05.546420  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:05.545250  8822 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:05:05.548305  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:05.549754  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:05.504127 15802 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.547266  8674 raft_consensus.cc:2376] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14. Re-sending same reply.
I20260709 13:05:05.550520  8676 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
W20260709 13:05:05.551760  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 231971us
W20260709 13:05:05.561071  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 221891us
I20260709 13:05:05.563879  8828 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:05.563501  8796 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:05.571038  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:05.572011  8680 raft_consensus.cc:2410] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 21 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:05:05.573287  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:05.575090  9381 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:05.542871 15624 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:05.580096  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:05.580441  8823 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:05:05.580862  8747 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 22.
I20260709 13:05:05.582247  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:05.582258  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:05.581123  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:05.584090  8675 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:05:05.586825  8827 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:05.545060 15796 raft_consensus.cc:740] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:05:05.587438  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 84130.9us
I20260709 13:05:05.589854  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:05.585552  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:05.592305  8824 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:05.593722  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:05.583088  8822 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:05.597559 15836 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:05.598194 15836 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.570502 15729 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.592667  8794 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:05.569073 15633 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:05.570050 15465 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:05.598577 15836 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:05.551859 15603 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:05.602903 15390 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:05:05.588270 15133 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:05.641247  8722 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:05.645629  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:05.646471  8822 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:05.648373  9385 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:05:05.573382 15802 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.652004  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:05.652226  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47018.1us
I20260709 13:05:05.653671  9355 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:05.607203 15817 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.607509 15624 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:05.645696 15133 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.573340 15840 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.648324 15830 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:05:05.674928  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 93864.2us
I20260709 13:05:05.674958  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:05.677459  9355 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:05.612665 15729 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.687613  9386 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:05.638731 15681 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
W20260709 13:05:05.694931  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.648701 15603 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 13:05:05.679432 15133 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:05.710268  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:05.711102  8676 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:05.679706 15804 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:05.683369 15795 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:05.722085  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 77753.4us
I20260709 13:05:05.724536  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:05.705725 15830 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:05.663542 15817 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.700662 15720 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:05.740275 15795 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:05.743042  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:05.743175 15744 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.743973  8823 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:05:05.744622 15744 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:05.747097  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:05.747560  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:05.750866  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 182028us
I20260709 13:05:05.759456  8677 raft_consensus.cc:2410] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
W20260709 13:05:05.760520  8566 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7aff04f53d4845a6a4fa491fd2a06145 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:05.760480  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:05.766956  8678 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
W20260709 13:05:05.839697  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 72144.2us
I20260709 13:05:05.844470  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:05.850833  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 115702us
I20260709 13:05:05.851964  8722 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:05.743176 15795 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.845358  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:05.752686 15818 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:05.868333 15720 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:05.868605 15853 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:05.869374  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:05.873047  8676 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:05.878696  8749 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:05.748046 15841 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:05:05.856107 15825 consensus_meta.cc:287] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.552s	user 0.014s	sys 0.000s
I20260709 13:05:05.700232 15796 consensus_queue.cc:260] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:05.729370 15833 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.261s
I20260709 13:05:05.881599  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:05.882076  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:05.892129  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:05.893784  8676 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 13:05:05.899124  8676 pending_rounds.cc:85] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 12
I20260709 13:05:05.899897  8676 pending_rounds.cc:107] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:05:05.900259  8676 raft_consensus.cc:2889] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:05.901175  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:05.874462 15795 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:05.898901  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 284254us
I20260709 13:05:05.878127 15720 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.961407  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:05.908674 15846 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:05.921916 15818 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:05.874032 15624 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:05:05.899895  9352 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:05.901896 15825 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:05.949724  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:05.942395 15635 consensus_meta.cc:287] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.689s	user 0.012s	sys 0.002s
I20260709 13:05:05.883149 15476 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:05.957181  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:05.961158  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:05.881778 15841 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:05.963892  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:05.969246 15720 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:05.972100  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:06.011833  8676 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:05.980078  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:05.946380 15711 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:05.928956 15465 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 13:05:05.905211 15832 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.051s
W20260709 13:05:05.996033  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.968833 15846 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.907202 15847 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.324s
I20260709 13:05:06.015762  9381 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:06.013144  8824 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
W20260709 13:05:06.036123  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 79007.2us
I20260709 13:05:05.994432  8825 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:05:05.974807 15766 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:05.976961 15676 consensus_queue.cc:1059] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 13:05:06.004212 15457 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:06.016381 15729 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:05.942028 15830 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:06.005115 15802 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:06.003710 15841 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:05.993016 15635 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:06.035421 15846 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:06.078356  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:06.003440 15476 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:05.972543 15848 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.304s
I20260709 13:05:06.091045  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:06.065706 15825 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:06.095777  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:06.071347 15457 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.992920 15818 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:06.076804 15840 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:06.025691 15849 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.347s
I20260709 13:05:06.035682 15711 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:06.085266 15817 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:06.087225 15839 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.106015  8824 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
W20260709 13:05:06.101428  8830 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.011s	sys 0.005s
I20260709 13:05:06.117714  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:06.076249 15766 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:06.113610 15457 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:06.110080 15874 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:06.181022  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:06.184980  8676 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:06.182103  8824 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
W20260709 13:05:06.192060  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 292253us
I20260709 13:05:06.115049  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:06.138878  8827 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
W20260709 13:05:06.138127  8830 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.011s	sys 0.006s
I20260709 13:05:06.131521 15465 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:06.149802 15855 consensus_queue.cc:1059] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.303s
W20260709 13:05:06.223836  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 132917us
I20260709 13:05:06.225487  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:06.151388 15866 log.cc:927] Time spent T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:05:06.158468 15818 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:06.159268 15868 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.213s
I20260709 13:05:06.162799 15843 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.343312  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:06.085062 15857 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.152s
I20260709 13:05:06.098172 15851 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.382s
I20260709 13:05:06.125195 15873 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:06.165524 15852 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.318s	user 0.001s	sys 0.001s
W20260709 13:05:06.208110  8748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.008s	sys 0.005s
I20260709 13:05:06.127738 15825 raft_consensus.cc:697] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:06.132275 15817 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:05:06.147397  8822 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.500s	user 0.015s	sys 0.000s
I20260709 13:05:06.411582  8822 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:06.412526  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:06.233857  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 381883us
I20260709 13:05:06.413985  8722 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:06.415642  8722 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:06.239377 15880 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.298969  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:06.301026 15839 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:06.334851 15580 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.431376  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:06.180088 15874 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:06.115137 15872 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 13:05:06.373705  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:06.388947  8748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.008s	sys 0.006s
W20260709 13:05:06.236716 15799 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.006s	sys 0.010s
I20260709 13:05:06.411973  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:06.423800  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:06.425164  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 382909us
I20260709 13:05:06.163527 15835 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:06.432971  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:06.436862  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:05:06.437253  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 211830us
W20260709 13:05:06.437628  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.442854  9385 raft_consensus.cc:2393] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:06.454959  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:06.459586  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:06.451344  8676 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 21.
W20260709 13:05:06.453822  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.454527  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:06.452638  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:06.453634 15580 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:06.403988 15873 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:06.470780  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 506893us
I20260709 13:05:06.471727  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:06.472148 15580 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:06.472759 15873 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:06.427103  8645 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.502s (SENT) [suppressed 35 similar messages]
I20260709 13:05:06.469732  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:06.460827  8824 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:06.417531 15729 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:06.496073  8754 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.019s	sys 0.026s
W20260709 13:05:06.518297  8754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.019s	sys 0.027s
I20260709 13:05:06.502373  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:06.508073  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:06.466250  9387 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
W20260709 13:05:06.461730 15799 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.006s	sys 0.011s
W20260709 13:05:06.462132 15657 consensus_meta.cc:287] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.162s	user 0.016s	sys 0.001s
I20260709 13:05:06.494552  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:06.403072 15840 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:06.514381  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1466) took 1013 ms. Trace:
I20260709 13:05:06.453634 15804 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:06.421023 15802 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:06.528571  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:06.376080 15843 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:06.453930 15481 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:06.443912 15880 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:06.473122 15835 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:06.539594  8749 consensus_meta.cc:287] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.646s	user 0.014s	sys 0.000s
I20260709 13:05:06.737566  8749 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:06.539918 15580 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:05:06.739289  8794 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:06.560658  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 66472.3us
I20260709 13:05:06.740479  8647 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:06.741781  8753 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:06.642241  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 184622us
I20260709 13:05:06.743280  9381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 1006) took 1142 ms. Trace:
I20260709 13:05:06.743709  9381 rpcz_store.cc:276] 0709 13:05:05.601145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.601335 (+   190us) service_pool.cc:224] Handling call
0709 13:05:05.635036 (+ 33701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:06.036949 (+401913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.743012 (+706063us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54005b13d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:06.743195 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":874917760}
I20260709 13:05:06.455719 15874 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:06.742163 15901 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:06.746562  8753 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:05:06.742265  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:06.649912  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3109) took 1068 ms. Trace:
I20260709 13:05:06.710602  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2714) took 1237 ms. Trace:
I20260709 13:05:06.688022 15892 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.755805  8821 rpcz_store.cc:276] 0709 13:05:05.472925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.473058 (+   133us) service_pool.cc:224] Handling call
0709 13:05:05.526126 (+ 53068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.562912 (+ 36786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.710158 (+1147246us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b5400127354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:06.710495 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38450944}
I20260709 13:05:06.665213  9386 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
I20260709 13:05:06.765012 15855 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:06.751809 15340 raft_consensus.cc:3055] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:06.753161 15847 raft_consensus.cc:3055] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:06.731562 15711 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:06.755668  8679 rpcz_store.cc:276] 0709 13:05:05.581704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.582001 (+   297us) service_pool.cc:224] Handling call
0709 13:05:05.599267 (+ 17266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:05.626901 (+ 27634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:05.626929 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:05.651787 (+ 24858us) log.cc:844] Serialized 26 byte log entry
0709 13:05:05.730959 (+ 79172us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:05.731010 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.731100 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:06.649510 (+918410us) raft_consensus.cc:1685] finished
0709 13:05:06.649547 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.649801 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4678,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":42861,"spinlock_wait_cycles":1620864,"thread_start_us":44040,"threads_started":1,"wal-append.queue_time_us":973979}
I20260709 13:05:06.774293 15340 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:06.739181 15840 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:06.557319 15896 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:06.788833 15896 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.789296 15896 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:06.771112 15886 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:06.790679 15720 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.761s	user 0.017s	sys 0.000s
I20260709 13:05:06.790879 15886 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:06.793504 15835 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:06.795733 15887 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.790302 15840 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:06.598531 15885 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:05:06.645920 15698 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.303s	user 0.013s	sys 0.004s
W20260709 13:05:06.598163  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 126489us
W20260709 13:05:06.719970  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 292840us
W20260709 13:05:06.763247 15814 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.399s	user 0.014s	sys 0.003s
I20260709 13:05:06.584968 15147 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:06.598124 15854 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.262s	user 0.000s	sys 0.002s
W20260709 13:05:06.600119 15869 log.cc:927] Time spent T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.431s	user 0.000s	sys 0.001s
I20260709 13:05:06.533021 15465 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:06.766700 15809 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.426s	user 0.014s	sys 0.005s
I20260709 13:05:06.720250 15766 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:06.597820 15898 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:06.668143 15624 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:06.602957 15635 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:06.774230 15899 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:05:06.779801 15744 consensus_meta.cc:287] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.035s	user 0.018s	sys 0.000s
I20260709 13:05:06.787220 15340 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:06.765175 15804 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:06.722635 15476 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:06.790230 15901 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:06.791275  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:06.791173 15720 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:06.689304 15882 log.cc:927] Time spent T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.488s	user 0.002s	sys 0.000s
I20260709 13:05:06.756628  8830 rpcz_store.cc:276] 0709 13:05:05.500582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.501301 (+   719us) service_pool.cc:224] Handling call
0709 13:05:05.537030 (+ 35729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.537326 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:06.208308 (+670982us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.513160 (+304852us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.513360 (+   200us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:06.513412 (+    52us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:06.513473 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.513916 (+   443us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.514202 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26832,"prepare.queue_time_us":551426,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":100,"raft.queue_time_us":90078,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":39870,"spinlock_wait_cycles":25599872,"thread_start_us":921445,"threads_started":2,"wal-append.queue_time_us":513098}
W20260709 13:05:06.597960 15837 log.cc:927] Time spent T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 13:05:06.559988  9380 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
W20260709 13:05:06.646036 15867 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.404s	user 0.001s	sys 0.000s
W20260709 13:05:06.799645  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 347041us
I20260709 13:05:06.766098 15894 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:06.563237 15633 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:06.767050 15681 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.654389  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:06.804733  9384 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:06.889564  8750 raft_consensus.cc:2410] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:05:06.803087  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:06.806823  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:06.779945 15133 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.051s	user 0.008s	sys 0.007s
W20260709 13:05:06.787719 15836 consensus_meta.cc:287] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.135s	user 0.010s	sys 0.005s
I20260709 13:05:06.594408 15879 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.552503  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:06.818395  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1367) took 1252 ms. Trace:
I20260709 13:05:06.779896 15847 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:06.803026 15851 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.803591 15885 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.537475 15729 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:06.597780 15825 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:06.840014  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:06.921206  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 148995us
I20260709 13:05:06.810988 15814 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:06.872068 15624 raft_consensus.cc:697] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:06.796444 15887 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:06.894134 15633 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.835182 15147 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.802145 15802 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:06.764961 15892 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:06.806908 15698 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:06.916232  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 113320us
I20260709 13:05:06.950600  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:06.828696 15886 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:06.870744  8827 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:05:06.835886 15804 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:06.872784 15766 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:06.817005 15465 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:06.779757 15481 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:06.922410  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:06.938131  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:06.960889  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:06.923799 15711 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:06.825567 15809 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:06.909660 15340 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:06.953784  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3110) took 1351 ms. Trace:
I20260709 13:05:06.909068 15894 raft_consensus.cc:697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:06.959105  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:06.905862  8748 rpcz_store.cc:276] 0709 13:05:05.566177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.566719 (+   542us) service_pool.cc:224] Handling call
0709 13:05:05.589182 (+ 22463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.589408 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:05.619153 (+ 29745us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.817836 (+1198683us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:06.817919 (+    83us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:06.817990 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.818093 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.818299 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50527,"prepare.queue_time_us":29772,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":1534,"raft.queue_time_us":146750,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":74,"thread_start_us":1086264,"threads_started":2,"wal-append.queue_time_us":914464}
I20260709 13:05:06.933668 15633 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:06.905350 15681 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:06.828430 15898 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:06.972076  8675 rpcz_store.cc:276] 0709 13:05:05.602346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.602671 (+   325us) service_pool.cc:224] Handling call
0709 13:05:05.620119 (+ 17448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:05.718431 (+ 98312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:05.718483 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:05.719209 (+   726us) log.cc:844] Serialized 26 byte log entry
0709 13:05:05.969789 (+250580us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:05.969854 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.739037 (+769183us) spinlock_profiling.cc:243] Waited 97.7 ms on lock 0x7b54003395d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:06.739089 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:06.940472 (+201383us) raft_consensus.cc:1685] finished
0709 13:05:06.940504 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.953668 (+ 13164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1599,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":11694,"spinlock_wait_cycles":224718208,"thread_start_us":212219,"threads_started":1,"wal-append.queue_time_us":343997}
I20260709 13:05:06.903888 15133 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:06.918708 15885 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:06.972805  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:06.980284  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.977275  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3174) took 1117 ms. Trace:
I20260709 13:05:06.991855  8751 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:06.978324  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3113) took 1305 ms. Trace:
I20260709 13:05:06.983903  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:06.960845 15804 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:06.906020 15836 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:06.999019  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:06.998180  9388 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:07.003106  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:06.979095 15830 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:06.992007  8752 rpcz_store.cc:276] 0709 13:05:05.860076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.860225 (+   149us) service_pool.cc:224] Handling call
0709 13:05:05.868949 (+  8724us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:05.901985 (+ 33036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:05.902033 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:05.942535 (+ 40502us) log.cc:844] Serialized 26 byte log entry
0709 13:05:06.083070 (+140535us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:06.089304 (+  6234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.089571 (+   267us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:06.977043 (+887472us) raft_consensus.cc:1685] finished
0709 13:05:06.977070 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.977207 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32148,"raft.queue_time_us":151524,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":18066,"thread_start_us":125357,"threads_started":2,"wal-append.queue_time_us":171803}
I20260709 13:05:06.992408  8674 rpcz_store.cc:276] 0709 13:05:05.673039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.673378 (+   339us) service_pool.cc:224] Handling call
0709 13:05:05.684998 (+ 11620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:05.685459 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:05.685494 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:05.698396 (+ 12902us) log.cc:844] Serialized 26 byte log entry
0709 13:05:05.876548 (+178152us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:05.876593 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.876664 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:06.978036 (+1101372us) raft_consensus.cc:1685] finished
0709 13:05:06.978070 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.978243 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6285,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":5028,"thread_start_us":135397,"threads_started":1,"wal-append.queue_time_us":387250}
I20260709 13:05:06.920589 15851 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:06.922863 15899 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.013185  8826 raft_consensus.cc:2393] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 22: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:05:07.002878 15795 consensus_meta.cc:287] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.019s	user 0.008s	sys 0.008s
W20260709 13:05:07.005865 15457 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.774s	user 0.018s	sys 0.000s
I20260709 13:05:07.016846  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:06.998433 15711 raft_consensus.cc:697] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:07.019249 15907 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.019481  8822 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:06.955677 15147 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:07.007671 15841 consensus_meta.cc:287] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.903s	user 0.007s	sys 0.008s
I20260709 13:05:07.013434 15744 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:06.921653 15879 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.026453  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.011152 15830 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:07.030568  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:06.975955 15766 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.039366  8753 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:07.041301  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3176) took 1180 ms. Trace:
I20260709 13:05:06.990208 15874 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:07.092069  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 169727us
I20260709 13:05:07.044242 15899 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.055081 15912 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:07.042924 15851 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.055614 15907 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.045890 15802 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:07.029887 15476 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:05:07.093243 15846 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.981s	user 0.013s	sys 0.001s
I20260709 13:05:07.092304 15925 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:07.096336  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.050356  8754 rpcz_store.cc:276] 0709 13:05:05.860913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.861021 (+   108us) service_pool.cc:224] Handling call
0709 13:05:05.883986 (+ 22965us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:06.519067 (+635081us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:06.519119 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:06.557379 (+ 38260us) log.cc:844] Serialized 26 byte log entry
0709 13:05:06.812780 (+255401us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:06.812826 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.812926 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:07.040961 (+228035us) raft_consensus.cc:1685] finished
0709 13:05:07.041004 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:07.041222 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":202799,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":4555,"thread_start_us":788822,"threads_started":2,"wal-append.queue_time_us":313917}
I20260709 13:05:07.049439 15817 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:07.044574 15830 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:07.094038  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:07.095590  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 65161.5us
I20260709 13:05:07.044226 15836 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:07.100350 15580 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:06.994508 15847 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.044634 15481 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:07.136159  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:07.040990 15879 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:07.027527 15457 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.040750 15880 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:07.028851 15795 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.101051 15899 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.121338  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.102401 15892 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.056687 15841 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.134335  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.041818 15711 consensus_queue.cc:237] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.120762 15580 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.110203 15925 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.027760 15624 consensus_queue.cc:237] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.137171  8647 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:07.112074 15846 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.119586 15720 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:07.159698  8754 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 13:05:07.148521 15814 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.167043  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:07.192679  8755 raft_consensus.cc:2376] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17. Re-sending same reply.
I20260709 13:05:07.146397 15481 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.108049 15912 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.191969  8754 pending_rounds.cc:85] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 14
I20260709 13:05:07.149348 15898 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.117084 15817 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.149184 15907 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:07.185616 15918 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:05:07.159536 15744 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:07.150192 15681 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.190163  8752 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:05:07.149930 15698 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.203979 15133 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:07.192524  8753 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:07.144383 15476 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.148804 15887 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.149711 15729 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.194255  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:07.254913 15916 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:05:07.256124 15841 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.257686 15912 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.244889 15814 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.260411 15795 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.261196 15926 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:07.261829 15926 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.262001 15814 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:07.262243 15926 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:07.263180 15147 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:07.264415  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 70260.3us
I20260709 13:05:07.265300  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:07.167997 15580 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:07.175824 15809 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:07.193325 15836 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:05:07.270104 15896 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:07.237321  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 107822us
I20260709 13:05:07.278573 15147 consensus_queue.cc:1059] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:05:07.279912  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.280464  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.281287  8645 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:07.164043 15879 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:07.205233  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 89713.6us
I20260709 13:05:07.167634 15843 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:07.292587  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:07.164242 15899 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.175590 15925 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:07.204602  8754 pending_rounds.cc:107] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:05:07.299274  8754 raft_consensus.cc:2889] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:07.204923 15924 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.239305 15802 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.241158 15846 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:07.250849 15911 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:05:07.256157 15916 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:07.266773 15744 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:07.338229 15744 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:07.250334 15476 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.338879 15744 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.340512 15476 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:07.174232 15914 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:05:07.341022 15744 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:07.342429  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:07.343001 15899 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:07.104413 15894 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.343647  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:07.344206  9385 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:07.289606  9384 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:05:07.267195  9389 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:07.291958  9382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 1004) took 1847 ms. Trace:
I20260709 13:05:07.348397  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:07.349120  9384 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:07.290349 15766 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.303807 15857 raft_consensus.cc:993] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: : Instructing follower 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:05:07.305001 15457 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.209578 15481 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.314949 15874 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.279901 15729 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:07.355588 15729 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:07.356734 15729 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.358086 15729 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:07.249136 15465 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.316308  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:07.317164 15928 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:05:07.319765  8566 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7aff04f53d4845a6a4fa491fd2a06145 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:07.336066 15802 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.360179 15928 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:07.360621 15928 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:07.360692 15857 raft_consensus.cc:1081] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Signalling peer 0e8c59a1f0394eae8ce33b99f10efadc to start an election
I20260709 13:05:07.361074 15928 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.354462 15481 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:07.361975 15802 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.319115  8647 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:07.327622 15927 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:07.363037  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 44019.1us
I20260709 13:05:07.363430 15927 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:07.363876 15927 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.364228 15927 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:07.294164 15147 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:07.290719 15879 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.297101 15836 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:07.337210 15896 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.338336 15916 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:07.284082  8796 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:07.375082  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:07.375085  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:07.375936  9384 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:07.376116  8830 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:05:07.301180  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:07.347817  9382 rpcz_store.cc:276] 0709 13:05:05.444047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.444203 (+   156us) service_pool.cc:224] Handling call
0709 13:05:05.444711 (+   508us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:05.576828 (+132117us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:05.576863 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:05.617957 (+ 41094us) log.cc:844] Serialized 52 byte log entry
0709 13:05:05.703467 (+ 85510us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:05.703906 (+   439us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.739274 (+ 35368us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.761194 (+ 21920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.761282 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:07.275888 (+1514606us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b1800311b00. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:05:07.291586 (+ 15698us) raft_consensus.cc:1685] finished
0709 13:05:07.291619 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:07.291861 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113754,"prepare.queue_time_us":18394,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":78840,"raft.queue_time_us":307880,"raft.run_cpu_time_us":474,"raft.run_wall_time_us":102474,"spinlock_wait_cycles":16682112,"thread_start_us":147603,"threads_started":2,"wal-append.queue_time_us":86492}
I20260709 13:05:07.294816 15340 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.379302  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:07.369630  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:07.334146 15924 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.395521  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:07.373468  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3171) took 1515 ms. Trace:
I20260709 13:05:07.381278  8825 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 25: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:07.377596  9383 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:05:07.382699 15916 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.383746 15926 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.386888 15340 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.387331  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:07.378973 15836 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:07.393106  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1468) took 1650 ms. Trace:
W20260709 13:05:07.395594  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 111564us
I20260709 13:05:07.371450  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:07.405534  8830 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:07.371379 15879 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:07.397917  8756 rpcz_store.cc:276] 0709 13:05:05.858124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.858258 (+   134us) service_pool.cc:224] Handling call
0709 13:05:05.858702 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:05.985378 (+126676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:05.985418 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:05.994180 (+  8762us) log.cc:844] Serialized 26 byte log entry
0709 13:05:07.108332 (+1114152us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b18004dffe0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:05:07.244960 (+136628us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:07.245004 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.245074 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:07.351935 (+106861us) raft_consensus.cc:1685] finished
0709 13:05:07.352073 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:07.373386 (+ 21313us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12143,"raft.queue_time_us":166140,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":20567,"spinlock_wait_cycles":2579840,"thread_start_us":233902,"threads_started":2,"wal-append.queue_time_us":1306756}
I20260709 13:05:07.406356 15830 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.411131 15924 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.408412  8823 rpcz_store.cc:276] 0709 13:05:05.742305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.391987 (+1649682us) service_pool.cc:224] Handling call
0709 13:05:07.392427 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:07.392662 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:07.392699 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:07.392761 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:07.392808 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.392881 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:07.393026 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:07.422442  9383 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:05:07.369510 15147 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:07.407763  9382 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
I20260709 13:05:07.414567  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:07.417618  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:07.422474  8796 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:07.429117  8796 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:07.416024 15920 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:07.437605 15920 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.438761 15830 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:07.440160 15920 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:07.440929 15580 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.374410 15766 consensus_queue.cc:1048] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:05:07.424881  8825 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:07.439703  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:07.435206  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:07.445835  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:07.446033  9387 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:07.407393 15476 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:07.452630 15916 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:07.432806 15926 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.449512 15899 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:07.459609 15899 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.442900 15836 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.460682 15804 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.461623 15633 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.466754  8748 raft_consensus.cc:2393] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:07.466521 15923 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:07.449224 15924 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:07.429857 15147 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:07.487701  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:07.477461  9381 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:05:07.498073  8825 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:07.470741  9385 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 22.
I20260709 13:05:07.471531  8645 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:07.477150  9389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 1002) took 2033 ms. Trace:
I20260709 13:05:07.446717 15744 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:07.503350  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:07.500016  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:07.445518 15465 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.503438  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:07.451133 15476 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:07.500767  9389 rpcz_store.cc:276] 0709 13:05:05.443277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.443437 (+   160us) service_pool.cc:224] Handling call
0709 13:05:07.476919 (+2033482us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54006275d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:07.477060 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3400423168}
I20260709 13:05:07.473855  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:07.449152 15894 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:07.510749  8796 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:07.504179  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:07.471257 15624 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:07.523650  8830 raft_consensus.cc:2393] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:05:07.524381  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 50628.9us
I20260709 13:05:07.474997 15340 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:07.471601 15926 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:07.503266 15744 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.503984 15830 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:07.503038 15873 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.529985  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:07.536684  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.537271  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:07.504545 15923 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:07.509928 15899 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.531929 15766 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 13:05:07.524020 15476 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:07.530997 15711 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:07.552471 15711 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 22 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:07.553381 15711 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.535405 15744 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:07.554915 15580 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:07.536705  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:07.565394  8823 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:07.537927  8830 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:05:07.528864 15912 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:07.525548 15924 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:07.540462 15920 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:07.544569 15927 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.545527 15926 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.524854 15465 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.526858 15894 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:07.595093  8823 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:07.595446  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.596079  8750 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:07.597512  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.547693 15147 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.560000 15476 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.537447  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:07.559767 15633 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:07.535244  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.605233  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 67852.9us
I20260709 13:05:07.556993  8828 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:07.606284  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2680) took 1630 ms. Trace:
I20260709 13:05:07.600055 15928 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.608378  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:07.609242 15885 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.605s	user 0.009s	sys 0.003s
I20260709 13:05:07.609396 15926 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.609809  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:07.609730 15885 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.620754 15925 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.621383 15885 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:07.631191  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:07.632064  8675 raft_consensus.cc:2393] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:07.633051  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:07.578898  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.635028  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 3177) took 1773 ms. Trace:
I20260709 13:05:07.585615  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:07.587476 15933 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:05:07.588364 15865 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:07.591187  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.591475 15804 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.591871  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.528273 15624 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.604499  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.576650 15920 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:07.574851 15924 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:07.610688  8678 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:07.610723  8755 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:05:07.575340 15340 consensus_queue.cc:1059] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:05:07.611373  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:07.612113  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:07.614070  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:07.614789  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:07.564003 15830 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.600955 15894 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:07.618407  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:07.661952  8680 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:07.573760 15065 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:07.664492  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.624274 15927 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.625384  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:07.668622  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:07.628109  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:07.630251 15855 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:07.609851 15928 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.606761  8828 rpcz_store.cc:276] 0709 13:05:05.975696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.556308 (+1580612us) service_pool.cc:224] Handling call
0709 13:05:07.556753 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:07.605924 (+ 49171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.606205 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:07.679522  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.637437  8821 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:05:07.680537  8755 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:07.681653  8793 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:07.599471  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.643009  8754 raft_consensus.cc:2368] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 21. Current term is 22.
I20260709 13:05:07.635443  8751 rpcz_store.cc:276] 0709 13:05:05.861205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.941443 (+ 80238us) service_pool.cc:224] Handling call
0709 13:05:07.634786 (+1693343us) spinlock_profiling.cc:243] Waited 990 ms on lock 0x7b54001581d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:07.634962 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2277151104}
I20260709 13:05:07.683094  8747 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
I20260709 13:05:07.640123  8748 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:05:07.654016  8749 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:05:07.617210 15465 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.657896  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:07.668788  8681 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:07.626092 15912 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:07.691354  8679 raft_consensus.cc:2410] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
W20260709 13:05:07.692167  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.669631  8677 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:05:07.570765 15873 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.682252  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:07.693812  8750 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 21: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:07.687212  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:07.694775  8678 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:07.655479 15924 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:07.661300 15926 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:07.651944 15624 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:07.651980 15865 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:07.696357  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:07.701817  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:07.651551 15920 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:07.697726  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:07.700310  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:07.706067  8828 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:07.706519  8756 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:07.701671  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.707695  8749 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:07.709906  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.688457 15855 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:07.711231  8747 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:07.696502 15836 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:07.705833  8821 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:07.695050 15912 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:07.718142  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:07.722147  8828 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:07.720736  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.736368  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:07.736712  8794 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:07.737208  8755 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:05:07.737864  9353 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:07.741304  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:07.741787  8794 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:07.741995  9353 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:07.742082  8821 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:07.726716 15912 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:07.750322  9353 leader_election.cc:400] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 21. Current term is 22.
I20260709 13:05:07.754240  9353 leader_election.cc:403] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:05:07.732064 15839 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.768200  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:07.757980  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:07.769634  8678 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:07.724061 15836 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:07.709266 15926 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.761234 15847 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:07.758085  8756 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:07.748170 15835 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:07.757140 15819 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:07.756716 15774 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:07.786067  8678 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:07.774624 15836 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:07.771112 15823 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:07.786541  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:07.777271 15835 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.790074 15819 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.793365 15774 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:07.799883 15481 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:07.800993 15835 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:07.824556 15774 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:07.830685 15863 raft_consensus.cc:3055] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:07.831276 15863 raft_consensus.cc:740] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:07.839618  8825 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:05:07.840029  8828 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:07.846355  8823 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 20. Current term is 21. Ops: [19.12-20.13]
I20260709 13:05:07.815980 15819 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:07.824164 15147 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:07.860167  8827 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2689) took 1761 ms (client timeout 1896 ms). Trace:
W20260709 13:05:07.862327 15938 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:05:07.870916  8821 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 13:05:07.833416 15830 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:07.877955  8826 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 1467) took 2139 ms (client timeout 2776 ms). Trace:
W20260709 13:05:07.878465  8826 rpcz_store.cc:269] 0709 13:05:05.738700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.738963 (+   263us) service_pool.cc:224] Handling call
0709 13:05:07.877686 (+2138723us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54003ab6d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:07.877856 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2865791360}
I20260709 13:05:07.880734 15881 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:07.881234 15881 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:07.883502 15926 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:07.868314  8681 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 22.
W20260709 13:05:07.865983  8827 rpcz_store.cc:269] 0709 13:05:06.098432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.230139 (+131707us) service_pool.cc:224] Handling call
0709 13:05:07.859891 (+1629752us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b540009ced4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:07.860054 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1029432704}
W20260709 13:05:07.886884 15939 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.114s	user 0.001s	sys 0.001s
I20260709 13:05:07.882234 15881 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.905330 15881 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:07.911351 15942 log.cc:927] Time spent T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:05:07.782004 15865 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.913183 15865 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:07.913805  8722 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:07.915969  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2693) took 1771 ms. Trace:
I20260709 13:05:07.832239 15863 consensus_queue.cc:260] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.916651  8822 rpcz_store.cc:276] 0709 13:05:06.144570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.423413 (+278843us) service_pool.cc:224] Handling call
0709 13:05:07.915733 (+1492320us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b54003ddbd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:07.915898 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1303819648}
I20260709 13:05:07.917795 15863 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:07.915796 15943 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:05:07.921249  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:07.884935 15873 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:07.925729  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 184001us
I20260709 13:05:07.926548 15875 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:05:07.927891 15848 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:07.939991 15896 consensus_queue.cc:1059] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:05:07.946228 15896 consensus_queue.cc:1243] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:07.946913 15896 raft_consensus.cc:3055] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:07.947288 15896 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:07.948269 15896 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.949070 15624 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:07.949705 15896 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:07.950418  8672 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:05:07.953243 15879 consensus_meta.cc:287] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.001s
I20260709 13:05:07.954826  9383 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:07.957795  9385 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:05:07.964258 15941 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.147s	user 0.001s	sys 0.001s
I20260709 13:05:07.963686  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:07.955888 15819 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.937939 15848 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:07.953848 15823 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 21. Current term is 22.
I20260709 13:05:07.979945 15819 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:07.983951  8794 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:07.990056  8794 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:07.995067  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:07.983355 15873 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:08.001519  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:07.957293 15879 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:08.005779 15711 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:05:08.017168 15476 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:08.019332 15925 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:08.011947 15855 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:07.998934 15481 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:08.024829  9380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 999) took 2591 ms (client timeout 1697 ms). Trace:
W20260709 13:05:08.025368  9380 rpcz_store.cc:269] 0709 13:05:05.433753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.433992 (+   239us) service_pool.cc:224] Handling call
0709 13:05:08.024584 (+2590592us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400018bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:08.024713 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2421467520}
W20260709 13:05:08.027707 15947 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:05:08.028162  8679 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:08.030189  9381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 887) took 1272 ms. Trace:
I20260709 13:05:07.998522 15830 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:08.065398  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:07.995846 15949 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:08.041626 15657 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:08.071553 15657 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:08.072130  9388 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 1003) took 2628 ms (client timeout 2468 ms). Trace:
I20260709 13:05:08.042071 15848 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:08.072592  9388 rpcz_store.cc:269] 0709 13:05:05.443651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.447719 (+  4068us) service_pool.cc:224] Handling call
0709 13:05:08.071928 (+2624209us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54001ccd54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:08.072051 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3433931008}
W20260709 13:05:08.073792  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.076864  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.077188 15926 consensus_queue.cc:1059] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:08.077692  8677 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:08.077826  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:07.992986 15948 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:05:08.078802 15147 raft_consensus.cc:3055] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:08.079288 15147 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:08.080345 15147 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:08.081671 15147 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:08.082374  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:08.083341  8821 raft_consensus.cc:2410] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:05:08.043831 15823 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.050663 15729 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.975157 15924 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:07.995293  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0836489040000288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16970000000000002s for a response
I20260709 13:05:08.054459  8830 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:08.054666  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2682) took 2061 ms. Trace:
I20260709 13:05:08.106400  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.058779  9381 rpcz_store.cc:276] 0709 13:05:06.758238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.765188 (+  6950us) service_pool.cc:224] Handling call
0709 13:05:07.477093 (+711905us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:07.498244 (+ 21151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.029938 (+531694us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b54006275d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:08.030116 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1614743424}
W20260709 13:05:08.109258  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.085182  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.111440  8677 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 14.
I20260709 13:05:08.098515  8824 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:05:08.101366  8675 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 18: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:08.077121  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:08.117313  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.105458  8829 rpcz_store.cc:276] 0709 13:05:05.993393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.584180 (+590787us) service_pool.cc:224] Handling call
0709 13:05:06.615131 (+ 30951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:07.006992 (+391861us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:07.007039 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:07.007099 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:07.007159 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.042473 (+1035314us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b54003acad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:08.042548 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.054574 (+ 12026us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":869830400}
I20260709 13:05:08.070051 15476 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.139977  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:08.141845  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.064390 15873 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.068045 15949 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.142345  9385 raft_consensus.cc:2376] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16. Re-sending same reply.
I20260709 13:05:08.139813  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:08.147300  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.072526 15920 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:08.097800 15676 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:08.098347 15603 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.151345  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:08.152988  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:08.150187  8674 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
W20260709 13:05:08.153479  8574 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40096 (request call id 143) took 51650 ms (client timeout 15000 ms). Trace:
I20260709 13:05:08.156169  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.153839  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:08.160382  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.161240  8756 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:08.162880  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.166944  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.168231 15920 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:08.168454  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:08.169660  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.170601  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.171159  8677 raft_consensus.cc:2393] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:08.172295  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.171880  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.174517  8755 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:05:08.175407  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:08.152696 15941 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:05:08.177937  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.190985  8754 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 25.
I20260709 13:05:08.181370  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:08.165424  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 207332us
W20260709 13:05:08.193826  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.186676  8756 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:05:08.186995  8748 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 13: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:08.187465  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:08.196955  9385 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:05:08.153023 15744 consensus_meta.cc:287] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.004s
I20260709 13:05:08.151507 15885 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.198153  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:08.160056  8574 rpcz_store.cc:269] 0709 13:04:16.503501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.505177 (+  1676us) service_pool.cc:224] Handling call
0709 13:05:08.153395 (+51648218us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:05.067097 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:05.067884 (+   787us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:05.067951 (+    67us) write_op.cc:435] Acquired schema lock
0709 13:05:05.067974 (+    23us) tablet.cc:662] Decoding operations
0709 13:05:05.069025 (+  1051us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:05.069146 (+   121us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:05.069178 (+    32us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:05:05.069901 (+   723us) tablet.cc:701] Row locks acquired
0709 13:05:05.069951 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:05:05.070141 (+   190us) write_op.cc:270] Start()
0709 13:05:05.070325 (+   184us) write_op.cc:276] Timestamp: P: 1783602305070066 usec, L: 0
0709 13:05:05.070369 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:05:05.199356 (+128987us) log.cc:844] Serialized 2496 byte log entry
0709 13:05:05.588467 (+389111us) op_driver.cc:464] REPLICATION: finished
0709 13:05:06.149420 (+560953us) write_op.cc:301] APPLY: starting
0709 13:05:06.149590 (+   170us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:06.157534 (+  7944us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:06.157698 (+   164us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:06.162091 (+  4393us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:06.162495 (+   404us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=0,mrs_lookups=0
0709 13:05:06.162572 (+    77us) write_op.cc:312] APPLY: finished
0709 13:05:06.163024 (+   452us) log.cc:844] Serialized 65 byte log entry
0709 13:05:06.163297 (+   273us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:06.163436 (+   139us) write_op.cc:454] Released schema lock
0709 13:05:06.163524 (+    88us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:06.224978 (+ 61454us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:06.243042 (+ 18064us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":548497,"apply.run_cpu_time_us":13825,"apply.run_wall_time_us":30698,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29183,"num_ops":6,"prepare.queue_time_us":673414,"prepare.run_cpu_time_us":11740,"prepare.run_wall_time_us":292550,"raft.queue_time_us":194620,"raft.run_cpu_time_us":53314,"raft.run_wall_time_us":364085,"replication_time_us":478785,"thread_start_us":768805,"threads_started":4,"wal-append.queue_time_us":105200,"wal-append.run_cpu_time_us":33563,"wal-append.run_wall_time_us":2586594}]]}
I20260709 13:05:08.141659 15729 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.241079  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:08.149935 15476 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.107339 15823 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.157034 15877 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.151733 15875 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:08.149456 15949 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:08.102689 15774 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.202929  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.244309  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.245028  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:08.292032  9385 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 23: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:08.240433 15198 raft_consensus.cc:2764] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:05:08.251024  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2781) took 1241 ms. Trace:
I20260709 13:05:08.253521  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.200426 15847 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.257759 15481 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:08.258888  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.260732  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:08.262883  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:08.276486 15911 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:05:08.242117  8821 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 13.
I20260709 13:05:08.272195  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:08.270129  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:08.287946  9381 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:08.289634  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:08.152537 15819 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.087s
I20260709 13:05:08.292683  8752 rpcz_store.cc:276] 0709 13:05:07.009555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.009766 (+   211us) service_pool.cc:224] Handling call
0709 13:05:07.028231 (+ 18465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:07.248132 (+219901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.250712 (+1002580us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54002feed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:08.250953 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370639744}
I20260709 13:05:08.313889  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.153126 15873 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.296677  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.291193  8679 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:08.300177  8830 raft_consensus.cc:1240] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:08.300419  8749 raft_consensus.cc:2393] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 22: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:08.304571  8823 raft_consensus.cc:2393] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 17: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:08.173285 15633 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:08.320668  8794 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:08.321388  8647 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:08.315790  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:08.171888 15676 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:08.316958  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:08.333673  8756 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 23: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:08.318657  8752 raft_consensus.cc:2393] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:08.327795  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:08.177841 15603 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:08.274449 15855 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:08.332047  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:08.336336  8753 raft_consensus.cc:3060] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:08.336545  8674 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:08.334918  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:08.337711  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:08.291702 15624 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:08.343639  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:08.208069 15744 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:08.351447  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.358049  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.358911  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:08.360267  8752 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:08.358035  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.361209  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:08.360351  8827 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:08.361742  8821 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:08.260685 15885 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.347725  8794 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:08.278417 15862 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:05:08.320189 15927 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:08.368543  8749 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:08.370950  8751 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:08.371096  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.376446  8677 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 24.
I20260709 13:05:08.292273 15877 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:08.385617  8822 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 19: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:08.388407  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:08.394183  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:08.391490  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.379689 15481 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:08.406775 15481 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:08.407210 15603 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:08.408054 15481 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:08.408936  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.409641  8679 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:05:08.409934 15885 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:08.394763  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:08.411242 15879 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:08.411662 15881 consensus_meta.cc:287] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.506s	user 0.011s	sys 0.000s
I20260709 13:05:08.343439 15633 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:08.414067 15744 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:08.414005 15633 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.367190 15934 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:08.419004 15657 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:08.462139  8719 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:08.420980 15774 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:08.422995 15877 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:08.423408  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:08.466565  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:08.423704 15873 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:08.424569  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:08.425688 15823 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:08.426247 15729 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:08.428457  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:08.428735  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:08.432607  8645 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:08.434337  9382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 892) took 1039 ms. Trace:
I20260709 13:05:08.434543  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:08.333016 15925 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 13:05:08.435499  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.475268  9382 rpcz_store.cc:276] 0709 13:05:07.395105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.395253 (+   148us) service_pool.cc:224] Handling call
0709 13:05:08.434138 (+1038885us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b54005ea1d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:08.434272 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23093504}
I20260709 13:05:08.476578  8675 raft_consensus.cc:2410] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 23 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:05:08.438139 15796 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:08.309979 15847 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.359174 15720 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:08.454282  8680 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:08.374480 15927 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:08.420511 15912 consensus_meta.cc:287] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.653s	user 0.008s	sys 0.004s
I20260709 13:05:08.489214 15912 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:08.465816  8719 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:08.467449  8828 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 25.
I20260709 13:05:08.395077 15920 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:08.493144 15920 raft_consensus.cc:697] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:08.493479  8676 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 18: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:08.493737  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:08.494477  8828 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:08.494282 15920 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.494902  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:08.495527  8824 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:08.292435 15875 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:08.497788 15875 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:08.478108  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:08.498795  8752 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:08.478372  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 45846.5us
I20260709 13:05:08.502074  8645 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:08.481560  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:08.500151  8719 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:08.505985  8678 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:08.508734  8828 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:08.511309  8821 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:08.455875 15814 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.513257  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:05:08.478984  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50632.7us
I20260709 13:05:08.463150 15934 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.512156  8829 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:08.510444  8749 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:05:08.518494  9352 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:08.468924 15603 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:08.527320  8645 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:08.537716  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:08.538271  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.537873  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:08.530330  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 374097us
I20260709 13:05:08.540969  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:08.541864  8747 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:08.542213  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:08.544171  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:08.544456  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:08.544961  8756 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:08.545311  8825 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:05:08.545723  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:08.487116 15823 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:08.546775  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:08.548273  9352 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:08.549898  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:08.505369 15927 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:08.559804  8753 raft_consensus.cc:2468] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:05:08.491969 15912 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:08.558815  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:08.561023  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.550011  8828 raft_consensus.cc:2393] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 21: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:08.559379  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:08.516024 15877 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:08.503137 15817 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:08.565707  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.572525  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.525310 15729 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:05:08.569835  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 248502us
I20260709 13:05:08.526983 15774 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:08.608016  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:08.535099 15934 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:08.563747 15876 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:08.585937  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:08.543246 15840 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.586009  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.652511  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:08.605080  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:08.537292 15848 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:05:08.608886  9382 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:05:08.504928 15720 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.558830 15849 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:05:08.579087  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:08.559954 15823 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:05:08.594725 15930 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:05:08.632011  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.529968 15814 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.606519 15729 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:08.626825  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:08.692762  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:08.693746  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 52775.4us
I20260709 13:05:08.627849 15885 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:08.613814 15970 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:08.696286  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:08.697552  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.701503  8824 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
W20260709 13:05:08.716485  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 137514us
W20260709 13:05:08.732788  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.733191  8825 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:08.625438 15876 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:08.723255  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:08.725531  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:08.715724 15823 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:08.727628  8675 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:05:08.749720  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 53536.7us
I20260709 13:05:08.725852  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:08.637009 15925 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.735136  8823 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
W20260709 13:05:08.736150 15957 log.cc:927] Time spent T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:05:08.738488  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:08.740648  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:08.787405  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.787899  9381 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:08.788205  8675 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:05:08.746156 15894 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.065s	user 0.012s	sys 0.000s
I20260709 13:05:08.789494  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:08.789919 15894 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:08.791440 15960 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:05:08.792734 15894 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:08.670586 15848 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.793815 15848 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:08.777165  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:08.799678  8824 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:08.757685  8678 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:08.632804 15877 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 13:05:08.802305  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:08.803292 15603 raft_consensus.cc:3055] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:08.782745  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.805490  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.623090 15774 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.808993 15774 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:08.788553  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.812487  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:08.777196 15876 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:08.803905 15603 raft_consensus.cc:740] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:08.820530 15603 consensus_queue.cc:260] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:08.787007  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:08.823493  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:08.806376  8677 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:08.781289  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:08.828366  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:08.829499  8756 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 19: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:08.830669  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.840937  9383 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:05:08.843091  8796 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:08.843377 15969 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:08.844058 15969 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:08.844811 15969 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:08.851269  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:08.852500  9388 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 22: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:08.727227 15881 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:08.843523  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:08.843992  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.856397  8678 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:08.856863  8677 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 13.
W20260709 13:05:08.863523  8681 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.009s	sys 0.031s
W20260709 13:05:08.867399  8681 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.010s	sys 0.032s
I20260709 13:05:08.730530 15970 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:05:08.872845  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 312670us
W20260709 13:05:08.838783 15836 consensus_meta.cc:287] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.028s	user 0.011s	sys 0.003s
I20260709 13:05:08.784775 15817 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:08.875613  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:08.777431 15885 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:08.880080  9381 raft_consensus.cc:2393] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 19: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:08.880926  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:08.835551 15959 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:08.883234  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.891711 15133 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:05:08.888527  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:08.894004  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:08.864913  8674 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.528s	user 0.006s	sys 0.007s
W20260709 13:05:08.894953  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 169150us
I20260709 13:05:08.691728 15840 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:08.890271  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 362971us
I20260709 13:05:08.895630  8674 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 15.
I20260709 13:05:08.898299  8647 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:08.888520  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:08.912225  9388 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:08.854389 15676 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:08.863176 15873 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:08.854570 15774 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:08.876475 15865 consensus_meta.cc:287] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.963s	user 0.008s	sys 0.009s
I20260709 13:05:08.931234  9386 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:08.918031  8674 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:08.869577 15926 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.875113 15880 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:08.826156 15925 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.881731 15970 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:08.944412  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:08.889887 15817 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:08.955090  9387 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:08.912432 15912 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:08.876425 15744 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:05:08.870839 15711 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:05:08.963661  9352 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:08.951174  8678 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:08.967995 15865 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:08.982954  8827 consensus_meta.cc:287] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.615s	user 0.010s	sys 0.003s
I20260709 13:05:08.983553  8827 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 22.
I20260709 13:05:08.984728  8752 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:08.985580  8678 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:05:08.947468 15880 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.956234 15926 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.945487 15847 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:08.964953 15896 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.015s	user 0.014s	sys 0.003s
W20260709 13:05:09.016350  8748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.009s	sys 0.040s
I20260709 13:05:08.987162 15819 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:08.944967 15774 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:08.997356 15934 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:08.998245 15814 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:08.998667 15729 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:08.958091 15817 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:08.999399 15976 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:08.999420 15720 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:08.994484 15925 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:09.002296 15977 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:09.006800 15885 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:09.010654 15863 consensus_meta.cc:287] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.092s	user 0.010s	sys 0.004s
I20260709 13:05:08.918380 15964 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:05:09.034425  8821 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:08.932327 15676 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:05:09.041108  8747 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:08.997356 15876 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.012440 15840 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:09.012859  8679 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
I20260709 13:05:09.060449  8679 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 21.
I20260709 13:05:08.989634 15894 raft_consensus.cc:3055] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:05:09.013918 15963 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:05:08.958449 15873 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:05:09.016696  8748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.009s	sys 0.041s
I20260709 13:05:08.908443 15836 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.014550 15880 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:09.014310 15839 raft_consensus.cc:2764] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:05:08.987340 15848 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.089365 15848 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 26 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:09.074507  8679 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:09.091017  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2912) took 1028 ms. Trace:
I20260709 13:05:09.091478  8679 rpcz_store.cc:276] 0709 13:05:08.062808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.073748 (+1010940us) service_pool.cc:224] Handling call
0709 13:05:09.074250 (+   502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.090520 (+ 16270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.090930 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:09.026031 15978 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:09.059688 15896 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:05:09.099366 15835 consensus_meta.cc:287] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.262s	user 0.014s	sys 0.000s
I20260709 13:05:09.100020 15835 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:09.059265 15729 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 13:05:09.111131 15969 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.113749 15969 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:09.086274 15836 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:09.090063 15796 consensus_meta.cc:287] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.603s	user 0.009s	sys 0.002s
I20260709 13:05:09.082027 15934 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:09.144088  9384 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
I20260709 13:05:09.084096 15926 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:09.152367 15340 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:09.153033 15340 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:09.063143 15894 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:09.094337 15896 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:09.034474 15977 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:05:09.118369 15927 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.540s	user 0.008s	sys 0.000s
I20260709 13:05:09.165393 15927 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:09.167022  8675 raft_consensus.cc:2393] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:09.168453 15927 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:09.126009 15949 consensus_meta.cc:287] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.823s	user 0.008s	sys 0.005s
I20260709 13:05:09.169183  8753 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:05:09.169554 15949 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.171653  9387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 896) took 1283 ms. Trace:
I20260709 13:05:09.172399 15949 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:09.172173  9387 rpcz_store.cc:276] 0709 13:05:07.888411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.894767 (+  6356us) service_pool.cc:224] Handling call
0709 13:05:09.171433 (+1276666us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400627ad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:09.171576 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2375282560}
I20260709 13:05:09.127840  9382 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:05:09.174328  8822 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:09.121392  9383 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:05:09.177425  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1478) took 1089 ms. Trace:
I20260709 13:05:09.180531  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2800) took 1016 ms. Trace:
I20260709 13:05:09.181046  8751 rpcz_store.cc:276] 0709 13:05:08.164568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.169389 (+  4821us) service_pool.cc:224] Handling call
0709 13:05:09.180300 (+1010911us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400139a54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:09.180450 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461156608}
I20260709 13:05:09.177955  8830 rpcz_store.cc:276] 0709 13:05:08.087655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.108230 (+ 20575us) service_pool.cc:224] Handling call
0709 13:05:08.108618 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.320700 (+212082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.177089 (+856389us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54003ac5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:09.177348 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":439855872}
I20260709 13:05:09.122916 15835 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:09.205663  8753 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:05:09.157307  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 259109us
I20260709 13:05:09.163007  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3143) took 1544 ms. Trace:
I20260709 13:05:09.219929  8681 rpcz_store.cc:276] 0709 13:05:07.644080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.900513 (+256433us) service_pool.cc:224] Handling call
0709 13:05:07.901603 (+  1090us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:07.901905 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:08.306480 (+404575us) log.cc:844] Serialized 14 byte log entry
0709 13:05:09.162489 (+856009us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:09.162557 (+    68us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:09.162605 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.162750 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.162926 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":370027,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":146119,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":1205703,"threads_started":3,"wal-append.queue_time_us":668094}
I20260709 13:05:09.153602 15843 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:09.226547  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 445634us
W20260709 13:05:09.202276 15963 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:05:09.232547  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.202951 15977 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:09.234551  8681 raft_consensus.cc:1240] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:05:09.238507  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2914) took 1138 ms. Trace:
I20260709 13:05:09.239041  8681 rpcz_store.cc:276] 0709 13:05:08.099837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.223895 (+1124058us) service_pool.cc:224] Handling call
0709 13:05:09.234208 (+ 10313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.238067 (+  3859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.238422 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:09.240345  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.220252  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:09.244238  8647 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:09.251173 15969 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:09.251354 15876 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:09.251911 15969 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:09.252969 15969 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.226951 15823 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:09.254724 15340 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:09.255920 15340 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:09.256819 15823 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:09.258195  8681 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:05:09.258400  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:09.248893 15927 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:09.258989  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2915) took 1155 ms. Trace:
I20260709 13:05:09.259086 15927 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:09.259618 15927 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:09.260437 15970 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.261072  9385 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:09.259387  8681 rpcz_store.cc:276] 0709 13:05:08.103170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.241537 (+1138367us) service_pool.cc:224] Handling call
0709 13:05:09.257926 (+ 16389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.258702 (+   776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.258928 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:09.267465  8719 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:09.268983  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:09.269152 15876 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:09.270478  8719 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:05:09.278721  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 261858us
I20260709 13:05:09.280573  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:09.236931 15843 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:09.269748 15876 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:09.297430  8719 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:09.269767 15964 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:09.281745 15966 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:09.285359 15926 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:09.294302  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:09.306298 15964 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.289664 15817 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.333737 15964 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:09.336395  9384 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 895) took 1862 ms (client timeout 2086 ms). Trace:
W20260709 13:05:09.336954  9384 rpcz_store.cc:269] 0709 13:05:07.474395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.485303 (+ 10908us) service_pool.cc:224] Handling call
0709 13:05:09.336040 (+1850737us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54003b2ed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:09.336316 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3733164416}
W20260709 13:05:09.337603  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.289042 15848 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:09.298159 15970 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:09.289309 15676 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:05:09.339983  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55168 (request call id 1474) took 1275 ms. Trace:
I20260709 13:05:09.340229 15880 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:09.340749  8827 rpcz_store.cc:276] 0709 13:05:08.064246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.065072 (+   826us) service_pool.cc:224] Handling call
0709 13:05:09.339625 (+1274553us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b540032ee54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:09.339892 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":676578176}
I20260709 13:05:09.341614  8825 raft_consensus.cc:1240] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 20. Ops: []
I20260709 13:05:09.342854  8826 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:09.275333 15894 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.347630 15894 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:09.306586 15966 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.351078 15966 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:09.352424 15880 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:09.352917  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2804) took 1152 ms. Trace:
I20260709 13:05:09.355479  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3133) took 1739 ms. Trace:
I20260709 13:05:09.355986  8673 rpcz_store.cc:276] 0709 13:05:07.615686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.702246 (+ 86560us) service_pool.cc:224] Handling call
0709 13:05:07.716244 (+ 13998us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:07.716673 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:07.903110 (+186437us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b18002f90e0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df409455 00007f32e5ef13c7 00007f32e25039a7 00007f32e25130bf 00007f32e2513038
0709 13:05:08.065862 (+162752us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.455783 (+389921us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:08.456331 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 13:05:08.456860 (+   529us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:08.456909 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.456986 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.355110 (+898124us) raft_consensus.cc:1685] finished
0709 13:05:09.355230 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.355405 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":423992,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":2627,"raft.queue_time_us":23848,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":29957,"spinlock_wait_cycles":5920640,"thread_start_us":481374,"threads_started":2,"wal-append.queue_time_us":640312}
I20260709 13:05:09.357831  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:09.358814  8672 raft_consensus.cc:2376] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 21. Re-sending same reply.
I20260709 13:05:09.359246 15876 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.360419  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:09.361249  8672 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:09.279116 15896 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.363178 15896 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:09.280349 15873 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.317094  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2803) took 1128 ms. Trace:
I20260709 13:05:09.372478  8755 rpcz_store.cc:276] 0709 13:05:08.188537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.192812 (+  4275us) service_pool.cc:224] Handling call
0709 13:05:08.263033 (+ 70221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:08.541349 (+278316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:08.541389 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:08.548637 (+  7248us) log.cc:844] Serialized 26 byte log entry
0709 13:05:09.024599 (+475962us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:09.024652 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.025536 (+   884us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.316743 (+291207us) raft_consensus.cc:1685] finished
0709 13:05:09.316767 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.317025 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":39662,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":9047,"thread_start_us":715878,"threads_started":2,"wal-append.queue_time_us":660338}
I20260709 13:05:09.317276 15774 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:09.318048  8747 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:05:09.374927  8747 pending_rounds.cc:85] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 11
I20260709 13:05:09.375054  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53352 (request call id 2926) took 1121 ms. Trace:
I20260709 13:05:09.375295  8747 pending_rounds.cc:107] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:05:09.375569  8747 raft_consensus.cc:2889] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:09.375511  8680 rpcz_store.cc:276] 0709 13:05:08.267510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.293254 (+ 25744us) service_pool.cc:224] Handling call
0709 13:05:09.367512 (+1074258us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54002a8ad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:09.367841 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238824192}
I20260709 13:05:09.376950  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:09.377626  8680 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:05:09.316228 15924 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:09.358461  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:09.353505  8748 rpcz_store.cc:276] 0709 13:05:08.200134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.200926 (+   792us) service_pool.cc:224] Handling call
0709 13:05:08.268530 (+ 67604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:08.427428 (+158898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:08.427482 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:08.428056 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:05:09.073034 (+644978us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:09.073101 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.352299 (+279198us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400157cd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:09.352378 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.352455 (+    77us) raft_consensus.cc:1685] finished
0709 13:05:09.352477 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.352814 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4836,"raft.queue_time_us":12322,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":10428,"spinlock_wait_cycles":364335872,"thread_start_us":569393,"threads_started":1,"wal-append.queue_time_us":635080}
I20260709 13:05:09.380295  8673 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:09.388026  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 493977us
W20260709 13:05:09.383252  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.378592  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:09.399202  8680 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:05:09.401118  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:09.402324  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:09.402988  8680 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:05:09.403995  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:09.404572  8680 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 12.
W20260709 13:05:09.379596  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 461034us
I20260709 13:05:09.410003 15862 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:09.410795 15862 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:09.411840 15862 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.412510 15817 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:09.416324 15982 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:09.416872 15982 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:09.416919 15977 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.419404  9389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 1010) took 1487 ms. Trace:
I20260709 13:05:09.422243  8679 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:09.425693  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2916) took 1314 ms. Trace:
I20260709 13:05:09.426342  8680 rpcz_store.cc:276] 0709 13:05:08.111059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.424446 (+1313387us) service_pool.cc:224] Handling call
0709 13:05:09.424899 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.425141 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.425179 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:09.425245 (+    66us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:09.425314 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.425397 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.425588 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:09.417900 15982 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:09.434149  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 145259us
I20260709 13:05:09.380566 15580 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:05:09.460414  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.444983  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:09.463505  9352 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:09.473697  9352 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:09.419876  9389 rpcz_store.cc:276] 0709 13:05:07.932193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.941054 (+  8861us) service_pool.cc:224] Handling call
0709 13:05:08.439171 (+498117us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:08.439738 (+   567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:08.439780 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:08.464163 (+ 24383us) log.cc:844] Serialized 52 byte log entry
0709 13:05:08.729145 (+264982us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:08.973021 (+243876us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.974591 (+  1570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.974714 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:08.974762 (+    48us) raft_consensus.cc:1685] finished
0709 13:05:08.974773 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.419164 (+444391us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54005ea1d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:09.419319 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":398334,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":73,"raft.queue_time_us":407,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":31503,"spinlock_wait_cycles":1134900992,"thread_start_us":435439,"threads_started":2,"wal-append.queue_time_us":228897}
I20260709 13:05:09.474995  9352 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:05:09.483022  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 184285us
W20260709 13:05:09.488173  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 81686.8us
I20260709 13:05:09.489619  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:09.487445  9352 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:09.459407 15972 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:09.502058  8676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 3127) took 1894 ms (client timeout 1554 ms). Trace:
I20260709 13:05:09.476533 15834 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:09.471669 15580 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:09.477034 15926 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:09.482861 15819 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:09.505631  8753 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:05:09.506112  8677 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:09.502596  8676 rpcz_store.cc:269] 0709 13:05:07.607284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.611084 (+  3800us) service_pool.cc:224] Handling call
0709 13:05:09.501757 (+1890673us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54002413d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:09.501968 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1923623808}
I20260709 13:05:09.482461 15774 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:09.479490 15991 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:09.332741 15927 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.494081 15982 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:09.493099 15657 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:05:09.566068  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 76530.4us
I20260709 13:05:09.567543  8645 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:09.564496  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 119587us
W20260709 13:05:09.585939  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.502889 15340 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:05:09.590869  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 195041us
I20260709 13:05:09.591627  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2911) took 1529 ms. Trace:
I20260709 13:05:09.481132 15877 consensus_queue.cc:1059] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:05:09.594341  8678 rpcz_store.cc:276] 0709 13:05:08.062550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.015602 (+953052us) service_pool.cc:224] Handling call
0709 13:05:09.016085 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.017707 (+  1622us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:09.282719 (+265012us) log.cc:844] Serialized 14 byte log entry
0709 13:05:09.591149 (+308430us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:09.591212 (+    63us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:09.591268 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.591375 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.591552 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57131,"prepare.queue_time_us":493922,"thread_start_us":505762,"threads_started":2}
I20260709 13:05:09.596714  8676 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:09.597844  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2918) took 1411 ms. Trace:
I20260709 13:05:09.598284  8676 rpcz_store.cc:276] 0709 13:05:08.186235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.557997 (+1371762us) service_pool.cc:224] Handling call
0709 13:05:09.596500 (+ 38503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.597478 (+   978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.597765 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:09.596654  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:09.607244  8645 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:09.601337  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:09.532759 15819 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.532177 15774 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:09.594090 15340 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.619930 15340 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:09.620683 15926 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:09.627326  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.633041  8647 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:09.563155 15991 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:09.634105 15972 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:09.636795 15972 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:09.594681 15147 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:09.558480 15990 consensus_queue.cc:1059] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.129s
I20260709 13:05:09.594399 15879 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:09.639696 15924 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:09.640264 15924 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.640691 15924 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:09.641844  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2744) took 1251 ms (client timeout 1596 ms). Trace:
W20260709 13:05:09.642271  8823 rpcz_store.cc:269] 0709 13:05:08.390380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.424288 (+ 33908us) service_pool.cc:224] Handling call
0709 13:05:09.641664 (+1217376us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b54002684d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:09.641782 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":610541824}
I20260709 13:05:09.594228 15839 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:09.643072 15839 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:09.643467 15839 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:09.645068  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:09.645776  8824 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:05:09.646345 15977 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:09.646420  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:09.647068 15977 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:09.648027 15988 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:09.648343  8825 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:05:09.650462  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:09.651281  8830 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 13.
I20260709 13:05:09.638100 15972 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.653661  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2807) took 1357 ms. Trace:
I20260709 13:05:09.654098  8749 rpcz_store.cc:276] 0709 13:05:08.296029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.332461 (+ 36432us) service_pool.cc:224] Handling call
0709 13:05:09.653445 (+1320984us) spinlock_profiling.cc:243] Waited 8.93 ms on lock 0x7b540031bd54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:09.653579 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20540672}
I20260709 13:05:09.652801  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2746) took 1261 ms. Trace:
I20260709 13:05:09.656072  8822 rpcz_store.cc:276] 0709 13:05:08.391011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.541830 (+150819us) service_pool.cc:224] Handling call
0709 13:05:09.652576 (+1110746us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b540009d8d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:09.652734 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449051520}
I20260709 13:05:09.657301 15875 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:09.528772 15834 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.662750 15834 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:09.663715 15990 raft_consensus.cc:3055] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:09.664167 15990 raft_consensus.cc:740] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:09.666589  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:09.668030  9384 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:09.665081 15990 consensus_queue.cc:260] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:09.666712  9389 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:05:09.674770  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:09.660673  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55676 (request call id 1428) took 1307 ms. Trace:
I20260709 13:05:09.667102  8756 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:09.667071  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:09.675705  8678 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:09.676147  9383 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:09.666796  8674 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:09.675544  8675 rpcz_store.cc:276] 0709 13:05:08.353002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.882969 (+529967us) service_pool.cc:224] Handling call
0709 13:05:09.660392 (+777423us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54002576d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:09.660584 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":635399552}
I20260709 13:05:09.679306  9352 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:09.680500  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:09.671330  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:09.592540 15657 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.643477  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:09.689633  8823 raft_consensus.cc:2393] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 21: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:09.672775 15580 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.676795  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2927) took 1405 ms. Trace:
I20260709 13:05:09.699205  8678 rpcz_store.cc:276] 0709 13:05:08.271485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.616589 (+1345104us) service_pool.cc:224] Handling call
0709 13:05:09.675440 (+ 58851us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.676424 (+   984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.676708 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:09.670094  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:09.706137  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:09.708877  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 218505us
I20260709 13:05:09.711220  8678 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:05:09.715862  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:09.719924  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:09.720968  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 124406us
I20260709 13:05:09.721876  8722 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:09.719898  8796 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:09.723976  9384 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:09.670878 15819 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:09.670521 15774 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:09.735718  8796 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:09.697892 15657 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:09.687357 15848 raft_consensus.cc:697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:09.678689  8675 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116"
dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:53352
I20260709 13:05:09.752071  9383 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:09.734333 15676 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:09.720113 15633 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:05:09.787987  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 66175.5us
I20260709 13:05:09.789152  8722 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:09.791193  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 46796.1us
I20260709 13:05:09.791378  8675 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:09.791851  8675 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:09.791965  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:09.720083 15988 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:09.679410 15875 consensus_queue.cc:1059] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:05:09.801348  8672 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 26.
W20260709 13:05:09.811493 16003 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:05:09.813521  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3147) took 1704 ms. Trace:
I20260709 13:05:09.813978  8681 rpcz_store.cc:276] 0709 13:05:08.109079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.267083 (+1158004us) service_pool.cc:224] Handling call
0709 13:05:09.267539 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.267798 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.267825 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:09.268358 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 13:05:09.476141 (+207783us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:09.476192 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.476295 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.813195 (+336900us) raft_consensus.cc:1685] finished
0709 13:05:09.813221 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.813440 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":207351,"threads_started":1,"wal-append.queue_time_us":232855}
I20260709 13:05:09.815585 15894 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:09.816169 15894 raft_consensus.cc:697] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:09.816347 15970 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:09.816790 15970 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:09.817176 15970 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:09.817206 15894 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:09.734089 15834 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:09.818941 15834 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:05:09.819227 16004 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:05:09.819406 15834 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:09.788447 15633 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:09.820675 15633 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.821969 15633 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:09.824705 15877 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:09.757485 15990 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:09.826215 16001 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:05:09.734524 15969 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.728127 15823 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.719866 15989 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:09.842550  9353 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:09.753829 15817 raft_consensus.cc:2764] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:05:09.753499 15580 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:09.854466  9353 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:09.854776  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.800717 15988 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:09.881039  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:09.863139  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3203) took 1418 ms. Trace:
I20260709 13:05:09.885238  8821 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
W20260709 13:05:09.887526 15896 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.524s	user 0.015s	sys 0.002s
I20260709 13:05:09.887416  8573 catalog_manager.cc:5697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 19. New cstate: current_term: 19 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:09.888278 15896 consensus_queue.cc:1059] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:09.889065 15896 consensus_queue.cc:1243] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:09.889575 15896 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.869812 15877 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:09.889969  8573 catalog_manager.cc:5697] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 17 to 20, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 20 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:09.891453 15843 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:09.892050 15843 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:09.893427 15877 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:09.893940 15877 raft_consensus.cc:697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:05:09.895584  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.895757 15875 raft_consensus.cc:3055] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:09.896136 15875 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:09.897356  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3202) took 1452 ms. Trace:
I20260709 13:05:09.897815  8756 rpcz_store.cc:276] 0709 13:05:08.444474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.873141 (+428667us) service_pool.cc:224] Handling call
0709 13:05:09.666875 (+793734us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.687483 (+ 20608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.897128 (+209645us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b540031bd50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:09.897287 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1739620736}
I20260709 13:05:09.893934 15848 consensus_queue.cc:237] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.901491  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2907) took 1839 ms. Trace:
I20260709 13:05:09.843084 16008 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.103s
I20260709 13:05:09.901976  8674 rpcz_store.cc:276] 0709 13:05:08.061615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.950730 (+889115us) service_pool.cc:224] Handling call
0709 13:05:09.666567 (+715837us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.677404 (+ 10837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.901245 (+223841us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b54002576d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:09.901420 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1600828672}
I20260709 13:05:09.902657 16008 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:09.892412  8573 catalog_manager.cc:5697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:09.903083 16008 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:09.903334  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:09.904107  8755 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:05:09.903949  8573 catalog_manager.cc:5697] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 11 to 16, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 16 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:09.906133  8573 catalog_manager.cc:5697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 7 to 15. New cstate: current_term: 15 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:09.908632  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2932) took 1558 ms. Trace:
I20260709 13:05:09.910821  8678 rpcz_store.cc:276] 0709 13:05:08.350431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.703053 (+1352622us) service_pool.cc:224] Handling call
0709 13:05:09.703644 (+   591us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.712597 (+  8953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.908310 (+195713us) spinlock_profiling.cc:243] Waited 7.09 ms on lock 0x7b54003a20d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:09.908562 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16310144}
I20260709 13:05:09.852165 15989 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.912573  8678 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:09.915299  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1431) took 1561 ms. Trace:
I20260709 13:05:09.915709  8678 rpcz_store.cc:276] 0709 13:05:08.353987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.911968 (+1557981us) service_pool.cc:224] Handling call
0709 13:05:09.912367 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.914995 (+  2628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.915223 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:09.917644  8678 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:05:09.906388 15966 consensus_meta.cc:287] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.555s	user 0.012s	sys 0.001s
I20260709 13:05:09.918457  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1432) took 1564 ms. Trace:
I20260709 13:05:09.896986 15875 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:09.921460  8678 rpcz_store.cc:276] 0709 13:05:08.354340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.916962 (+1562622us) service_pool.cc:224] Handling call
0709 13:05:09.917316 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.918168 (+   852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.918401 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:09.890030 15896 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:09.930011  8673 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.536s	user 0.013s	sys 0.001s
I20260709 13:05:09.930670  8673 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:05:09.913687  8573 catalog_manager.cc:5697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:09.884356 15988 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:09.937369  8573 catalog_manager.cc:5697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:09.948421  8573 catalog_manager.cc:5697] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 12 to 14, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 14 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:09.952234  8573 catalog_manager.cc:5697] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 9 to 19, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 19 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:09.883267  8750 rpcz_store.cc:276] 0709 13:05:08.444748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.241357 (+796609us) service_pool.cc:224] Handling call
0709 13:05:09.246887 (+  5530us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:09.247367 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.247399 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:09.248090 (+   691us) log.cc:844] Serialized 52 byte log entry
0709 13:05:09.398842 (+150752us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:09.457662 (+ 58820us) log.cc:844] Serialized 14 byte log entry
0709 13:05:09.862595 (+404933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.862742 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.862814 (+    72us) raft_consensus.cc:1685] finished
0709 13:05:09.862833 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.863052 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":72810,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":199289,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":139,"thread_start_us":554119,"threads_started":2,"wal-append.queue_time_us":147594}
W20260709 13:05:09.961032  8752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2812) took 1568 ms (client timeout 1910 ms). Trace:
I20260709 13:05:09.854390 15969 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:09.967264 15964 consensus_meta.cc:287] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.631s	user 0.013s	sys 0.000s
I20260709 13:05:09.858395 15823 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:09.965274  8751 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:09.946571 15876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.003s	sys 0.009s
I20260709 13:05:09.969787 15927 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:09.957998 15985 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:09.975598  8752 rpcz_store.cc:269] 0709 13:05:08.392729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.394502 (+  1773us) service_pool.cc:224] Handling call
0709 13:05:09.960718 (+1566216us) spinlock_profiling.cc:243] Waited 87.7 ms on lock 0x7b540008ff54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:09.960936 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201815296}
I20260709 13:05:09.925055 15966 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:09.979463  8573 catalog_manager.cc:5697] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 12 to 15, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 15 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:09.920954 15989 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:09.964035 15875 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:09.904026 16008 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:10.007351 15876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.003s	sys 0.009s
I20260709 13:05:10.020419  8573 catalog_manager.cc:5697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 4 to 11, term changed from 13 to 15, VOTER e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) evicted. New cstate: current_term: 15 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:10.030891  8573 catalog_manager.cc:5697] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 12 to 21, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 21 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:10.032913  8573 catalog_manager.cc:5697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:09.997875 15774 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:10.041625  8573 catalog_manager.cc:5697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:09.966161 15877 consensus_queue.cc:237] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:10.031560  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 239648us
I20260709 13:05:10.010306 15964 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:10.047941  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 341927us
I20260709 13:05:10.010054 15340 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:10.048925  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:10.049891 16017 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:10.050434 16017 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:10.046495  8573 catalog_manager.cc:5697] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:10.052660  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:10.050886 16017 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:10.043601 16008 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:10.022923 15985 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:10.062896  8573 catalog_manager.cc:5697] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:10.064626  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.068305  9355 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:10.064220  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 384968us
W20260709 13:05:10.064296  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 328655us
I20260709 13:05:10.074257  8573 catalog_manager.cc:5697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 12 to 15, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 15 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:10.073787 15982 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:10.080282 15340 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:10.080399 16010 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:05:10.081239 16009 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:05:10.081539  8673 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:05:10.088514 16017 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:10.091164  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 236770us
W20260709 13:05:10.078771  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.090443 15862 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:10.169205  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:10.103602  9355 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:10.171005  8747 raft_consensus.cc:2368] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 25. Current term is 26.
I20260709 13:05:10.171366  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:10.107949  8573 catalog_manager.cc:5697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:10.172207  8752 raft_consensus.cc:2368] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14.
I20260709 13:05:10.173080  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:10.096621  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:10.062292 15876 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:10.089784  8675 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:10.158978  8680 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:10.177670  8681 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 19. Ops: []
I20260709 13:05:10.069967 15981 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:05:10.156849  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.053697 15924 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:10.163092 15926 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:10.125761 15927 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.174888  8719 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:10.173656  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:10.179844  8573 catalog_manager.cc:5697] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 11 to 18, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 18 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:10.158058 15896 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:10.193434  8573 catalog_manager.cc:5697] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 7 to 17. New cstate: current_term: 17 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:10.154714 15925 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:05:10.184247  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 87714.5us
I20260709 13:05:10.125958 15982 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:10.189332  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:10.158239 15340 raft_consensus.cc:2804] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:10.192660  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:10.193315  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:10.182523  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:10.197563  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:10.206807  8749 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:05:10.187646  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:10.203233  8675 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:10.208196  9383 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:05:10.208513  8573 catalog_manager.cc:5697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 15 to 17, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 17 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:10.211344  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:10.213768  8573 catalog_manager.cc:5697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:10.218015  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:53352 (request call id 2931) took 1877 ms. Trace:
I20260709 13:05:10.218480  8675 rpcz_store.cc:276] 0709 13:05:08.339206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.678510 (+1339304us) service_pool.cc:224] Handling call
0709 13:05:10.208132 (+529622us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.220278  8678 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:05:10.224555  8573 catalog_manager.cc:5697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:10.233856  8573 catalog_manager.cc:5697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:10.249171  9384 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:05:10.206636 16011 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 13:05:10.265051  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 99969.7us
I20260709 13:05:10.177718 15862 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:10.259004  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:10.273994  9383 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:10.195986 16022 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:10.219074 16015 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:10.269892  8573 catalog_manager.cc:5697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 15 to 18, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 18 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:10.189980 15981 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:10.275840  9384 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:05:10.293721  9384 pending_rounds.cc:85] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Aborting all ops after (but not including) 13
I20260709 13:05:10.286092  8752 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:10.294178  9384 pending_rounds.cc:107] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:05:10.294557  9384 raft_consensus.cc:2889] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:10.296509  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:10.200057 15927 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:10.255995  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:10.295245  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:10.299276  8675 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 18: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:10.299225  8752 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:10.259660 15977 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.302402  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:10.303426  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:10.304738  8749 raft_consensus.cc:3055] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:10.305253  8749 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:10.208036 15925 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:10.304757  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 182732us
I20260709 13:05:10.305213  8675 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:10.307994  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:10.309693  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:10.273154 15966 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:10.312844  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:10.239923 16006 consensus_queue.cc:1059] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 13:05:10.207195 15926 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:10.313742  8675 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:05:10.239508 16007 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 13:05:10.276000 16023 raft_consensus.cc:2764] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:05:10.277992 16011 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 26 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:10.288157 16022 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:10.322801  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:10.298079 15873 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.301643 15774 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:10.306568  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:10.327975  8675 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 18: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:10.326095  8749 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:10.332494  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 228962us
I20260709 13:05:10.348948  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:10.316449 16021 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:10.389201  8749 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:05:10.389999  8754 raft_consensus.cc:3060] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:10.322700 16017 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:10.379153 16024 log.cc:927] Time spent T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:05:10.362457 15834 consensus_meta.cc:287] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.001s
W20260709 13:05:10.470860  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.330268 16022 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:10.354537 16006 consensus_queue.cc:1243] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:05:10.400215  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:10.505479  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:10.505944  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:10.425248  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 118743us
I20260709 13:05:10.309767 16016 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.134s
W20260709 13:05:10.500936 15580 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.629s	user 0.004s	sys 0.008s
W20260709 13:05:10.502751 15633 consensus_meta.cc:287] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.005s
I20260709 13:05:10.340969 16028 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:10.332826 15894 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.342291 16023 raft_consensus.cc:2764] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
W20260709 13:05:10.507757 15879 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.868s	user 0.014s	sys 0.000s
I20260709 13:05:10.341183 15817 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:05:10.436045 15970 consensus_meta.cc:287] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.600s	user 0.012s	sys 0.003s
I20260709 13:05:10.353048 16007 consensus_queue.cc:1243] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:05:10.313470 15977 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:10.558911 15977 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:10.561241 15843 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.669s	user 0.006s	sys 0.005s
I20260709 13:05:10.561862 15843 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:10.458673 15839 consensus_meta.cc:287] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.792s	user 0.005s	sys 0.008s
I20260709 13:05:10.562932 15839 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:10.563834 16025 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 13:05:10.564054  8673 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:10.535629 16014 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.371s	user 0.000s	sys 0.001s
W20260709 13:05:10.565312 16026 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 13:05:10.532061 16020 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:10.382226 15873 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:10.358282 15927 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:10.552057 15879 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:10.358335 15835 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.566731 16020 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:10.567265 15835 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:10.567834 15989 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.531s	user 0.013s	sys 0.000s
I20260709 13:05:10.568449 15873 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:10.568297 15989 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:10.334735 15964 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:10.380419 15476 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:10.569031 16020 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:10.569917 15835 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:10.426455  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:10.461098 15988 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:10.571539  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:10.572362  8675 raft_consensus.cc:2368] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16.
I20260709 13:05:10.572739 15988 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:10.573257  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:10.574045  8675 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 20: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:10.356715 15925 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:10.354291 15924 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:10.574803 15989 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:10.564060 15982 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:10.577263 15839 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:10.502365 15876 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:10.394188 15981 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:10.397186 15985 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:10.532884  9352 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:05:10.578945 15657 consensus_meta.cc:287] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.733s	user 0.004s	sys 0.009s
I20260709 13:05:10.579118  8756 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:10.579387  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 46619.2us
I20260709 13:05:10.579483 15657 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:10.462069 15340 raft_consensus.cc:697] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:05:10.581004  8793 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.830s	user 0.013s	sys 0.041s
W20260709 13:05:10.581416  8793 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.013s	sys 0.042s
I20260709 13:05:10.582065 15657 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:10.581792 15340 consensus_queue.cc:237] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:10.584810  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2917) took 2432 ms. Trace:
I20260709 13:05:10.538954  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2756) took 2011 ms. Trace:
I20260709 13:05:10.585374  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:10.585299  8680 rpcz_store.cc:276] 0709 13:05:08.152552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.427632 (+1275080us) service_pool.cc:224] Handling call
0709 13:05:09.477970 (+ 50338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:10.177341 (+699371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.584514 (+407173us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b54000da7d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:10.584735 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1565481728}
I20260709 13:05:10.585976  8821 rpcz_store.cc:276] 0709 13:05:08.527746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.127638 (+599892us) service_pool.cc:224] Handling call
0709 13:05:09.146561 (+ 18923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.885825 (+739264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.538563 (+652738us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b540016a154. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:10.538878 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1697841408}
I20260709 13:05:10.586200  8675 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 13.
W20260709 13:05:10.544797  8566 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7aff04f53d4845a6a4fa491fd2a06145 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:10.587602  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:10.589604  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:10.592190  8754 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:10.595153  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:10.595842  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2908) took 2534 ms. Trace:
I20260709 13:05:10.596302  8677 rpcz_store.cc:276] 0709 13:05:08.061870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.956407 (+894537us) service_pool.cc:224] Handling call
0709 13:05:09.503075 (+546668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.559407 (+ 56332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.595584 (+1036177us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54002413d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:10.595763 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255927168}
I20260709 13:05:10.597684  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:10.590164  8675 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:10.535048 16030 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:10.599716  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:10.600157 16030 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:10.600451  8672 raft_consensus.cc:2376] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17. Re-sending same reply.
I20260709 13:05:10.601482  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:10.601240 16030 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:10.603840 15959 raft_consensus.cc:3055] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:10.604239 15959 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:10.605070 15959 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:10.606065 15959 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:10.607321 15885 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:10.607734 15885 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:10.608542 15885 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:10.609799 15885 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:10.556552 16029 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
W20260709 13:05:10.516601  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 214437us
W20260709 13:05:10.627084  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:10.627631  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:10.577675 15819 consensus_meta.cc:287] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.821s	user 0.015s	sys 0.000s
I20260709 13:05:10.628012  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2936) took 2201 ms. Trace:
I20260709 13:05:10.628713  8678 rpcz_store.cc:276] 0709 13:05:08.426709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.925795 (+1499086us) service_pool.cc:224] Handling call
0709 13:05:09.926208 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:10.230364 (+304156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.627666 (+397302us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54003cf854. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:10.627939 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664381696}
I20260709 13:05:10.424921 16021 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:10.457844 15969 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:10.582770 15657 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:10.612237  8673 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
W20260709 13:05:10.638994  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 356466us
W20260709 13:05:10.614171  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 872568us
I20260709 13:05:10.639475  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:10.641510  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:10.598558  8677 raft_consensus.cc:2393] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:10.616552  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:10.642997  8829 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:05:10.643666  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:10.616727  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:10.644806  8824 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 25: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:05:10.645452  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.646867  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:10.647744  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.647846  8821 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
I20260709 13:05:10.649677  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:10.617616  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:10.618350  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:10.650556  9384 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:10.650889  8827 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:10.539285 15823 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:10.652077  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 217509us
I20260709 13:05:10.630054  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:10.612958  8672 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:10.652813  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:10.631508  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1385) took 2185 ms. Trace:
I20260709 13:05:10.622344  8749 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:10.653264  8830 raft_consensus.cc:2368] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14.
I20260709 13:05:10.654596  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:10.551419 16028 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:10.515645 15834 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:10.646545  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:10.656577  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:10.537904 15894 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:10.349655 16011 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:10.555505 15817 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:10.611976 15970 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:11.031298  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.680984 15580 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.377s
I20260709 13:05:11.037685  8747 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:10.657910  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:10.624966 15835 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:11.049276  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:10.999812  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:11.075942  8755 raft_consensus.cc:3060] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:05:10.659584  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:11.074694  8747 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:11.077013  8672 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:05:10.851853  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:10.974956 15877 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.011s	sys 0.012s
I20260709 13:05:11.025394  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:11.071390 15966 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:10.989120 16044 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.053599  8753 rpcz_store.cc:276] 0709 13:05:08.446426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.207041 (+760615us) service_pool.cc:224] Handling call
0709 13:05:09.219237 (+ 12196us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.511115 (+291878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.631087 (+1119972us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b540038a9d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:10.631415 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":657875712}
I20260709 13:05:10.679548  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:10.659535  8675 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:05:11.090358  8677 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:05:11.090866  8678 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:10.719189 15926 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:10.728451  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2913) took 2637 ms. Trace:
I20260709 13:05:10.684103  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:10.659584  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:10.638392 15990 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.813s	user 0.011s	sys 0.006s
I20260709 13:05:10.840296  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:11.103148  8756 raft_consensus.cc:2368] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20.
I20260709 13:05:11.103814  8823 raft_consensus.cc:2393] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:10.681806  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:11.073182  8754 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:10.926930  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:11.107268  8676 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:05:11.108039  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 3153) took 1030 ms. Trace:
I20260709 13:05:11.108490  8676 rpcz_store.cc:276] 0709 13:05:10.077700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.926430 (+848730us) service_pool.cc:224] Handling call
0709 13:05:11.107946 (+181516us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.742197  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3206) took 2296 ms. Trace:
I20260709 13:05:10.695905  8829 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:10.897656  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.727731  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2742) took 2338 ms. Trace:
I20260709 13:05:10.680987  8645 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:10.664093  8796 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:10.818912  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:11.115432  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 434479us
W20260709 13:05:11.116479  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.000486  9352 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:05:11.047827 16027 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.581s	user 0.002s	sys 0.001s
I20260709 13:05:11.080533  8674 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:10.655431 16006 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.437s
W20260709 13:05:11.064638 16008 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.993s	user 0.015s	sys 0.000s
I20260709 13:05:10.740845 15603 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:10.903757 16022 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:11.011580 15949 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.032s	user 0.005s	sys 0.010s
I20260709 13:05:10.638510 16021 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:10.702808 15875 consensus_meta.cc:287] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.679s	user 0.007s	sys 0.007s
I20260709 13:05:11.077502  8749 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:11.118351 15959 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:11.052606  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:11.141912  8828 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:10.928027 16015 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:10.659003  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 350306us
I20260709 13:05:11.144069  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:11.132669 15949 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.160s	user 0.005s	sys 0.012s
I20260709 13:05:10.848250 15873 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:11.082412  8747 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:10.832391 16038 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.844008 15848 raft_consensus.cc:3055] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:10.714876 15862 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:10.726537 15991 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:10.837319 15147 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.681407 15823 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.092432  8675 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20. Ops: []
W20260709 13:05:10.799100  8722 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.013s	sys 0.042s
W20260709 13:05:11.161113  8722 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.018s	sys 0.052s
I20260709 13:05:11.091166  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3150) took 1018 ms. Trace:
I20260709 13:05:11.161316  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3152) took 1083 ms. Trace:
I20260709 13:05:11.161600  8677 rpcz_store.cc:276] 0709 13:05:10.073254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.075395 (+1002141us) service_pool.cc:224] Handling call
0709 13:05:11.089911 (+ 14516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.090916 (+  1005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.091113 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.161761  8675 rpcz_store.cc:276] 0709 13:05:10.077449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.091183 (+1013734us) service_pool.cc:224] Handling call
0709 13:05:11.091511 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.161024 (+ 69513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.161255 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:10.635552 15819 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.171339  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:11.095492  8679 rpcz_store.cc:276] 0709 13:05:08.090823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.092733 (+1001910us) service_pool.cc:224] Handling call
0709 13:05:09.093233 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.422948 (+329715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.728107 (+1305159us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b5400175a54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:10.728378 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":755864960}
I20260709 13:05:11.115670  8751 rpcz_store.cc:276] 0709 13:05:08.445546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.182283 (+736737us) service_pool.cc:224] Handling call
0709 13:05:09.962683 (+780400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:10.016451 (+ 53768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.741855 (+725404us) spinlock_profiling.cc:243] Waited 759 ms on lock 0x7b540008ff50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:10.742107 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1745559296}
I20260709 13:05:11.141667 15959 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:11.117033  8827 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
W20260709 13:05:11.117535  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 117167us
I20260709 13:05:11.079738  8752 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 14: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:11.110839  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:10.655202 15876 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.447s
I20260709 13:05:11.114138  8826 rpcz_store.cc:276] 0709 13:05:08.389393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.389548 (+   155us) service_pool.cc:224] Handling call
0709 13:05:09.342517 (+952969us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.379476 (+ 36959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.727360 (+1347884us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b540032ee50. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:10.727610 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2114225408}
I20260709 13:05:11.080060  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:11.175503  8823 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:05:11.176467  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:11.163780 16028 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.177053  8823 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:11.119957  8645 leader_election.cc:400] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14.
I20260709 13:05:10.660279 15476 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.436s
I20260709 13:05:11.177630  8645 leader_election.cc:403] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:05:10.683077 16019 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.410s
W20260709 13:05:11.130015  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.103392 15978 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:11.133320  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3155) took 1054 ms. Trace:
I20260709 13:05:10.645620 15657 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.122655 15926 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:11.084635 15835 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.123693 16029 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:05:11.146169  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:11.200445  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 175158us
W20260709 13:05:10.743584  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.216054  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:11.216310  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:11.147420  8755 raft_consensus.cc:2468] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
W20260709 13:05:11.217242  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.091950 15966 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.211818 15834 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:11.229089 15966 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:11.171075 15147 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.232867  8673 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:11.237438  8748 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:11.134027 15580 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:11.094568 16044 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.158520  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2935) took 2807 ms. Trace:
I20260709 13:05:11.258679  8681 rpcz_store.cc:276] 0709 13:05:08.351555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.815446 (+1463891us) service_pool.cc:224] Handling call
0709 13:05:09.815835 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:10.178143 (+362308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.158211 (+980068us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b5400298e54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:11.158448 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":831586688}
W20260709 13:05:11.163082  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 950248us
I20260709 13:05:11.163511  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3160) took 1002 ms. Trace:
I20260709 13:05:11.174026 15823 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:11.117089  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.170415  8828 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 14.
I20260709 13:05:11.266781  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:11.267598  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:11.267848  8672 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:11.122093  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:11.269768  8829 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:11.178076 15991 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:11.186991  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 522900us
I20260709 13:05:11.155961 15848 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:11.189435  8677 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:05:11.188905  8676 rpcz_store.cc:276] 0709 13:05:10.078473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.109647 (+1031174us) service_pool.cc:224] Handling call
0709 13:05:11.132294 (+ 22647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.132560 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:11.132684 (+   124us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:11.132948 (+   264us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:11.132981 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.133056 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.133235 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.276744  8824 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:05:11.243106 16029 raft_consensus.cc:697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:11.195556 15657 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:11.203089 15926 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:11.204737  8679 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [16.13-18.14]
I20260709 13:05:11.206760 16006 consensus_queue.cc:1059] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:05:11.189278 15885 consensus_meta.cc:287] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.579s	user 0.014s	sys 0.003s
I20260709 13:05:11.208173 15819 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:11.159524 16038 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.192443 16036 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:05:11.210368 15835 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:11.211148 15970 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:11.211308 16049 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:05:11.211786  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:11.212415 15817 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:11.188822 15978 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:11.282836 15147 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:11.290889 15894 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:11.149219  8678 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:05:11.239590  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1429) took 2886 ms. Trace:
I20260709 13:05:11.334790  8680 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:11.157014 15873 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.188024 15990 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14.
I20260709 13:05:11.242177  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:11.245481 15834 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.246120 15880 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:11.243287  8823 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 26.
I20260709 13:05:11.257164  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:11.259734  8675 rpcz_store.cc:276] 0709 13:05:10.161100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.162639 (+1001539us) service_pool.cc:224] Handling call
0709 13:05:11.162996 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.163199 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:11.163223 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:11.163265 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:11.163295 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.163345 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.163453 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:11.083495 15877 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.012s	sys 0.013s
I20260709 13:05:11.346302  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:11.347093  8823 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:11.348376  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:11.349133  8823 raft_consensus.cc:3060] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:11.243741 16019 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.266829  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:11.179999  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 63494.2us
I20260709 13:05:11.269336  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:11.271502  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:11.239840 15340 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:11.241515 15982 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.274731  8796 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:11.187134 16030 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.279366  8752 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:11.280315  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2945) took 1109 ms. Trace:
I20260709 13:05:11.291383  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:11.251412 16043 raft_consensus.cc:2804] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:11.295735  8676 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:05:11.384455  8676 pending_rounds.cc:85] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 14
I20260709 13:05:11.384907  8676 pending_rounds.cc:107] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:05:11.385244  8676 raft_consensus.cc:2889] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:11.297050  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:11.299533  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:11.301474  8672 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:05:11.301925 16035 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.305186  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3149) took 1232 ms. Trace:
I20260709 13:05:11.308455  8753 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:11.281441 15876 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:11.389906  8673 rpcz_store.cc:276] 0709 13:05:10.072931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.000034 (+927103us) service_pool.cc:224] Handling call
0709 13:05:11.000401 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.233334 (+232933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.304934 (+ 71600us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54001fa254. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:11.305123 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":534027648}
I20260709 13:05:11.334043  8674 rpcz_store.cc:276] 0709 13:05:08.353336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.903119 (+1549783us) service_pool.cc:224] Handling call
0709 13:05:09.903461 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.119253 (+1215792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.238474 (+119221us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54002562d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:11.239513 (+  1039us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2706222336}
I20260709 13:05:11.248162 15959 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.306706 16007 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.334268  9352 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:11.341557  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:11.397007  8827 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:11.343602  8679 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:05:11.345105  8675 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:05:11.337666  8830 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:11.281225 15991 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:11.358541  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 113414us
I20260709 13:05:11.367741  8755 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:05:11.366817 16022 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:11.370766  8823 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:05:11.372651  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:11.410254  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 76134.5us
W20260709 13:05:11.373106  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 98517.7us
I20260709 13:05:11.415701  8796 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:11.373586  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:11.326004 16045 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:05:11.376348  8822 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:05:11.378226  8677 rpcz_store.cc:276] 0709 13:05:10.170889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.162651 (+991762us) service_pool.cc:224] Handling call
0709 13:05:11.182661 (+ 20010us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.279916 (+ 97255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.280237 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.151098 16015 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.368673  8645 leader_election.cc:400] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 25. Current term is 26.
I20260709 13:05:11.406121  8827 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:05:11.421406  8645 leader_election.cc:403] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 13:05:11.422297  8673 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:11.351459 15834 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.428473  8825 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:05:11.429131  8794 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:11.431742  8794 leader_election.cc:400] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20.
I20260709 13:05:11.432189  8794 leader_election.cc:403] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:05:11.383145 15817 raft_consensus.cc:2804] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:11.436724  9389 raft_consensus.cc:1240] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 22. Current term is 23. Ops: []
W20260709 13:05:11.425271  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 56740.6us
I20260709 13:05:11.437415  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:11.443677  8645 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:11.447676  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:11.449546  9385 raft_consensus.cc:1240] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:11.369757 16019 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.457994  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:11.424311 16057 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:11.461666 16057 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:05:11.461879 16054 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:05:11.406271 16043 raft_consensus.cc:697] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:11.462920 16057 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.464429 15834 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:11.454799 16051 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:05:11.376510 15340 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.402415 16035 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.383055 15982 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.437819 15894 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:11.468081 15991 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:11.468465 15982 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:11.469398  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:11.470340  9380 raft_consensus.cc:2368] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18.
I20260709 13:05:11.469836 16043 consensus_queue.cc:237] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.437743 15990 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 25. Current term is 26.
I20260709 13:05:11.471884  9381 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:11.449638 16022 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.472931 16022 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.344992 15848 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.475088 15848 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:11.419654 16007 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.478991  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:05:11.480818 16056 log.cc:927] Time spent T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 13:05:11.482014 16055 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:05:11.483755  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:11.484766  9381 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:11.484845 16044 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:11.418440 15774 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:11.485683 15774 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.486181  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:11.389393 16030 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:11.487362 16047 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:05:11.487519  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:11.409634 15876 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.488220  9380 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:11.489169  9382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 1240) took 1824 ms. Trace:
I20260709 13:05:11.490042 15876 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:11.490304 15885 consensus_queue.cc:1175] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" responder_term: 19 status { last_received { term: 17 index: 13 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [16.13-18.14]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:05:11.491057  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:11.492676  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:11.492823 16019 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:11.458549 16052 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:11.384007 15978 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.493381  9381 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:11.395325 15926 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.489602  9382 rpcz_store.cc:276] 0709 13:05:09.665096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.668086 (+  2990us) service_pool.cc:224] Handling call
0709 13:05:09.668418 (+   332us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:09.668939 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.668980 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:09.687310 (+ 18330us) log.cc:844] Serialized 52 byte log entry
0709 13:05:10.735461 (+1048151us) spinlock_profiling.cc:243] Waited 7.13 ms on lock 0x7b18003f0820. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:05:11.138274 (+402813us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:05:11.138318 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.138399 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.488803 (+350404us) raft_consensus.cc:1685] finished
0709 13:05:11.488831 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.489097 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":146,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":21970,"spinlock_wait_cycles":16400128,"thread_start_us":402416,"threads_started":1,"wal-append.queue_time_us":1478943}
I20260709 13:05:11.496882  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:11.459331  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:11.434432 15877 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.502601  9385 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:11.503777  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:11.505553  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 48135.2us
I20260709 13:05:11.512387  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:11.507335  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 75795.4us
I20260709 13:05:11.508555  9387 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:11.508893  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:11.513197  9381 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 23.
I20260709 13:05:11.513351  9382 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:11.515161  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:11.516155  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:11.516310  9387 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:11.509204  8822 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:11.521454  8796 leader_election.cc:400] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18.
I20260709 13:05:11.454651 15147 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:11.522534  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:11.524863  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3151) took 1451 ms. Trace:
I20260709 13:05:11.528542  8680 rpcz_store.cc:276] 0709 13:05:10.077090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.083983 (+1006893us) service_pool.cc:224] Handling call
0709 13:05:11.334481 (+250498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.342387 (+  7906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.524580 (+182193us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54001ff9d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:11.524775 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":550017920}
I20260709 13:05:11.458904 15817 raft_consensus.cc:697] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:11.518393  9353 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:11.557873  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:11.558756  9385 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:11.522027  8796 leader_election.cc:403] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:05:11.532362  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:11.561049  9353 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:11.580348  8647 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.131:37857 timed out after 1.527s (SENT) [suppressed 15 similar messages]
I20260709 13:05:11.514830 15880 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:11.560173  8681 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:05:11.511643 16019 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:11.595927  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 81710.5us
I20260709 13:05:11.383780 16029 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:11.570830  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.594774  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:11.630148  8680 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:11.530287 15978 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:11.605005  8794 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:11.640769  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.645563  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:11.649880  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:11.607887 15147 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:11.651500  8752 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:11.607559 16020 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18.
I20260709 13:05:11.606581 16016 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:11.656919  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:11.600873 16007 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:11.668334  9385 raft_consensus.cc:2410] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:05:11.521994 15877 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.531762 15819 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.129s
I20260709 13:05:11.526517 15982 consensus_queue.cc:1059] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 13:05:11.681562  8754 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:11.652715 15949 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:11.554975 15970 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.127s
I20260709 13:05:11.684741 15873 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:11.683337  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:11.685406  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:11.685015 16030 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:11.596738 16060 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:11.690692  9380 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:05:11.690773  8756 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:11.558954 16052 consensus_queue.cc:1059] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.125s
W20260709 13:05:11.698050  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 328779us
I20260709 13:05:11.700762 15774 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:11.702423 16038 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:11.704288  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.704389  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:11.705312  8681 raft_consensus.cc:2393] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:05:11.705998 16021 consensus_meta.cc:287] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.560s	user 0.015s	sys 0.000s
I20260709 13:05:11.706575 16021 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.707094  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:11.664450 16016 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.631217 16019 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.705870 15875 raft_consensus.cc:3055] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:11.707787  8677 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 23.
I20260709 13:05:11.631801 16044 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:05:11.683907 15147 raft_consensus.cc:697] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:11.739535 15964 raft_consensus.cc:3055] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:11.740103 15964 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:11.739810 15147 consensus_queue.cc:237] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.741302 15964 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.745036  8747 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:11.718678 15657 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.747648  9355 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:11.710120  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:11.749354  8829 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:05:11.659162 15823 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:11.706322  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:11.756031  8681 raft_consensus.cc:2393] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 23: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:11.713286  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:11.662978 16022 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:11.696587 15970 raft_consensus.cc:3055] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:11.715065 16007 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:05:11.722817  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 142510us
I20260709 13:05:11.771780  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:11.772521  8679 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:11.687291 15996 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:11.732810 16021 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:11.737800 15949 raft_consensus.cc:697] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:11.784080 15949 consensus_queue.cc:237] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.708563  8748 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:11.786275  8678 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:11.747673  9388 raft_consensus.cc:2393] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:11.776499 16030 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:05:11.719444 16016 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:11.755079 15990 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:05:11.717077  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.764534  8681 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:05:11.775014 15996 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:11.793895 15996 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.795585 15996 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:11.756165 16064 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:05:11.728320 15875 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:11.775619  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:11.747041 16052 consensus_queue.cc:1243] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:11.765077 15970 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:11.766816  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:11.727644 15817 consensus_queue.cc:237] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:11.716317  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 456362us
W20260709 13:05:11.803257  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:11.867367  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:11.867858  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.771327 16060 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 13:05:11.777491 16007 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.869328 16060 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:11.869772 16060 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:11.870746 15982 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:05:11.880705  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.881522 15875 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.894261 15991 raft_consensus.cc:3055] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:11.894956 15991 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:11.896095 15991 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.897676 15991 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:11.921867 16028 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:11.922704 16028 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:11.924139 16028 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:11.928268  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2734) took 1289 ms (client timeout 1561 ms). Trace:
W20260709 13:05:11.928803  8822 rpcz_store.cc:269] 0709 13:05:10.638995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.640672 (+  1677us) service_pool.cc:224] Handling call
0709 13:05:11.928023 (+1287351us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b54002288d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:11.928197 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":600100736}
I20260709 13:05:11.930652 16072 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.931737  9388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 1245) took 1303 ms. Trace:
I20260709 13:05:11.931481 16072 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:11.932114  9388 rpcz_store.cc:276] 0709 13:05:10.628831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.629686 (+   855us) service_pool.cc:224] Handling call
0709 13:05:11.931550 (+1301864us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b54003b2ed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:11.931680 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1562026368}
I20260709 13:05:11.935563 15959 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:11.841867  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:11.938548  8680 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 22.
I20260709 13:05:11.944690 16015 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:11.947778 16058 log.cc:927] Time spent T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Append to log took a long time: real 0.493s	user 0.005s	sys 0.000s
I20260709 13:05:11.950374 15835 raft_consensus.cc:3055] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:11.984042 15835 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:11.852451  8677 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:05:11.849747 16021 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:11.985497 16021 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.985772 15835 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:11.952324 15990 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:11.991093 15990 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:11.953728  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3221) took 1602 ms. Trace:
I20260709 13:05:11.955943 16035 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:11.993175 15894 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.993964 15894 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:11.885694 16060 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:12.000039 16060 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:12.000624  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2765) took 1248 ms. Trace:
I20260709 13:05:11.766023 15964 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:11.993017  8753 rpcz_store.cc:276] 0709 13:05:10.351216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.089777 (+738561us) service_pool.cc:224] Handling call
0709 13:05:11.090163 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.389982 (+299819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.953337 (+563355us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b5400156dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:11.953637 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500806528}
I20260709 13:05:12.007849 15835 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:12.012650  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.017609 16028 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:11.967520  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 445007us
I20260709 13:05:11.774806 15819 raft_consensus.cc:2804] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:11.862850 15970 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:12.022464 15819 raft_consensus.cc:697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:12.022867 15970 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:11.987926  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.023613 15819 consensus_queue.cc:237] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:12.025547  8794 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:12.026872 16059 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:12.027417 16059 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:12.027441  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.028005 16059 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:12.028718  8753 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:12.056897  8753 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:05:12.057056  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:12.098232  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 322643us
I20260709 13:05:12.098955  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:12.099911  8830 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
W20260709 13:05:12.098290  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 229341us
I20260709 13:05:12.102203  9355 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:12.103874  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:12.105715  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.103152  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.105522  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:12.101536  8794 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:12.001221  8828 rpcz_store.cc:276] 0709 13:05:10.752600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.368039 (+615439us) service_pool.cc:224] Handling call
0709 13:05:11.368468 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.371081 (+  2613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:11.371114 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:11.371567 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 13:05:11.572002 (+200435us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:11.626359 (+ 54357us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.630849 (+  4490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.630942 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.948037 (+317095us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b180022e040. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:05:11.997584 (+ 49547us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b180022e040. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:05:12.000261 (+  2677us) raft_consensus.cc:1685] finished
0709 13:05:12.000297 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.000532 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4126,"prepare.queue_time_us":58920,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":8343,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":3214,"spinlock_wait_cycles":65214464,"thread_start_us":150635,"threads_started":1,"wal-append.queue_time_us":279095}
I20260709 13:05:12.122200  8722 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:12.122562  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.123605  8749 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:05:12.123620  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:12.131994  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.131846  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:12.133709  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:12.134627  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.135000  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:12.135180  8754 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:05:12.135766  8824 raft_consensus.cc:2393] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 17: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:12.136307  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:12.138002  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:12.142676 15839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.004s	sys 0.031s
W20260709 13:05:12.143381 15839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.004s	sys 0.031s
W20260709 13:05:12.149797  9354 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.065s	sys 0.140s
W20260709 13:05:12.150629  9354 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.065s	sys 0.140s
I20260709 13:05:12.138685  8825 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
W20260709 13:05:12.161363  8793 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.554s (SENT)
I20260709 13:05:12.160372  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.160018  8751 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:12.153673  9388 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:05:12.161868  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:12.159332  8677 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 26 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
W20260709 13:05:12.164395  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.153348  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.166129  8754 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:12.168407  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.170082  8821 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:05:12.162591  8793 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:12.163116  8751 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:12.172175  8679 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:12.173871  8756 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:12.173691  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.181934  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:12.183521  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 61391.5us
I20260709 13:05:12.184849  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:12.185704  8675 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:12.139084 15875 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:12.151654 16035 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:12.189414  8752 raft_consensus.cc:2410] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:05:12.133299 15862 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:12.188624  8722 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:12.159464 16039 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:05:12.215512  8675 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:05:12.191983  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:12.216578  8673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.020s	sys 0.034s
I20260709 13:05:12.198855  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.161128 16023 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:12.150946 16036 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:12.203089  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.204255  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.136767 15885 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.218175  8752 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
W20260709 13:05:12.216944  8673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.020s	sys 0.034s
I20260709 13:05:12.221489  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:12.267655  8752 raft_consensus.cc:2376] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16. Re-sending same reply.
I20260709 13:05:12.269197  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.270129  8752 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:05:12.151439 16045 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:05:12.151119 16051 consensus_queue.cc:1059] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
W20260709 13:05:12.273574  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.190971  8751 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:12.275213  8749 raft_consensus.cc:1275] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:05:12.275568  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.189383 16029 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:12.289255  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.300908  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.180087 15839 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:05:12.215701 15603 consensus_meta.cc:287] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.076s	user 0.018s	sys 0.005s
W20260709 13:05:12.292927  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 131112us
I20260709 13:05:12.152509 16069 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.063s
W20260709 13:05:12.298487  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 106556us
I20260709 13:05:12.313345  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:12.314500  8672 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:12.295060  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.315699  8753 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:12.296723  8751 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:12.316437  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:12.202905 15657 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:12.197495 15340 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:12.327387 16069 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.327951 16069 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:12.330240 16069 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:12.335402 15877 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:12.337571 16051 raft_consensus.cc:3055] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:12.338205 16051 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:12.339372 16051 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:12.340993 16051 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:12.301131 16052 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:05:12.306538  8679 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:12.343801 16052 raft_consensus.cc:3055] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:12.189999 15873 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:12.221221 16023 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.346909 16023 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:12.221127 16036 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:12.307958 16039 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:05:12.350948 16036 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:12.220836 15926 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.015s	sys 0.033s
I20260709 13:05:12.394381 15147 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:12.308221  8752 raft_consensus.cc:2368] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19.
I20260709 13:05:12.278065  8755 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 23.
I20260709 13:05:12.317737  8680 raft_consensus.cc:2410] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:05:12.318063  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:12.220258 16012 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:12.319474  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:12.301049 15774 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.300976 15823 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:12.344327 16052 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:12.314785  9352 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:05:12.359742  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.303778 15633 consensus_queue.cc:1175] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7aff04f53d4845a6a4fa491fd2a06145" responder_term: 22 status { last_received { term: 19 index: 15 } last_committed_idx: 15 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 21. Current term is 22. Ops: [21.16-21.16]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:05:12.442906  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 128394us
I20260709 13:05:12.195925 16035 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.443835 16035 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:12.444031  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.445444  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:12.426324  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 113988us
I20260709 13:05:12.440580  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.447885 15147 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:12.449183  8756 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:05:12.449730  8756 pending_rounds.cc:85] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 11
I20260709 13:05:12.450029  8756 pending_rounds.cc:107] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:05:12.441427  8750 raft_consensus.cc:2410] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:05:12.223426  8722 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:12.447079 16015 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:12.450327  8756 raft_consensus.cc:2889] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:12.386071 15926 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.018s	sys 0.045s
W20260709 13:05:12.452427  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.454360  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:12.308362 15839 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:12.295856 15896 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.155s
W20260709 13:05:12.467789  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.491751  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:12.276057 15885 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:12.276871 16022 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.164s
I20260709 13:05:12.428117 16069 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:12.442538 16039 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:05:12.524950  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 73404.7us
I20260709 13:05:12.537489  8824 raft_consensus.cc:3060] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:12.463390  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.564587  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 246652us
I20260709 13:05:12.536103 15839 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:12.490661 16032 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.057s
I20260709 13:05:12.454972 16012 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:05:12.566270  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 342869us
I20260709 13:05:12.576296  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:12.490916 16015 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.495339 15862 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:05:12.428917 16079 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:05:12.451550 16008 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:12.573081  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.626909  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.589476  8824 raft_consensus.cc:2468] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
I20260709 13:05:12.449218 15340 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.171s
I20260709 13:05:12.579947  8647 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:12.595866  8754 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:12.597074 16015 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:12.451782 15774 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:12.564181 16039 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.575277 15926 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.319711 16029 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:12.642041  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.629194  9355 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:12.630108  8750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2817) took 2452 ms (client timeout 2114 ms). Trace:
W20260709 13:05:12.648200  8750 rpcz_store.cc:269] 0709 13:05:10.166289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.183759 (+ 17470us) service_pool.cc:224] Handling call
0709 13:05:12.618837 (+2435078us) spinlock_profiling.cc:243] Waited 2.22 s on lock 0x7b540034a154. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:12.619110 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5097489792}
I20260709 13:05:12.554001 16072 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:12.627280 15877 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:12.632077  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 52206.8us
I20260709 13:05:12.660781  8751 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:05:12.661631  8751 pending_rounds.cc:85] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 15
I20260709 13:05:12.661945  8751 pending_rounds.cc:107] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:05:12.597930 15862 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:12.662714 15862 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:12.601543 16007 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:12.617344 16021 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:12.640764  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.593444 15603 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:05:12.574384  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:05:12.677536  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 103275us
I20260709 13:05:12.625139 16019 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:12.598490 15876 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:05:12.575670 16069 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:12.646057 16044 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:12.664337  8824 raft_consensus.cc:3055] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:12.662012  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:12.692374  8677 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:12.662361  8751 raft_consensus.cc:2889] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:12.641575 16039 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:12.661208 15848 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.186s	user 0.015s	sys 0.018s
I20260709 13:05:12.699436  8794 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:12.657606  8747 consensus_meta.cc:287] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.912s	user 0.015s	sys 0.029s
I20260709 13:05:12.687121 15147 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:12.688668  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.637666 16085 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:12.615865 16008 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:12.701545  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:12.700877  8794 leader_election.cc:400] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19.
I20260709 13:05:12.696292  8824 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:12.598939 16052 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:12.707942 15147 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:12.697320  9388 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:05:12.679347  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:12.695443  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 203202us
I20260709 13:05:12.675563 16072 raft_consensus.cc:2804] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:12.708382  8794 leader_election.cc:403] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:12.699447  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:12.702203  8747 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 13:05:12.715355  8747 pending_rounds.cc:85] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 12
I20260709 13:05:12.715826  8747 pending_rounds.cc:107] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:05:12.716219  8747 raft_consensus.cc:2889] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:12.705020  8793 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.019s	sys 0.027s
I20260709 13:05:12.711834  8681 raft_consensus.cc:2368] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16.
W20260709 13:05:12.717721  8793 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.020s	sys 0.027s
W20260709 13:05:12.718063  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 546406us
W20260709 13:05:12.727171  8793 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.690491 16032 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:05:12.758667  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 59275.5us
I20260709 13:05:12.682585 15877 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:12.736763 15147 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:12.737610  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:05:12.703272 15970 consensus_meta.cc:287] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.680s	user 0.015s	sys 0.018s
W20260709 13:05:12.767226  8793 leader_election.cc:336] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.704s (SENT)
I20260709 13:05:12.767973  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:12.737739  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:12.769857  8673 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 22.
I20260709 13:05:12.763072  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:12.763165  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:12.771224  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 195028us
I20260709 13:05:12.772042  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:12.772446  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:12.772749  8678 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:05:12.773171  8677 raft_consensus.cc:2393] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 21: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:12.773603  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:12.773922  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:12.736745  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2828) took 2453 ms. Trace:
W20260709 13:05:12.752707  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.650091 15926 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:12.757233  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:12.725257 15990 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.734s	user 0.045s	sys 0.005s
I20260709 13:05:12.710763 15873 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:12.775257  8677 raft_consensus.cc:2410] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
W20260709 13:05:12.729439 15863 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
W20260709 13:05:12.729811 15959 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.793s	user 0.019s	sys 0.031s
I20260709 13:05:12.709332  8824 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:12.777956 15991 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.880s	user 0.017s	sys 0.021s
I20260709 13:05:12.777092  8747 rpcz_store.cc:276] 0709 13:05:10.283329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.283505 (+   176us) service_pool.cc:224] Handling call
0709 13:05:10.283959 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.037517 (+753558us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b54003fa320. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2470dc0 00007f32e24fb1c0 00007f32e24fc231 00007f32e24f8d8f 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c
0709 13:05:11.744440 (+706923us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b58004a3fd0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24550aa 00007f32e24d7447 00007f32e24effe7 00007f32e24f17de 00007f32e24fbb49 00007f32e24fc416 00007f32e24f8d8f 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4
0709 13:05:12.716892 (+972452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.736662 (+ 19770us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":977844864}
I20260709 13:05:12.804584  8830 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:05:12.805413  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.806735  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.704471 16085 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:12.783403 15147 consensus_queue.cc:1059] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:05:12.786451  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:12.786286  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:12.691236 16007 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 13:05:12.800379  8824 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:12.781512  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.800714  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 216904us
I20260709 13:05:12.819797  8678 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:05:12.776365 16083 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.758694 16032 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:12.783975 15880 consensus_meta.cc:287] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.175s	user 0.023s	sys 0.028s
I20260709 13:05:12.785851 15970 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:12.816639  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:12.759811 15476 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19.
I20260709 13:05:12.803345  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:12.836697 16059 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.795s	user 0.023s	sys 0.002s
I20260709 13:05:12.675091 16012 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:12.827400  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:12.872349 15774 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:12.815526 16085 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:12.872934 15774 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.873427 15774 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:12.823316 15959 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:12.858306 15880 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20.
I20260709 13:05:12.876921 15880 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:12.877481 15880 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:12.831048  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:12.878722 15340 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:12.879228 15340 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.879743 15340 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:12.783324 16052 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:12.883821 15894 consensus_meta.cc:287] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.889s	user 0.010s	sys 0.011s
I20260709 13:05:12.884351 15894 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:12.844767 16083 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:12.838775 15885 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:12.887260 15885 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.887784 15885 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:12.895442  9386 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 903) took 1483 ms. Trace:
I20260709 13:05:12.890180 16039 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:12.896041  9386 rpcz_store.cc:276] 0709 13:05:11.412014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.412144 (+   130us) service_pool.cc:224] Handling call
0709 13:05:12.895174 (+1483030us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400469954. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:12.895363 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2946920192}
I20260709 13:05:12.898605 16039 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:12.786010 15873 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.899389 15873 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:12.859896 16077 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 13:05:12.904846  8674 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:12.704641 16044 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:12.834218  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.843412  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:12.759523 16057 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:05:12.848114  9384 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.002s	sys 0.000s
W20260709 13:05:12.849112  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 87873.6us
W20260709 13:05:12.856209  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 88992.7us
I20260709 13:05:12.788419 16089 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:12.863662  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 60422.8us
I20260709 13:05:12.935057  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:12.823089  8719 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:05:12.840595 15835 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.832s	user 0.017s	sys 0.004s
I20260709 13:05:12.817500 16045 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.867103  9382 raft_consensus.cc:1240] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:12.858371 15970 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.871342  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:12.956162  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:12.871628  8679 raft_consensus.cc:1240] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:12.957347  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:12.822767 15982 raft_consensus.cc:3055] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:12.967031  8829 raft_consensus.cc:2410] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 13:05:12.879307 16086 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:05:12.775591 16072 raft_consensus.cc:697] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:12.898725  9389 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
W20260709 13:05:12.842764  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 54912.3us
W20260709 13:05:12.984273  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.913672  8674 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20. Ops: []
W20260709 13:05:12.928192  8793 leader_election.cc:336] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.941s (SENT)
I20260709 13:05:12.928166  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:12.914453  8824 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:05:12.985880  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 57735.5us
W20260709 13:05:12.910432  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.986538  8793 leader_election.cc:336] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 2.036s (SENT)
I20260709 13:05:12.844933 16032 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:12.946171  8645 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:12.787905 15990 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:12.863013 16095 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:12.961897  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 138886us
W20260709 13:05:12.967701 16016 consensus_meta.cc:287] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.176s	user 0.010s	sys 0.018s
I20260709 13:05:12.911866 15991 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:12.930591  9384 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.003s	sys 0.000s
W20260709 13:05:12.987166  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.968374  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:13.009838  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:12.933826 15894 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:12.965996 15970 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:12.949582 16094 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:05:13.029443  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 83387.5us
I20260709 13:05:13.030144  8680 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:12.981856 16044 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:12.909869 16059 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.025405  8749 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
W20260709 13:05:13.064525  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 52515.8us
I20260709 13:05:12.966367 15982 raft_consensus.cc:740] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:12.973415 16083 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:13.030937  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:12.944540 16089 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:12.972962 16096 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:12.948848 16057 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:13.020268 15964 consensus_meta.cc:287] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.018s	user 0.012s	sys 0.007s
I20260709 13:05:13.027647 15991 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.077491  8794 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:13.078951 16069 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:13.073371 16096 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:05:13.079968 15875 consensus_meta.cc:287] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.882s	user 0.008s	sys 0.004s
W20260709 13:05:13.081552  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.082646  8825 pending_rounds.cc:85] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 12
I20260709 13:05:13.097826  8825 pending_rounds.cc:107] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:05:13.098223  8825 raft_consensus.cc:2889] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:13.060194 15885 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:13.084574 16059 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:13.068486 16029 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:13.072698 16044 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.013s
I20260709 13:05:13.019853 16016 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.059890 16007 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.093262 15991 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:12.972278 16045 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:13.080024 16087 log.cc:927] Time spent T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 13:05:12.974467 15877 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.126762  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:13.127456  8793 leader_election.cc:400] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 15. Current term is 16.
W20260709 13:05:13.176690  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 49450us
I20260709 13:05:13.094442 16072 consensus_queue.cc:237] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:13.123160  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 92284.2us
I20260709 13:05:13.092677 16083 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
W20260709 13:05:13.102342  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.184008  8645 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:13.140056 16069 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:05:13.091289 15863 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.152866  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:13.177561  9380 raft_consensus.cc:1240] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:05:13.013432 16095 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.071218 16089 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:13.101650 16096 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:13.176757  8722 connection.cc:256] RPC call timeout handler was delayed by -1.1444264509999584s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
I20260709 13:05:13.177902  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:13.193622  8722 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:13.138326  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.155407 16028 consensus_meta.cc:287] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.137s	user 0.022s	sys 0.002s
I20260709 13:05:13.068423 16094 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.188019  8681 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
W20260709 13:05:13.184008  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 255912us
I20260709 13:05:13.093775 15873 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.190050  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:13.198225  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:13.179571 16019 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:13.199509  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:13.212805  8827 raft_consensus.cc:2393] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 14: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:13.128880 16088 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:13.201326  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:13.148531 15896 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:13.210103  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:13.226440  8755 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:13.210436  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:13.145340 15880 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:13.221084  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:13.212119  8828 raft_consensus.cc:2393] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 22: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:05:13.220465  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.219695  8822 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:13.228961  8681 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:13.240106  8823 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
W20260709 13:05:13.197912 16036 consensus_meta.cc:287] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.846s	user 0.008s	sys 0.005s
I20260709 13:05:13.185549 15875 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.192914 16039 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.185055 15774 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:13.255554  8828 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:13.147886 16007 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:13.237905  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 111009us
I20260709 13:05:13.233546  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:13.261974  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:05:13.290071  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.236243 16060 consensus_meta.cc:287] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.230s	user 0.014s	sys 0.011s
W20260709 13:05:13.271909  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.297385  8722 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:13.265928 16045 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:13.210477 15657 consensus_queue.cc:1048] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:05:13.233927 15926 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:13.216775 15949 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:13.294823  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:13.211633 16108 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.295487  8822 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:13.285948  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1505) took 1155 ms. Trace:
I20260709 13:05:13.300678  9387 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:13.235692  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 1252) took 1083 ms. Trace:
I20260709 13:05:13.306128  8681 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:13.171077 16085 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.148825 15982 consensus_queue.cc:260] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.226543 15964 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:13.228879 16019 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:05:13.253283 15839 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.654s	user 0.008s	sys 0.004s
I20260709 13:05:13.355098  8673 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:13.226897 16088 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:13.293588  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:13.235301 16059 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.308501 15885 raft_consensus.cc:2764] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 13:05:13.311642  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.311570 16121 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:13.290236 16039 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.315629  8827 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:13.200834 15863 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.242594 15896 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.259210 16022 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:13.295269 16099 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:13.335762  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 139046us
I20260709 13:05:13.296128 15880 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:13.389951  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:13.325100  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1501) took 1456 ms. Trace:
I20260709 13:05:13.332926  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:13.326453  8830 rpcz_store.cc:276] 0709 13:05:12.131158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.139650 (+  8492us) service_pool.cc:224] Handling call
0709 13:05:12.140059 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.805043 (+664984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.285649 (+480606us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b54002292d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:13.285873 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1527004544}
I20260709 13:05:13.317443 16012 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:05:13.331415  9384 rpcz_store.cc:276] 0709 13:05:12.152492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.152637 (+   145us) service_pool.cc:224] Handling call
0709 13:05:12.178458 (+ 25821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.181705 (+  3247us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 13:05:13.016827 (+835122us) log.cc:844] Serialized 14 byte log entry
0709 13:05:13.212063 (+195236us) log.cc:844] Serialized 14 byte log entry
0709 13:05:13.235160 (+ 23097us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.235232 (+    72us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:05:13.235290 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.235466 (+   176us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.235627 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21540,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":261,"thread_start_us":970923,"threads_started":2}
I20260709 13:05:13.233410 16101 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.119s
I20260709 13:05:13.313895 15990 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:13.335137 16082 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:13.251664 16036 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.288224 15875 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:13.329705 16051 consensus_meta.cc:287] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.988s	user 0.010s	sys 0.012s
I20260709 13:05:13.360055 16085 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:13.332454 15926 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:13.362237 16059 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.244254 16030 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:13.334201 16043 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.373843  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:13.322571 16122 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:13.408520 15880 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.391767  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:13.479522  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:13.480903  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:13.346077  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:13.355587 16019 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:13.362797 15839 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.343051 16083 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:05:13.390899  8826 rpcz_store.cc:276] 0709 13:05:12.099989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.100191 (+   202us) service_pool.cc:224] Handling call
0709 13:05:12.167742 (+ 67551us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:12.177771 (+ 10029us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:12.220264 (+ 42493us) log.cc:844] Serialized 14 byte log entry
0709 13:05:12.634207 (+413943us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:12.640042 (+  5835us) log.cc:844] Serialized 26 byte log entry
0709 13:05:12.640201 (+   159us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:12.640238 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.640321 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:13.324877 (+684556us) raft_consensus.cc:1685] finished
0709 13:05:13.324902 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.325038 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":433,"prepare.queue_time_us":86958,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":58,"raft.queue_time_us":3644,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":14944,"thread_start_us":305887,"threads_started":1,"wal-append.queue_time_us":475713}
I20260709 13:05:13.363415 16088 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:13.431802  8672 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:13.385979 15949 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.363899 16069 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:05:13.440798  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 230790us
I20260709 13:05:13.386686 16123 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.460330 16134 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:13.476504 16051 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:13.436014 16092 log.cc:927] Time spent T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.355s	user 0.000s	sys 0.002s
I20260709 13:05:13.336166 16108 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.468343  8827 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:13.472373  8828 raft_consensus.cc:2410] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:05:13.324150 16110 raft_consensus.cc:2764] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:05:13.482154  8822 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 20: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:13.445396 16082 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:13.488427  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:13.453111 15873 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:13.493376  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:13.474658 16109 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.280s	user 0.001s	sys 0.001s
I20260709 13:05:13.459513 15926 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:05:13.515882 16097 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.308s	user 0.002s	sys 0.000s
I20260709 13:05:13.556777  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:13.498531  8647 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:13.501312 16039 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:13.501627 16104 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:05:13.510262 15991 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.515004  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:13.567867  8826 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:13.567329  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:13.502698 15949 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.515879 16036 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:13.516268 16007 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:13.491608 16023 consensus_meta.cc:287] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.144s	user 0.008s	sys 0.007s
I20260709 13:05:13.520478  8675 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 22. Current term is 23. Ops: []
W20260709 13:05:13.531342  8829 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2748) took 1758 ms (client timeout 2053 ms). Trace:
I20260709 13:05:13.542464 16082 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:13.533630  8827 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 20. Current term is 21. Ops: [19.12-20.13]
I20260709 13:05:13.538213  8823 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:13.548062  8828 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 26 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 13:05:13.548396  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:13.555709  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:13.483289 16043 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:13.562520  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 64075us
I20260709 13:05:13.537034 16044 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.567329  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:13.490329 16122 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:13.530732 16012 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.558408 16094 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:13.459632 15990 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:05:13.532012 16051 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.569558 16085 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:13.596498 16095 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:13.604619  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:13.606227  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:13.606848  8679 raft_consensus.cc:1240] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:13.590559 16044 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.608659 16118 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:13.609716 16035 consensus_meta.cc:287] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.165s	user 0.022s	sys 0.011s
W20260709 13:05:13.610491  8754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2840) took 1443 ms (client timeout 1540 ms). Trace:
I20260709 13:05:13.611725  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:13.641517  8749 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:13.613222  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:13.642156  8680 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 18: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:13.590456  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:13.642283  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3234) took 1340 ms. Trace:
I20260709 13:05:13.642566  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:13.575555  8829 rpcz_store.cc:269] 0709 13:05:11.772880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.777505 (+  4625us) service_pool.cc:224] Handling call
0709 13:05:12.133544 (+356039us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b58001a1698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:05:13.518948 (+1385404us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54002297d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:13.531266 (+ 12318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2651362048}
I20260709 13:05:13.589736  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:13.616571  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.586835 16139 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.642992  8751 rpcz_store.cc:276] 0709 13:05:12.302244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.708058 (+405814us) service_pool.cc:224] Handling call
0709 13:05:12.708454 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:12.731503 (+ 23049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:12.731561 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:12.765529 (+ 33968us) log.cc:844] Serialized 26 byte log entry
0709 13:05:13.025410 (+259881us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:13.025496 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.025604 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:13.641957 (+616353us) raft_consensus.cc:1685] finished
0709 13:05:13.641993 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.642197 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3110,"raft.queue_time_us":49769,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":15109,"thread_start_us":202001,"threads_started":1,"wal-append.queue_time_us":302560}
I20260709 13:05:13.567410 16136 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.625535  8822 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
W20260709 13:05:13.591583 16126 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:05:13.635869  8673 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:13.637236  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:13.638695  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:13.604511 16008 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:13.654953  8672 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 20: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:05:13.641111  8754 rpcz_store.cc:269] 0709 13:05:12.167107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.173369 (+  6262us) service_pool.cc:224] Handling call
0709 13:05:13.610280 (+1436911us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b540038c2d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:13.610422 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":968089984}
I20260709 13:05:13.568006 15633 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:13.563556 15926 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:13.654141  8821 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:13.530866 16123 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.649757  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:13.590842 15896 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:13.642702  8826 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:13.657027  8829 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:05:13.667693  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:13.668077  8829 pending_rounds.cc:85] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 8
I20260709 13:05:13.668403  8829 pending_rounds.cc:107] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:05:13.663036  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3229) took 1477 ms. Trace:
W20260709 13:05:13.636451 16015 consensus_meta.cc:287] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.986s	user 0.008s	sys 0.007s
I20260709 13:05:13.658510  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:05:13.648280  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 58645.6us
W20260709 13:05:13.658286 15978 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 2.006s	user 0.014s	sys 0.018s
I20260709 13:05:13.624488 16085 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.668692  8829 raft_consensus.cc:2889] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:13.572692 16134 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:05:13.684386  8673 raft_consensus.cc:2376] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14. Re-sending same reply.
I20260709 13:05:13.685101  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:13.668941  8752 rpcz_store.cc:276] 0709 13:05:12.185726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.445128 (+259402us) service_pool.cc:224] Handling call
0709 13:05:12.567298 (+122170us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:12.584594 (+ 17296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:12.584646 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:12.616735 (+ 32089us) log.cc:844] Serialized 26 byte log entry
0709 13:05:12.756702 (+139967us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:12.847536 (+ 90834us) log.cc:844] Serialized 14 byte log entry
0709 13:05:12.847796 (+   260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.847893 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:13.662796 (+814903us) raft_consensus.cc:1685] finished
0709 13:05:13.662823 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.662969 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14418,"prepare.queue_time_us":305914,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":53869,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":10792,"thread_start_us":175274,"threads_started":2,"wal-append.queue_time_us":498664}
I20260709 13:05:13.656078 16035 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.585357 16023 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:13.664673 15970 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.596s	user 0.013s	sys 0.000s
W20260709 13:05:13.661922 15340 consensus_meta.cc:287] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.782s	user 0.014s	sys 0.000s
I20260709 13:05:13.701182  8829 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:05:13.702699 15147 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:13.721969 15147 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.722433 15147 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:13.724546 15949 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:13.730548  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:13.731654 16032 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.718s	user 0.012s	sys 0.002s
I20260709 13:05:13.732245 16032 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.697217 16085 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:13.735633 15839 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:13.706655 15862 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.035s	user 0.011s	sys 0.004s
I20260709 13:05:13.736816 15862 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.703882 16124 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 13:05:13.661584 16136 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.703281  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:13.621942 16012 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.668416 15633 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:05:13.753863  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.755673  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:13.659904 16069 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:13.762306  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:13.704697  8747 raft_consensus.cc:2410] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:05:13.731763  8751 raft_consensus.cc:2410] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:05:13.744467  8679 raft_consensus.cc:1275] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 8. (term mismatch)
I20260709 13:05:13.754945  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:13.784385  9382 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:13.786227 15147 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.789911  8826 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:13.751195  8676 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:13.791374  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3170) took 1091 ms. Trace:
I20260709 13:05:13.755327  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:13.776002  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 72807us
I20260709 13:05:13.792788  9382 raft_consensus.cc:2468] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 22.
I20260709 13:05:13.793221  9385 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:13.756191  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:13.793879  8679 pending_rounds.cc:85] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 7
I20260709 13:05:13.755229  8821 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:05:13.793875  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:13.794546  8679 pending_rounds.cc:107] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 17.8
I20260709 13:05:13.794891  8679 raft_consensus.cc:2889] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:13.739699  8753 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 22. Current term is 23. Ops: []
W20260709 13:05:13.778249  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.717819 15996 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.867s	user 0.017s	sys 0.011s
I20260709 13:05:13.781332  9383 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:13.654765 16118 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.791869  8676 rpcz_store.cc:276] 0709 13:05:12.700440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.700581 (+   141us) service_pool.cc:224] Handling call
0709 13:05:13.750633 (+1050052us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b5800003698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:05:13.750948 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.791007 (+ 40059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.791300 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5101952}
I20260709 13:05:13.812085  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:13.809623  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:13.812765  8825 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:13.813123  8751 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:13.799417  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:14.204707  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:14.690673  8754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.959s	user 0.055s	sys 0.092s
I20260709 13:05:14.691412  8748 raft_consensus.cc:2410] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:05:14.692179  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:13.915190  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.694875  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:13.718822 15340 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:14.177299 15633 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:14.757579  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2964) took 2496 ms. Trace:
I20260709 13:05:13.811167 15839 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:13.809008 15848 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.771672 15848 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.771163 15875 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:13.811301 15896 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:13.719951 15970 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:14.758466  8675 rpcz_store.cc:276] 0709 13:05:12.261343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.261579 (+   236us) service_pool.cc:224] Handling call
0709 13:05:12.262032 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.575551 (+1313519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.756535 (+1180984us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400240ed4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:14.757191 (+   656us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2893113728}
I20260709 13:05:14.779670 15970 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:13.797364 15978 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.781083  9386 raft_consensus.cc:2368] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 22. Current term is 23.
I20260709 13:05:14.782027 15978 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:14.784718 16012 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:14.785997 15978 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:13.768872 16032 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:13.739118 16044 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:14.828564 16044 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:05:14.828876  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.829687 16044 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.748085 16094 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.814472 16069 raft_consensus.cc:697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:14.835006 16094 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:14.836264 16069 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.799588 16124 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:13.659055 16139 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:13.872618 16083 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.866240 16142 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.918869  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:14.919440 16142 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:14.921082 16095 consensus_queue.cc:1059] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:14.922945  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:14.080302 16143 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 13:05:13.770093 16122 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:13.748788 16123 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:14.287770 16128 log.cc:927] Time spent T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.509s	user 0.000s	sys 0.002s
W20260709 13:05:14.091741 16138 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.386s	user 0.001s	sys 0.000s
I20260709 13:05:14.955221  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1463) took 2172 ms. Trace:
W20260709 13:05:14.736706  8754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.056s	sys 0.092s
I20260709 13:05:14.957006  8722 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:14.957269 15875 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:14.940886 16142 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:14.958298 15875 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:14.762032 16136 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:14.766433  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:13.689702 16015 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:14.962287 16101 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.963274 16101 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:14.965076 16015 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:14.966189 16101 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:14.966377 16136 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:14.967238  8821 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:14.967367 16095 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:14.946768 16012 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:14.968992 16012 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:14.968631 16095 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:14.969442 16082 consensus_meta.cc:287] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.353s	user 0.010s	sys 0.018s
I20260709 13:05:14.970094 16082 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:14.970827 16012 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:14.972939 16082 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:14.790364 16029 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.974337 16029 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:14.976527  9388 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:05:14.977913 16029 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:14.793138 16153 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.801362 16118 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:14.805567 16023 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:14.808143 16035 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:14.808596  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:14.806214  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:14.817435  9383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 1023) took 1971 ms. Trace:
I20260709 13:05:14.813035  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2788) took 1268 ms. Trace:
I20260709 13:05:14.819957 15839 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:14.821662 15340 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:13.721710 16059 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:14.810564  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.811677 15848 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:14.817265 16051 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:14.848466  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:14.997459  8826 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:14.961606 15978 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:14.999037 15982 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:14.852200 15863 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:14.852062 16044 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:14.887518 16094 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:14.991308  8825 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:14.999394 15978 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:14.898037 15880 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:14.899226 16083 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:14.899791 16057 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:14.902330 16157 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:14.905185  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:14.909801 16155 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:14.938153  8749 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:14.918308 16139 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:14.936281 16096 consensus_meta.cc:287] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.639s	user 0.021s	sys 0.023s
I20260709 13:05:14.946441  8750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2858) took 1390 ms. Trace:
I20260709 13:05:14.952666 15633 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.954828 15896 raft_consensus.cc:3055] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:14.955801  8681 rpcz_store.cc:276] 0709 13:05:12.782208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.391550 (+609342us) service_pool.cc:224] Handling call
0709 13:05:13.391940 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.418907 (+ 26967us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.418941 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:13.419640 (+   699us) log.cc:844] Serialized 26 byte log entry
0709 13:05:13.574249 (+154609us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:13.574294 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.574400 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:14.954521 (+1380121us) raft_consensus.cc:1685] finished
0709 13:05:14.954572 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:14.954948 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26535,"raft.queue_time_us":47185,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":8354,"thread_start_us":143507,"threads_started":1,"wal-append.queue_time_us":287354}
W20260709 13:05:14.766579 16052 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.883s	user 0.018s	sys 0.003s
I20260709 13:05:14.788792  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2863) took 1058 ms. Trace:
W20260709 13:05:14.789104  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 989460us
I20260709 13:05:14.973062 15147 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:14.792404 16043 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:15.010485  8751 rpcz_store.cc:276] 0709 13:05:13.729992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.850831 (+120839us) service_pool.cc:224] Handling call
0709 13:05:14.788563 (+937732us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:14.966531  9389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 1249) took 3525 ms. Trace:
I20260709 13:05:14.985805  9380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 1254) took 2830 ms. Trace:
I20260709 13:05:14.984048  8822 rpcz_store.cc:276] 0709 13:05:13.544893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.564298 (+ 19405us) service_pool.cc:224] Handling call
0709 13:05:14.812463 (+1248165us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b540016ba54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:14.812769 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129511168}
I20260709 13:05:14.986424 15839 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:14.987627  8674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2969) took 2213 ms. Trace:
I20260709 13:05:14.983737  9383 rpcz_store.cc:276] 0709 13:05:12.845529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.845722 (+   193us) service_pool.cc:224] Handling call
0709 13:05:12.874959 (+ 29237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.796994 (+922035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.816513 (+1019519us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b540061b554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:14.817016 (+   503us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2083517824}
I20260709 13:05:14.988947  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2866) took 1191 ms. Trace:
I20260709 13:05:14.989217  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:14.983031 16153 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:14.993326  9382 raft_consensus.cc:1240] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:05:15.000603 15476 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:15.003545 15978 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.010136 15896 raft_consensus.cc:740] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:15.011478 15848 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:05:15.012722  8674 rpcz_store.cc:276] 0709 13:05:12.773746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.986029 (+212283us) service_pool.cc:224] Handling call
0709 13:05:12.986505 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.220060 (+233555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.220125 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:13.233055 (+ 12930us) log.cc:844] Serialized 26 byte log entry
0709 13:05:13.385424 (+152369us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:13.396379 (+ 10955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.397120 (+   741us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:14.979477 (+1582357us) raft_consensus.cc:1685] finished
0709 13:05:14.981877 (+  2400us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:14.987363 (+  5486us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":323480,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":8389,"thread_start_us":302658,"threads_started":2,"wal-append.queue_time_us":332376}
I20260709 13:05:15.013016  8755 rpcz_store.cc:276] 0709 13:05:13.797058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.805279 (+1008221us) service_pool.cc:224] Handling call
0709 13:05:14.988812 (+183533us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:15.015800 15848 consensus_queue.cc:1243] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:15.038861  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:15.039336  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:15.041402 16016 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.045667 16139 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:15.034101  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:15.048681 16057 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:15.012127  9389 rpcz_store.cc:276] 0709 13:05:11.441439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.443137 (+  1698us) service_pool.cc:224] Handling call
0709 13:05:12.898374 (+1455237us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:12.982339 (+ 83965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.964564 (+1982225us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b540046a120. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2470dc0 00007f32e24fca02 00007f32e24f8dd6 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2
0709 13:05:14.966066 (+  1502us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400469950. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:14.966312 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3278680192}
I20260709 13:05:15.036969 16153 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.018568  8829 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 22.
I20260709 13:05:15.018677 16096 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.053746 16096 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.020201 16083 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.020705 15896 consensus_queue.cc:260] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.051373 16153 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.004s
I20260709 13:05:15.056234 15896 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:15.056926  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 53924.2us
I20260709 13:05:15.012216  9380 rpcz_store.cc:276] 0709 13:05:12.153306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.153438 (+   132us) service_pool.cc:224] Handling call
0709 13:05:12.153811 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.186645 (+1032834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.982605 (+1795960us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001a3bd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:14.983195 (+   590us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2353446656}
I20260709 13:05:15.021392  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:15.058451  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:15.009344  8750 rpcz_store.cc:276] 0709 13:05:13.556205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.665765 (+109560us) service_pool.cc:224] Handling call
0709 13:05:13.666096 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.693453 (+ 27357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.693493 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:13.699038 (+  5545us) log.cc:844] Serialized 26 byte log entry
0709 13:05:13.866110 (+167072us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:13.866177 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.866288 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:14.945839 (+1079551us) raft_consensus.cc:1685] finished
0709 13:05:14.945900 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:14.946185 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17208,"raft.queue_time_us":28542,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":7078,"spinlock_wait_cycles":953600,"thread_start_us":166152,"threads_started":1,"wal-append.queue_time_us":1242271}
I20260709 13:05:15.060436  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:15.022213  9381 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:15.022930 15476 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.023037 16157 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:15.024806 15926 consensus_meta.cc:287] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.357s	user 0.019s	sys 0.036s
I20260709 13:05:15.024355 15633 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.066871 16157 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.069561 15633 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.025528 16155 raft_consensus.cc:697] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:05:15.072003 16151 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.267s	user 0.004s	sys 0.000s
I20260709 13:05:15.071671 16155 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.059011  8751 raft_consensus.cc:2468] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:05:15.027376 15875 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.030923 16105 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.020211  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:15.032521  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:15.036965  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2854) took 1481 ms. Trace:
I20260709 13:05:15.011523 15863 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.073817  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3238) took 2337 ms. Trace:
I20260709 13:05:15.084357  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:15.086737 15863 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.087929  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:15.088771 15848 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:15.089389 15848 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.089960 15863 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.084456 15926 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.090981  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:15.089572  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:15.077178  8747 rpcz_store.cc:276] 0709 13:05:13.555085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.555280 (+   195us) service_pool.cc:224] Handling call
0709 13:05:15.036470 (+1481190us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b540031d654. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:15.036739 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280882432}
I20260709 13:05:15.090983 16105 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.096947  8828 raft_consensus.cc:2410] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:05:15.097632 16030 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:15.109782  9381 raft_consensus.cc:2393] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:15.098335 16030 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:15.063421 16124 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:05:15.112278  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 117294us
I20260709 13:05:15.112731 16095 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.114313  9355 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:15.173851 16072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.467s	user 0.014s	sys 0.040s
W20260709 13:05:15.174780 16072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.470s	user 0.015s	sys 0.040s
I20260709 13:05:15.174918 16016 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:15.175624 16016 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.114828 16051 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:05:15.184767  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.116927 16043 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.118503  8749 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:05:15.119268  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:15.078042  8756 rpcz_store.cc:276] 0709 13:05:12.736733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.141510 (+404777us) service_pool.cc:224] Handling call
0709 13:05:13.172080 (+ 30570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.181731 (+  9651us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.181776 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:13.182343 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 13:05:13.752254 (+569911us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b1800451fc0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:05:14.013075 (+260821us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:14.013143 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.013266 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:15.071027 (+1057761us) spinlock_profiling.cc:243] Waited 38.4 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:15.073326 (+  2299us) raft_consensus.cc:1685] finished
0709 13:05:15.073350 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:15.073598 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9159,"raft.queue_time_us":41561,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":11887,"spinlock_wait_cycles":95060096,"thread_start_us":260552,"threads_started":1,"wal-append.queue_time_us":1609451}
W20260709 13:05:15.186216  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 71753.1us
I20260709 13:05:15.076761 15978 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.066907 15476 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.089969 15848 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:15.169981  8825 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:05:15.123302  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:15.176200 16016 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:15.177542  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 63507us
I20260709 13:05:15.206851  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:15.211794  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:15.203990  9386 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34754 (request call id 1026) took 2286 ms (client timeout 2468 ms). Trace:
W20260709 13:05:15.215911  9386 rpcz_store.cc:269] 0709 13:05:12.909530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.909713 (+   183us) service_pool.cc:224] Handling call
0709 13:05:15.110811 (+2201098us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54001ccd54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:15.195749 (+ 84938us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4190420352}
I20260709 13:05:15.223732  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:15.224090 16085 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.472s	user 0.005s	sys 0.045s
I20260709 13:05:15.224678  8750 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:05:15.225436 16160 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.226267 16160 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.226963  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:15.228032  8823 raft_consensus.cc:2368] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14.
I20260709 13:05:15.230041  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:15.223758  8566 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7aff04f53d4845a6a4fa491fd2a06145 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:15.226018  8749 raft_consensus.cc:1275] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:05:15.231508  8796 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:15.222159  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:15.123262 16153 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.091s
I20260709 13:05:15.242096  8826 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:05:15.267730 15880 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:15.272490  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 315745us
I20260709 13:05:15.229604 16160 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.276368  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:15.220731 15978 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.262053 16057 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:15.275146 15880 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.281533  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 1260) took 1444 ms. Trace:
I20260709 13:05:15.287016  9384 rpcz_store.cc:276] 0709 13:05:13.753848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.755821 (+  1973us) service_pool.cc:224] Handling call
0709 13:05:13.809727 (+ 53906us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.809998 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.810024 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:13.810531 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 13:05:14.898630 (+1088099us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:14.898767 (+   137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.898947 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:15.196417 (+297470us) raft_consensus.cc:1685] finished
0709 13:05:15.196439 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:15.198450 (+  2011us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":196741,"threads_started":1,"wal-append.queue_time_us":1053881}
I20260709 13:05:15.265662 15926 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.210157  8747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:05:15.291549  8747 raft_consensus.cc:606] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:15.291993  8747 raft_consensus.cc:614] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:15.224731 16085 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.298113  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:15.288666 15880 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:15.301265  8796 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:15.270665 16051 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.264534 16043 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.347838 16043 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.348634 16043 raft_consensus.cc:2804] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:15.349156 16043 raft_consensus.cc:697] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 22 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:15.350806 16122 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.351620 16122 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.355522  8676 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:15.357682  8674 raft_consensus.cc:1275] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:05:15.359400  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:15.296015 16160 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:15.317382 16089 consensus_meta.cc:287] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 2.105s	user 0.028s	sys 0.005s
W20260709 13:05:15.325984 16019 consensus_meta.cc:287] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 1.827s	user 0.013s	sys 0.017s
I20260709 13:05:15.279448 16095 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
W20260709 13:05:15.328964  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.335201 16069 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.334820 16124 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.337267 15978 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.304940 16057 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:15.304514 15863 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:15.340818  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 57758.7us
I20260709 13:05:15.311154 15926 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.344018  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:15.344461 16085 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.362375  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:15.392220  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:15.317472 16088 consensus_meta.cc:287] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.797s	user 0.013s	sys 0.017s
I20260709 13:05:15.402160  9387 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:05:15.350126 15839 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.497174 16069 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.498091 16057 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.497174 15863 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.503044 16160 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.505630 16160 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.516553 15863 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:15.518817  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 131785us
I20260709 13:05:15.519800  8827 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:05:15.520788 15862 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.521559 16159 consensus_queue.cc:1059] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:15.497339 15926 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.522964  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:15.523797 16147 log.cc:927] Time spent T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.723s	user 0.002s	sys 0.001s
I20260709 13:05:15.524012  8674 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:05:15.524652 15926 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.525396  8747 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 22.
I20260709 13:05:15.526641  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:15.527653 16150 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.724s	user 0.003s	sys 0.000s
I20260709 13:05:15.528319  8747 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:15.529639 16069 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.529793  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:15.531026  8821 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:05:15.531868  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2857) took 1976 ms. Trace:
I20260709 13:05:15.532095 16122 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.533255  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:15.513566  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:15.541388  8828 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:15.512701 16082 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.394125 16089 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.532284  8754 rpcz_store.cc:276] 0709 13:05:13.555932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.662660 (+106728us) service_pool.cc:224] Handling call
0709 13:05:13.663043 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.668602 (+  5559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.668636 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:13.686063 (+ 17427us) log.cc:844] Serialized 26 byte log entry
0709 13:05:14.956598 (+1270535us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:14.956721 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.956891 (+   170us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:15.526759 (+569868us) spinlock_profiling.cc:243] Waited 36.7 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:15.531500 (+  4741us) raft_consensus.cc:1685] finished
0709 13:05:15.531523 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:15.531795 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5147,"raft.queue_time_us":18017,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":18296,"spinlock_wait_cycles":84468608,"thread_start_us":1225280,"threads_started":1,"wal-append.queue_time_us":1098402}
I20260709 13:05:15.571552  8675 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 21: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:15.567072  9355 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:15.350210 16043 consensus_queue.cc:237] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.399628 16019 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:15.586295  8629 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 110496us
I20260709 13:05:15.594251  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:15.593550  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:15.595345  8678 raft_consensus.cc:2393] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:15.586383  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:15.599502  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:15.602767  8793 connection.cc:256] RPC call timeout handler was delayed by -1.4282770190000065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
I20260709 13:05:15.602516 15848 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:15.603349  8793 connection.cc:256] RPC call timeout handler was delayed by -1.4228770189999977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
W20260709 13:05:15.603701  8793 connection.cc:256] RPC call timeout handler was delayed by -1.350877018999995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
W20260709 13:05:15.604890  8793 connection.cc:256] RPC call timeout handler was delayed by -1.1159770189999563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19730000000000003s for a response
I20260709 13:05:15.605065 15848 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:15.605296  8793 connection.cc:256] RPC call timeout handler was delayed by -1.0871770190000234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2005s for a response
I20260709 13:05:15.605994 16023 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:15.606691 16023 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:05:15.608280 15896 consensus_meta.cc:287] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.551s	user 0.014s	sys 0.001s
I20260709 13:05:15.607828 16023 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.613822  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:15.614727  8754 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:15.615557 16051 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:15.514362 16088 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:15.617211  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.618221 16088 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.619055  8751 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 21: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:05:15.620237  8645 connection.cc:256] RPC call timeout handler was delayed by -1.4430109029999585s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
I20260709 13:05:15.599208  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:15.607904 16082 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:15.626401  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 65424.1us
I20260709 13:05:15.689476 16016 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.691193  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:15.692042  8676 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:15.693578 16082 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.695065  9381 raft_consensus.cc:2368] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 22. Current term is 23.
I20260709 13:05:15.695617  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2975) took 2576 ms. Trace:
W20260709 13:05:15.697732  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.696166  8680 rpcz_store.cc:276] 0709 13:05:13.119105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.680240 (+561135us) service_pool.cc:224] Handling call
0709 13:05:13.689412 (+  9172us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.706373 (+ 16961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.706412 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:13.706998 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 13:05:14.870096 (+1163098us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:14.870223 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.694776 (+824553us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b54003cfd54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:15.694853 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:15.694953 (+   100us) raft_consensus.cc:1685] finished
0709 13:05:15.694973 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:15.695383 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8287,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":76992,"spinlock_wait_cycles":38036864,"thread_start_us":151247,"threads_started":1,"wal-append.queue_time_us":1083599}
I20260709 13:05:15.698845 15926 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.699784 15926 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.703634 15926 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:15.703657  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 403953us
I20260709 13:05:15.656742  8794 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:15.625806 16142 raft_consensus.cc:3055] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:15.690759 15880 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.715747 16016 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:15.724771  8748 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2835) took 3668 ms (client timeout 2047 ms). Trace:
W20260709 13:05:15.728371  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 79563.7us
I20260709 13:05:15.728196  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:15.737639  8821 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:15.738759  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:15.732651  8748 rpcz_store.cc:269] 0709 13:05:12.056516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.056693 (+   177us) service_pool.cc:224] Handling call
0709 13:05:15.664581 (+3607888us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54002fd5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:15.724640 (+ 60059us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4046357760}
I20260709 13:05:15.747714  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:15.757685  8754 raft_consensus.cc:2410] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:05:15.727469 15476 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.771250 15476 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.705916 16051 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:15.773180 16051 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.773922 15476 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:15.776199 16095 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.776898 16095 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:15.782074 16030 consensus_meta.cc:287] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.004s
I20260709 13:05:15.797062 16030 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.766074  8830 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 21: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:15.766690  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:15.816264  8748 raft_consensus.cc:2376] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17. Re-sending same reply.
I20260709 13:05:15.817822  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:15.767277 16059 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:15.819041 16059 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:15.819615 16059 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:15.820253 15839 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.823977 16019 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:15.769637 15880 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.740855 15862 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:15.825246 15862 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:15.826769 15862 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.827685  8645 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:15.828235 15862 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:15.828583 15926 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.828805 16030 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.829429 15926 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.782816  8672 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:15.830816 16082 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.783268  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:15.831594 16082 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.832825  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:15.833809  8675 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:15.784852  8676 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:05:15.834821 16088 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.765993  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:15.835613 16088 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.837430  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:15.765565  8722 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:15.792727  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:15.791920 16091 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 2.417s	user 0.010s	sys 0.020s
I20260709 13:05:15.765391  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:15.802246  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:15.811074  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 58097.4us
I20260709 13:05:15.718392 16142 raft_consensus.cc:740] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:15.811303 15476 consensus_queue.cc:1048] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:15.813036 16089 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:15.793644 16095 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.846256  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:15.861649  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:15.863548  9388 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:15.753749 16136 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.106s
I20260709 13:05:15.869803  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2976) took 2750 ms. Trace:
I20260709 13:05:15.874432  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:15.881582  8756 raft_consensus.cc:3060] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:15.866010  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:15.881939  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:15.872707  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:15.865617  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:15.881000  8672 rpcz_store.cc:276] 0709 13:05:13.119635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.686800 (+567165us) service_pool.cc:224] Handling call
0709 13:05:13.692980 (+  6180us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:15.830963 (+2137983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.869409 (+ 38446us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b54002a8ad4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:15.869707 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4480432896}
I20260709 13:05:15.838572 16088 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.870911  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:15.883780  8748 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:05:15.846290 16082 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:15.878131 16177 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:05:15.901199  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:15.906713  8680 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:15.903859 15848 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:15.875339 15839 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:05:15.912964  8647 connection.cc:256] RPC call timeout handler was delayed by -1.9142360179999969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
I20260709 13:05:15.916407  8821 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 23.
I20260709 13:05:15.868600 16105 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.911342  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:15.923861  8672 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 22: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:15.887890 16051 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.864094 16016 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.927187  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:15.907871 15880 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:15.930462  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:15.908030 16019 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:15.938827  8680 raft_consensus.cc:2393] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 22: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:15.940915  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:15.908192 16124 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.946436  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2864) took 2216 ms. Trace:
I20260709 13:05:15.877257 16094 consensus_queue.cc:1059] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:05:15.955157  8676 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:15.956666  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:15.956934 16094 raft_consensus.cc:3055] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:15.957546 16094 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:15.915798 16157 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:15.958751 15848 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:15.959039 16164 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:15.959714 16164 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.959987 15848 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.960193 16094 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:15.961971 16094 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:15.942857  8675 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:15.948822  8755 rpcz_store.cc:276] 0709 13:05:13.730267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.170536 (+1440269us) service_pool.cc:224] Handling call
0709 13:05:15.171153 (+   617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:15.172005 (+   852us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:15.195303 (+ 23298us) log.cc:844] Serialized 14 byte log entry
0709 13:05:15.703356 (+508053us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b180043afe0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e24f1060 00007f32e2503716 00007f32e2514287 00007f32e2514208
0709 13:05:15.703822 (+   466us) log.cc:844] Serialized 14 byte log entry
0709 13:05:15.704039 (+   217us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:15.750277 (+ 46238us) log.cc:844] Serialized 26 byte log entry
0709 13:05:15.750560 (+   283us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:15.750638 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.750750 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:15.946158 (+195408us) raft_consensus.cc:1685] finished
0709 13:05:15.946187 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:15.946365 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47382,"prepare.queue_time_us":99610,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":3885,"raft.queue_time_us":453615,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":14216,"spinlock_wait_cycles":17889920,"thread_start_us":88624,"threads_started":2,"wal-append.queue_time_us":290773}
I20260709 13:05:15.964455  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:15.958312  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:15.927098 16111 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.963603  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:15.970090 16051 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:05:15.974337  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:15.975561  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:15.977435  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 149908us
I20260709 13:05:15.979871  8794 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:15.981067 16105 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:15.981869 16105 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:15.946204 16142 consensus_queue.cc:260] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:15.982445  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 240987us
I20260709 13:05:15.983392  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:15.983083 16105 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.984822  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:15.985384  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:15.985945  8750 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "149c2129693643678d9da7765768609c"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:05:15.986596 15978 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:15.987233 16124 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:15.988227 15978 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:15.994413 15926 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:15.962545 16164 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:16.005132  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 230594us
I20260709 13:05:16.006575  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:16.007777 16044 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:16.010596 15982 consensus_meta.cc:287] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.011s	user 0.013s	sys 0.005s
I20260709 13:05:16.011498  8681 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:05:16.012059 15970 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.012683 15970 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:16.013018 16111 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:16.013725  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:16.014536  8681 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:05:16.016273 15970 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:16.017788  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2847) took 3803 ms. Trace:
I20260709 13:05:16.018273  8753 rpcz_store.cc:276] 0709 13:05:12.214451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.534563 (+320112us) service_pool.cc:224] Handling call
0709 13:05:12.571404 (+ 36841us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.798410 (+1227006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.017208 (+2218798us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400157cd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:16.017553 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2686012672}
W20260709 13:05:16.019637 16146 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.210s	user 0.008s	sys 0.015s
I20260709 13:05:16.019711  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:16.021050  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:16.021801  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:16.022010 15991 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:16.023402  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:16.012249 16144 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 1.192s	user 0.002s	sys 0.006s
I20260709 13:05:16.021837  8676 raft_consensus.cc:2368] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 21. Current term is 22.
I20260709 13:05:16.045531  8681 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 23.
I20260709 13:05:16.034132  8750 raft_consensus.cc:606] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:15.974278  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:16.020665  8753 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:16.057097  8750 raft_consensus.cc:614] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:15.990299 16124 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:16.064556  8680 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:16.065514  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:16.067430  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:16.064030  8755 raft_consensus.cc:1813] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Handling vote request from an unknown peer e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:16.073829  8675 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:16.078512  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:16.079629  8681 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:16.081544  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:16.085515  8755 raft_consensus.cc:2368] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 19.
I20260709 13:05:16.087088  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:16.087282  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:16.083115  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:16.094453  8828 raft_consensus.cc:2368] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 22. Current term is 23.
I20260709 13:05:15.968647 16016 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:16.125118 16016 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:16.125896 16016 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:05:16.095994  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 110838us
I20260709 13:05:16.128252  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:16.130012  8722 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:16.102205  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:16.102991  9352 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:16.103269  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 177754us
I20260709 13:05:16.107695  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:16.112571 16111 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:16.113296  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:16.115108  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2971) took 3302 ms. Trace:
I20260709 13:05:16.115640  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:16.117321  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:16.067132 15982 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:16.071748 16043 consensus_queue.cc:1059] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:05:16.121747  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3248) took 1207 ms. Trace:
I20260709 13:05:16.139607  8821 raft_consensus.cc:2410] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 21 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:05:16.132390  8676 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 23.
I20260709 13:05:16.170176  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:16.093654  8672 raft_consensus.cc:2376] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 23. Re-sending same reply.
I20260709 13:05:16.151544  8680 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:05:16.168808  8749 rpcz_store.cc:276] 0709 13:05:14.923987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.290361 (+366374us) service_pool.cc:224] Handling call
0709 13:05:15.299012 (+  8651us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:15.309106 (+ 10094us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:15.309148 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:15.387938 (+ 78790us) log.cc:844] Serialized 26 byte log entry
0709 13:05:15.905536 (+517598us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:15.905591 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.905695 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:16.121400 (+215705us) raft_consensus.cc:1685] finished
0709 13:05:16.121434 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:16.121650 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":345132,"threads_started":1,"wal-append.queue_time_us":708571}
W20260709 13:05:16.132318 16176 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.805s	user 0.001s	sys 0.007s
I20260709 13:05:16.182020  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:16.176367  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 130335us
I20260709 13:05:16.189883  8824 raft_consensus.cc:2410] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:05:16.191021  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:16.192387  8647 leader_election.cc:400] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 13 election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 13. Current term is 14.
I20260709 13:05:16.200479  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:16.201507  8680 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:16.244315  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:16.245118  8673 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:16.152140  8677 rpcz_store.cc:276] 0709 13:05:12.781842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.363994 (+582152us) service_pool.cc:224] Handling call
0709 13:05:13.364363 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.373796 (+  9433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.373832 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:13.390250 (+ 16418us) log.cc:844] Serialized 26 byte log entry
0709 13:05:13.631750 (+241500us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:13.631808 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.631890 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:16.018965 (+2387075us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:16.024493 (+  5528us) raft_consensus.cc:1685] finished
0709 13:05:16.024516 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:16.084333 (+ 59817us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5612,"raft.queue_time_us":45655,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":21641,"spinlock_wait_cycles":38517888,"thread_start_us":184322,"threads_started":1,"wal-append.queue_time_us":355061}
I20260709 13:05:16.178086 16164 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:16.253144  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:16.181182 16051 raft_consensus.cc:2764] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:05:16.255653  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:16.262989  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:16.270798  8821 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:16.272691  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:16.273041  8756 raft_consensus.cc:2468] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 24.
I20260709 13:05:16.239940 15978 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:16.177511 16136 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.087s
I20260709 13:05:16.292348  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:16.279645  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 485280us
I20260709 13:05:16.240675 16111 raft_consensus.cc:3055] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:05:16.283355  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 175819us
W20260709 13:05:16.297116  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 445377us
I20260709 13:05:16.289739  8824 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:16.196002 16023 consensus_queue.cc:1048] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.079s
I20260709 13:05:16.284085  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:16.302001  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:16.302838  8821 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 26.
W20260709 13:05:16.302809  9384 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.000s	sys 0.016s
I20260709 13:05:16.308246  8676 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 24: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:05:16.315968  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 186075us
I20260709 13:05:16.296591 16111 raft_consensus.cc:740] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:16.320067  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:16.320998  8821 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:16.308235  9384 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.000s	sys 0.016s
W20260709 13:05:16.318534  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 148944us
I20260709 13:05:16.344012  8681 raft_consensus.cc:2468] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 23.
W20260709 13:05:16.346745 16192 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:05:16.348680 16195 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:16.349216  8677 pending_rounds.cc:85] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 11
I20260709 13:05:16.355011  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:16.356346 16111 consensus_queue.cc:260] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:16.366835 16099 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:16.394037 16136 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:05:16.400647  8719 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:05:16.421382  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 84414.6us
I20260709 13:05:16.301880 16059 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:16.454531  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:16.475613  8677 pending_rounds.cc:107] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:05:16.507579  8722 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:16.501117  8677 raft_consensus.cc:2889] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:16.509812  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 109296us
I20260709 13:05:16.464349  8675 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:16.476411  8823 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:16.481967  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:16.479455  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:16.494930  8824 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 23.
I20260709 13:05:16.516980  8681 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:16.496289  8828 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:16.499518  8673 raft_consensus.cc:2468] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 23.
I20260709 13:05:16.473834 16195 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.519062  8674 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:16.519614  8673 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:16.521301  8647 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:16.499061 16030 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.028s	sys 0.048s
I20260709 13:05:16.524628  8681 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 22. Current term is 23. Ops: [20.17-22.18]
I20260709 13:05:16.552222 16008 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:05:16.548187 16030 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.029s	sys 0.048s
I20260709 13:05:16.580221  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:16.583964  8794 leader_election.cc:400] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 19.
I20260709 13:05:16.584414  8794 leader_election.cc:403] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:16.585142  8821 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
I20260709 13:05:16.586416  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:16.587966  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:16.588652  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50309.6us
I20260709 13:05:16.589136  8828 raft_consensus.cc:2410] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:05:16.546952 16195 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:16.597132  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 331742us
I20260709 13:05:16.540567 15839 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:05:16.520159 16124 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:05:16.576020 16008 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:16.616739  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 194314us
W20260709 13:05:16.618846  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 114395us
I20260709 13:05:16.607121 16051 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.640408 16008 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:16.620432 16111 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.596720 15982 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:16.659610  8647 leader_election.cc:400] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 22. Current term is 23.
W20260709 13:05:16.661342  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 153863us
I20260709 13:05:16.660368  8647 leader_election.cc:403] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:16.672056  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:16.679132  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 470026us
I20260709 13:05:16.670498 16022 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:16.655823 15982 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:16.639354 15862 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.083s
I20260709 13:05:16.671620 16136 consensus_queue.cc:1059] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:05:16.707998 15982 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.730312  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:16.744601  9352 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:16.730657 16082 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 22. Current term is 23.
I20260709 13:05:16.736297 15949 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:16.754475  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 92900.8us
I20260709 13:05:16.752483  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:16.740566 16152 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.950s	user 0.001s	sys 0.008s
W20260709 13:05:16.754874  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 96261.7us
W20260709 13:05:16.757534  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 152129us
I20260709 13:05:16.748896 16099 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:16.750236 16105 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:16.778055  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:16.778443  8749 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:16.779027  8719 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:16.752195 16059 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:16.781363  9352 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:16.754871 15949 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:16.787763  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:16.784349 16105 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:16.795207 16105 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:16.799199  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:16.756086 16205 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:16.801721  8753 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 22: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:16.802713  9352 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:16.805781  8719 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:05:16.811950  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 96253.6us
W20260709 13:05:16.801582 16201 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:05:16.798858 15949 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:16.839548 16209 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:05:16.839763 16044 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:05:16.841203 16213 log.cc:927] Time spent T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:05:16.841738  8751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 3253) took 1730 ms (client timeout 2083 ms). Trace:
W20260709 13:05:16.842275  8751 rpcz_store.cc:269] 0709 13:05:15.111683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.168095 (+ 56412us) service_pool.cc:224] Handling call
0709 13:05:16.841466 (+1673371us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b540038c2d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:16.841659 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":930037632}
W20260709 13:05:16.843060  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 83925us
I20260709 13:05:16.843833  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:16.843989  8647 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:16.844610  8751 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:16.845631  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:16.845728 16082 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:16.845837  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:16.847002 16088 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:05:16.847487 16088 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 26 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:16.847961 16088 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:05:16.848242 15880 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:16.848875 15880 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:16.815953 16221 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:16.850895  8722 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:16.851114 16221 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:16.805306 16196 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
W20260709 13:05:16.851722  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:16.820113 16207 log.cc:927] Time spent T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:05:16.820029 16197 raft_consensus.cc:2804] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:16.827185 16205 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:16.827998 16030 raft_consensus.cc:2804] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:16.837191  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2861) took 3107 ms. Trace:
I20260709 13:05:16.830785 16108 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:16.832190  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:16.832666 16219 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:16.832986  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:16.837148  8827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1527) took 1303 ms. Trace:
I20260709 13:05:16.853080  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1412) took 1252 ms. Trace:
W20260709 13:05:16.866458  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 47568.7us
I20260709 13:05:16.871236  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:16.818548 16194 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.202s	user 0.001s	sys 0.001s
I20260709 13:05:16.872241  8749 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
W20260709 13:05:16.883450  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 98457.2us
I20260709 13:05:16.878962  8752 rpcz_store.cc:276] 0709 13:05:13.729323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.729494 (+   171us) service_pool.cc:224] Handling call
0709 13:05:13.729913 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.777694 (+ 47781us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.777749 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:13.813679 (+ 35930us) log.cc:844] Serialized 26 byte log entry
0709 13:05:14.007622 (+193943us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:14.007688 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.007807 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:16.582116 (+2574309us) spinlock_profiling.cc:243] Waited 50.6 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:16.836726 (+254610us) raft_consensus.cc:1685] finished
0709 13:05:16.836748 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:16.836986 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25913,"raft.queue_time_us":47795,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":19174,"spinlock_wait_cycles":116471040,"thread_start_us":193533,"threads_started":1,"wal-append.queue_time_us":976231}
I20260709 13:05:16.886144  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:16.883455  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:16.884981  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:16.873673  8722 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:16.850957  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:16.892942 16223 raft_consensus.cc:2764] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:05:16.893105  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:16.895812  8719 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:16.851678 16221 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.851183 16082 consensus_queue.cc:1059] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 13:05:16.900101  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:16.902263  8822 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:05:16.882282  8747 rpcz_store.cc:276] 0709 13:05:15.600988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.601144 (+   156us) service_pool.cc:224] Handling call
0709 13:05:15.601541 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:15.753500 (+151959us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:15.753551 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:15.791050 (+ 37499us) log.cc:844] Serialized 26 byte log entry
0709 13:05:16.265437 (+474387us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:16.265475 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.265546 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:16.852800 (+587254us) raft_consensus.cc:1685] finished
0709 13:05:16.852826 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:16.853005 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142162,"raft.queue_time_us":151947,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":28044,"thread_start_us":473809,"threads_started":1,"wal-append.queue_time_us":689541}
I20260709 13:05:16.902920  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:16.903688  8755 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.906842  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:16.908013  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:16.882216  8827 rpcz_store.cc:276] 0709 13:05:15.533877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.542295 (+  8418us) service_pool.cc:224] Handling call
0709 13:05:15.565881 (+ 23586us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:15.590463 (+ 24582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:15.590514 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:16.508419 (+917905us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b2c000f2de8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e0df2a11 00007f32e0e02372 00007f32e2534a01 00007f32e24f943a 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2
0709 13:05:16.603277 (+ 94858us) log.cc:844] Serialized 26 byte log entry
0709 13:05:16.749453 (+146176us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:16.749496 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.749994 (+   498us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:16.836664 (+ 86670us) raft_consensus.cc:1685] finished
0709 13:05:16.836691 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:16.836956 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33067,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":4416768,"thread_start_us":108560,"threads_started":1,"wal-append.queue_time_us":198690}
W20260709 13:05:16.901698  8830 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2781) took 3568 ms (client timeout 2180 ms). Trace:
I20260709 13:05:16.908702  8750 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
W20260709 13:05:16.909576  8830 rpcz_store.cc:269] 0709 13:05:13.326864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.493091 (+166227us) service_pool.cc:224] Handling call
0709 13:05:16.894738 (+3401647us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b540016b054. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:16.894976 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4819537280}
I20260709 13:05:16.912654  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:16.895970 16219 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.913313  8747 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:05:16.839654 16228 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 13:05:16.918519  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:16.919014  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:16.919848  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 27 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:16.921186  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:16.911192 16108 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:16.881968 15862 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:05:16.924060  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:16.924224  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:16.922770  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 138606us
I20260709 13:05:16.927742 16220 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:16.928400 16220 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:16.928092 16205 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:16.898433 15982 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:16.915897 15839 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:16.917568 16219 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.924885 16108 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:16.950309  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 43587.2us
I20260709 13:05:16.952368  8647 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:16.946399  8793 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:16.954762 16232 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.948576 16108 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:16.929519 16220 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:16.955716 16022 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:16.955510 16232 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:16.969774  8645 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.591s (SENT) [suppressed 22 similar messages]
I20260709 13:05:16.982398  8676 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:16.983114  8676 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:16.948758 16197 raft_consensus.cc:697] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:16.985433  8750 raft_consensus.cc:1240] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:16.985638  8752 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:05:16.988569 16241 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:16.989249 16241 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.989668 16241 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:16.993626 15926 raft_consensus.cc:3055] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:16.994181 15926 raft_consensus.cc:740] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:16.994143 16240 raft_consensus.cc:2764] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
I20260709 13:05:16.996574 15863 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:16.996426  8676 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:17.000001 16022 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:17.001535 16022 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.001989 16022 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:16.997035 15863 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:17.009994  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 136435us
I20260709 13:05:17.010959 16030 raft_consensus.cc:697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:17.011641 15862 consensus_queue.cc:1059] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:05:17.011222 16243 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:05:17.012107 16030 consensus_queue.cc:237] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:17.013129 15862 raft_consensus.cc:3055] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:17.014258  8722 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:17.015573  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.974409 16235 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:17.053102 16235 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.053673 16235 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:17.055119  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:17.055830  9386 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:05:17.056332  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:17.025463  9384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34746 (request call id 919) took 1449 ms. Trace:
W20260709 13:05:17.020116 16094 consensus_meta.cc:287] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 1.058s	user 0.013s	sys 0.006s
I20260709 13:05:17.007611 15863 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:17.033694  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:17.035045  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:17.078794  9387 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:17.037792 15982 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:17.041924 16051 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 19.
I20260709 13:05:17.043246  8751 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
W20260709 13:05:17.046141 16174 log.cc:927] Time spent T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 1.712s	user 0.010s	sys 0.011s
I20260709 13:05:17.044876  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 2994) took 1169 ms. Trace:
I20260709 13:05:17.088655  8677 rpcz_store.cc:276] 0709 13:05:15.875790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.315750 (+439960us) service_pool.cc:224] Handling call
0709 13:05:16.331170 (+ 15420us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:16.756658 (+425488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:16.756703 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:16.757371 (+   668us) log.cc:844] Serialized 52 byte log entry
0709 13:05:16.914378 (+157007us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:16.914421 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.914486 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:17.044594 (+130108us) raft_consensus.cc:1685] finished
0709 13:05:17.044622 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.044800 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10983,"raft.queue_time_us":42578,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":7136,"thread_start_us":267129,"threads_started":2,"wal-append.queue_time_us":189226}
I20260709 13:05:17.090999  9380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 1265) took 2028 ms. Trace:
I20260709 13:05:17.091627  9380 rpcz_store.cc:276] 0709 13:05:15.062284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.062485 (+   201us) service_pool.cc:224] Handling call
0709 13:05:15.111253 (+ 48768us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:15.224699 (+113446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:15.224741 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:15.225456 (+   715us) log.cc:844] Serialized 26 byte log entry
0709 13:05:15.266005 (+ 40549us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:15.266074 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.266184 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:17.045363 (+1779179us) spinlock_profiling.cc:243] Waited 9.19 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:17.090555 (+ 45192us) raft_consensus.cc:1685] finished
0709 13:05:17.090582 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.090910 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21138304,"thread_start_us":25735,"threads_started":1,"wal-append.queue_time_us":72899}
I20260709 13:05:17.050643 15862 raft_consensus.cc:740] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:17.023882 16232 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:17.075621  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 61418.5us
I20260709 13:05:17.054764 16008 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:17.097452 16008 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:17.080107  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:17.099435  8830 raft_consensus.cc:2393] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 20: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:17.100244  8676 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:17.094201 15862 consensus_queue.cc:260] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:17.079929 16197 consensus_queue.cc:237] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:17.085269  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:17.103463 15880 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:17.061767  9384 rpcz_store.cc:276] 0709 13:05:15.576434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.576884 (+   450us) service_pool.cc:224] Handling call
0709 13:05:15.590086 (+ 13202us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:15.636245 (+ 46159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:15.636300 (+    55us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:15.705065 (+ 68765us) log.cc:844] Serialized 78 byte log entry
0709 13:05:16.328296 (+623231us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:16.710088 (+381792us) log.cc:844] Serialized 14 byte log entry
0709 13:05:16.712181 (+  2093us) log.cc:844] Serialized 14 byte log entry
0709 13:05:16.716339 (+  4158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.716476 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:17.025160 (+308684us) raft_consensus.cc:1685] finished
0709 13:05:17.025179 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.025372 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5281,"prepare.queue_time_us":280686,"prepare.run_cpu_time_us":154,"prepare.run_wall_time_us":3986,"raft.queue_time_us":22779,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":15474,"spinlock_wait_cycles":11135744,"thread_start_us":768924,"threads_started":2,"wal-append.queue_time_us":755000}
I20260709 13:05:17.104110 15926 consensus_queue.cc:260] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:17.105645 15926 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:17.107474  8796 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:17.108980  8796 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:17.109037  9380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:17.109839  9380 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:17.110280 15880 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:17.111014 15880 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:17.111361 15880 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.112331 16051 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:17.113116  9388 raft_consensus.cc:1813] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Handling vote request from an unknown peer e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:17.112939  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:17.115885  9388 raft_consensus.cc:2368] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 19.
I20260709 13:05:17.129314  8755 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:17.111676 15880 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:17.173843  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:17.174988  9385 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:17.115816 15862 raft_consensus.cc:3060] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:17.191660  9384 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:05:17.195322  8796 leader_election.cc:400] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 19.
I20260709 13:05:17.187142  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:17.192602  8722 connection.cc:256] RPC call timeout handler was delayed by -2.05301045900012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24680000000000002s for a response
I20260709 13:05:17.209089  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:17.208880  9382 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 23.
I20260709 13:05:17.224730  8755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:05:17.225704  8755 raft_consensus.cc:606] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:17.228044  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 332287us
W20260709 13:05:17.240149  8722 connection.cc:256] RPC call timeout handler was delayed by -1.7718948480001018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
W20260709 13:05:17.240898  8722 connection.cc:256] RPC call timeout handler was delayed by -1.6531498470000088s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2041s for a response
W20260709 13:05:17.241297  8722 connection.cc:256] RPC call timeout handler was delayed by -1.568375858999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
W20260709 13:05:17.241708  8722 connection.cc:256] RPC call timeout handler was delayed by -1.5513948480000863s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2163s for a response
W20260709 13:05:17.242079  8722 connection.cc:256] RPC call timeout handler was delayed by -1.3641186590000416s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26789999999999997s for a response
W20260709 13:05:17.213694 16195 consensus_meta.cc:287] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.598s	user 0.009s	sys 0.005s
I20260709 13:05:17.221676 16008 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.260150  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:17.263787  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 326093us
W20260709 13:05:17.272450  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 320292us
I20260709 13:05:17.221410 15949 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:17.284403  8647 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:17.284833  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:17.296290  9386 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:17.237449 16221 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:17.282785 16111 consensus_meta.cc:287] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.622s	user 0.015s	sys 0.000s
I20260709 13:05:17.256003 16088 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:17.245069 16051 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:05:17.322247  8821 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:17.263396 16243 consensus_queue.cc:1059] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:05:17.318720  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:17.311029  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.315038  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:17.327777  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 43653.6us
I20260709 13:05:17.265914 16008 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:17.258527 16195 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:17.320115 16105 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.004s
I20260709 13:05:17.326547  9384 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:17.348997  8749 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:17.271525 16012 consensus_queue.cc:1059] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:05:17.319224 16022 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:17.330027 16219 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:17.383273  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:17.424654  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 65236.5us
I20260709 13:05:17.357690 16105 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:17.384338 15476 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:17.357758 16030 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 13:05:17.396932 15978 raft_consensus.cc:3055] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:17.557860  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:17.400411  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.406896  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:17.408034 16221 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:17.343662 16249 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:17.409880 15949 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:17.324764 16241 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:17.567481  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.412142 16195 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:17.384785 16257 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:17.417069  9387 raft_consensus.cc:2468] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:17.359124  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:17.578024  9385 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 24: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:17.427476  9383 raft_consensus.cc:1240] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:17.450202 16082 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:17.583853  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:17.432066  8676 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:05:17.401919 16088 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 27 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:17.485252  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 189608us
I20260709 13:05:17.541993  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:17.584527  8676 pending_rounds.cc:85] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 13
W20260709 13:05:17.543581  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 79925.4us
I20260709 13:05:17.462287 16108 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:17.541518  8753 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:17.425526  9353 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:17.521466  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 225827us
W20260709 13:05:17.585783  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.556993  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:17.587733  8824 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:17.399850  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:17.573369  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:17.588747  8821 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:17.588814  8828 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:17.574352  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:17.583179  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:17.583528  8825 raft_consensus.cc:2410] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 23 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:05:17.584888  8676 pending_rounds.cc:107] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:05:17.590202  8676 raft_consensus.cc:2889] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:17.584266 16082 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:17.614428 16082 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:17.615828 16219 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:17.560797 15978 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:17.616753 16219 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:17.617148 16219 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.617523 16219 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:17.617409 15978 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:17.618726 15978 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:17.625226 16257 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:17.626538 16257 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:17.599316  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:17.602267  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:17.604131  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:17.632409  8673 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:17.604465  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:17.633580  8680 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:17.601480  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:17.607149 16248 log.cc:927] Time spent T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:05:17.608345 16142 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:17.608618  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 183145us
I20260709 13:05:17.645102  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:17.608853  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 259155us
I20260709 13:05:17.610317  9387 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf"
dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
mode: GRACEFUL
new_leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
 from {username='slave'} at 127.0.0.1:33196
I20260709 13:05:17.610580 16108 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:17.611687 16195 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:17.615978 16082 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:17.605789  8675 raft_consensus.cc:1240] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
W20260709 13:05:17.635392  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:17.635555  9380 consensus_meta.cc:287] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.525s	user 0.013s	sys 0.002s
I20260709 13:05:17.657764  9380 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 23.
I20260709 13:05:17.569649 16012 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:17.643050  8676 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:05:17.643764  8674 raft_consensus.cc:1240] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 27 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 26. Current term is 27. Ops: []
I20260709 13:05:17.644138  8677 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:17.646180  9387 raft_consensus.cc:606] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEADER]: Received request to transfer leadership to TS e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:17.647827  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:17.649470  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:05:17.631448 16016 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.294s	user 0.021s	sys 0.043s
I20260709 13:05:17.662989  9385 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:17.585477 16249 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:17.599104 16197 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:17.653973  8828 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:17.678468  8672 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.017s	sys 0.041s
W20260709 13:05:17.678962  8672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.017s	sys 0.041s
I20260709 13:05:17.681658  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:17.689064  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 44058.4us
I20260709 13:05:17.696931  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:17.698307  8830 raft_consensus.cc:2393] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:17.699028  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:17.699505  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:17.707777  8722 leader_election.cc:400] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 22. Current term is 23.
I20260709 13:05:17.651777 16057 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:17.712149  9353 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:17.673877 16016 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.351s	user 0.021s	sys 0.043s
I20260709 13:05:17.716439  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:17.705617 15863 consensus_meta.cc:287] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.606s	user 0.007s	sys 0.006s
I20260709 13:05:17.674132 16012 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:17.685591 16197 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:17.729277  8645 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:17.745744  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:17.746519  8827 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
W20260709 13:05:17.754312  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 167829us
I20260709 13:05:17.755322  8830 raft_consensus.cc:1240] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:05:17.755894  8823 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:17.753424 16235 consensus_meta.cc:287] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.005s
W20260709 13:05:17.767426  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 119708us
I20260709 13:05:17.770853  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:17.771922  8827 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:17.746243 16022 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:17.730654 16258 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:17.755722 16057 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:17.779417  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 90847.8us
I20260709 13:05:17.744104  8719 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:17.778421  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 66393.6us
I20260709 13:05:17.716603  8680 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:05:17.784420  9353 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:17.760733 16105 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:17.738124 15863 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:17.789631  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:17.732517  8828 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:17.801309  8828 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:17.797063 16195 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:17.812773  9353 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:17.819205 16195 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:17.819715 16195 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.820077 16195 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:17.823678  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2871) took 1476 ms. Trace:
I20260709 13:05:17.824252  8753 rpcz_store.cc:276] 0709 13:05:16.346867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.918773 (+571906us) service_pool.cc:224] Handling call
0709 13:05:17.823332 (+904559us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54002ad354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:17.823576 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1411186560}
I20260709 13:05:17.825829  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:17.828107  9353 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:17.835057  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 91090.7us
I20260709 13:05:17.818835 16019 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:17.827205  8753 raft_consensus.cc:2376] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22. Re-sending same reply.
I20260709 13:05:17.775648 16221 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:17.843648  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:17.852873  8753 raft_consensus.cc:2393] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 24: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:17.854387  8722 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:17.854700  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:17.856016  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:17.856153  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:17.856676  8753 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:17.857000 16211 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:17.845021 16019 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:17.859241 16019 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:17.860746 16019 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:17.793452 16197 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:17.862731 16221 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:17.863878 16197 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:17.865955 16221 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:17.787753 16023 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:17.869084 16204 log.cc:927] Time spent T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 1.228s	user 0.002s	sys 0.001s
W20260709 13:05:17.875073  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 93437.7us
I20260709 13:05:17.876299  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:17.878331 16044 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:17.879433 16052 consensus_queue.cc:1059] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:17.880628 16105 consensus_queue.cc:1048] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:17.882296 16105 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:05:17.892158 16253 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.330s	user 0.000s	sys 0.000s
I20260709 13:05:17.894289 16228 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:17.894919 16228 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.895328 16228 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:17.897699  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2754) took 2569 ms. Trace:
I20260709 13:05:17.898140  8822 rpcz_store.cc:276] 0709 13:05:15.328360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.341604 (+ 13244us) service_pool.cc:224] Handling call
0709 13:05:16.895330 (+1553726us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:16.909012 (+ 13682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.897397 (+988385us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b540016b050. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:17.897602 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3571855104}
I20260709 13:05:17.792443 16249 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:17.739467 16008 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:17.907641  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 64384.5us
I20260709 13:05:17.786557 16250 raft_consensus.cc:2764] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:05:17.788523 16243 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:05:17.902745  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 126283us
I20260709 13:05:17.776360 16235 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:17.922664  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:17.938550  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 82504.2us
I20260709 13:05:17.912424 16243 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:17.902660 15949 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:17.962143  9353 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:17.967347  8753 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:17.968501  8754 raft_consensus.cc:2368] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 23. Current term is 24.
I20260709 13:05:17.969766  8566 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 364ad2097a2341cdb96752b9a370f9a9 with cas_config_opid_index 4: aborting the task: latest config opid_index 11; task opid_index 4
I20260709 13:05:17.942313 16205 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:17.918753 16252 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:17.939152 16136 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:17.971133 15863 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:17.939582 15926 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.833s	user 0.017s	sys 0.000s
I20260709 13:05:17.976950  8722 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:17.968717  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:17.979161  8753 raft_consensus.cc:2376] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19. Re-sending same reply.
W20260709 13:05:17.986965  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.937825 16008 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:17.795805 16012 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:17.992045  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:17.967444 16243 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:17.998843  8750 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
W20260709 13:05:17.999315  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.999763  8645 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:17.971539 16136 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:18.026839  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:18.019218 16044 raft_consensus.cc:697] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:18.001938 15863 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:18.043912  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:18.004168 16136 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:18.051345  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:18.045226  8827 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:18.005329 16273 raft_consensus.cc:2804] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:17.980327 16252 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.057763  8752 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:18.061458  9384 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:18.046707 16052 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:18.035629 16164 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.029320 16012 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:18.002418 16008 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:18.050041 15863 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:18.074059  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:05:18.079278  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.063789 15880 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.878s	user 0.009s	sys 0.008s
W20260709 13:05:18.083704  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 203964us
I20260709 13:05:18.070102 16052 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:18.089649  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:18.087246 16246 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:05:18.085621 15863 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:18.107779  8647 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:18.110419  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:05:18.119030  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 88309.7us
I20260709 13:05:18.120415  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:18.126972  8756 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 14.
W20260709 13:05:18.127637  8756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2868) took 1789 ms (client timeout 1847 ms). Trace:
I20260709 13:05:18.127671  8645 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:05:18.128085  8756 rpcz_store.cc:269] 0709 13:05:16.338311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.120066 (+1781755us) service_pool.cc:224] Handling call
0709 13:05:18.127561 (+  7495us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:18.078112 16164 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:18.101306  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:18.129760  8793 leader_election.cc:400] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 21. Current term is 22.
I20260709 13:05:18.107110 16044 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:18.130772 16273 raft_consensus.cc:697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:18.087564 15880 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.134871 16273 consensus_queue.cc:237] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.132897  9383 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 21: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:18.134889 16052 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:18.139253 16052 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:18.139739  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:18.139936  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:18.132332  8793 leader_election.cc:403] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:05:18.140692 15880 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:18.147240 16088 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:05:18.147487  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 54589.7us
I20260709 13:05:18.148159 16088 raft_consensus.cc:697] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:18.148561  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:18.149653 16276 raft_consensus.cc:476] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Not starting pre-election -- already a leader
I20260709 13:05:18.151454 16276 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:18.154402 16287 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.154563 16286 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:18.141964  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:18.155180 16287 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.156184 15880 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 21. Current term is 22.
I20260709 13:05:18.156399 15839 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:18.156654  8755 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 20: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:18.158396  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:18.160249 16278 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:05:18.131340 16279 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:05:18.163884 16235 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:18.149464 16088 consensus_queue.cc:237] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:18.114926 16082 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:18.170907  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 43702.6us
I20260709 13:05:18.168838  8824 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:05:18.167852  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:18.172492  8645 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:18.159492  8755 raft_consensus.cc:2368] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19.
I20260709 13:05:18.174747 16288 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:18.175369 16288 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:18.175760  8794 leader_election.cc:400] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19.
I20260709 13:05:18.175848 16288 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:18.176362  8794 leader_election.cc:403] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:05:18.177546 15880 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:18.180248  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:18.186344 16230 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.187403 16095 raft_consensus.cc:3055] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:18.187217 16230 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.188990  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:18.191155 16291 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:18.191669 16291 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.192018 16291 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:18.192662 15476 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.624s	user 0.009s	sys 0.002s
I20260709 13:05:18.193161 15476 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:18.193512  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.187903 16095 raft_consensus.cc:740] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:05:18.197088  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.198405 15863 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.200943 15863 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:18.195860 16095 consensus_queue.cc:260] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.206171 16095 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:18.139979 16164 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:18.207922  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:18.190397  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:18.209057 16111 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:05:18.212411  8755 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:18.216168  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:18.216435  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:18.217226  9383 raft_consensus.cc:2393] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:18.218477  9380 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:18.219669 16287 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:18.218294  9385 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:18.221662  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:18.222076 16216 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.419s	user 0.005s	sys 0.000s
W20260709 13:05:18.186384 16283 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:05:18.230860 16057 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.231541 16057 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.233175  9380 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:05:18.234508  8796 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:18.198378 16219 consensus_meta.cc:287] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.001s
I20260709 13:05:18.254974 16219 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.257609 16219 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:18.259689 16008 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:18.261433 16252 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:18.264588 16195 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:18.267511  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.269286 16276 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:18.270143 16243 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:18.276229  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 103862us
I20260709 13:05:18.245147 16258 raft_consensus.cc:2804] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:18.245425 16198 log.cc:927] Time spent T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 1.690s	user 0.002s	sys 0.002s
I20260709 13:05:18.246047 16287 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:05:18.247108  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:18.247859  9381 raft_consensus.cc:2393] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:18.249954  9353 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:18.250595  8748 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2867) took 1927 ms (client timeout 1627 ms). Trace:
I20260709 13:05:18.235116  9352 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:18.261634  8751 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:18.244210 16057 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:18.280741  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.281334  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:05:18.284602  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 224185us
I20260709 13:05:18.268647 16136 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:18.298214  8748 rpcz_store.cc:269] 0709 13:05:16.323220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.250380 (+1927160us) service_pool.cc:211] Skipping call since client already timed out
0709 13:05:18.250506 (+   126us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:05:18.298667  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 48817.1us
W20260709 13:05:18.298496  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 63449.1us
I20260709 13:05:18.306447 16197 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:18.309990  9384 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:18.310801  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 143034us
I20260709 13:05:18.311717 16258 raft_consensus.cc:697] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:18.317816  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:18.331785  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.295800 16276 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:18.306545 16072 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:18.342865  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:05:18.330354 16274 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:05:18.366080  8680 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:05:18.372016  8722 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:18.372371  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 54694.6us
I20260709 13:05:18.307600 15970 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.373834  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:18.325418 15880 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19.
I20260709 13:05:18.400231  8722 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:18.380642  9387 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:18.374094  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:18.402227  8748 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 20
W20260709 13:05:18.407477  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.361972 16197 raft_consensus.cc:697] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:18.369635 16019 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:18.430485  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:18.431131  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1528) took 2793 ms. Trace:
I20260709 13:05:18.432135  8673 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:18.377238 16287 consensus_queue.cc:1243] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 23.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 13:05:18.382164 16219 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:18.425179  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:18.415570 15970 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:18.436179  8675 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:05:18.380306 16276 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:18.436654  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:18.437008  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:18.432052  8826 rpcz_store.cc:276] 0709 13:05:15.638198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.638417 (+   219us) service_pool.cc:224] Handling call
0709 13:05:15.647460 (+  9043us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:15.647768 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:15.647795 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:15.719448 (+ 71653us) log.cc:844] Serialized 26 byte log entry
0709 13:05:16.280250 (+560802us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:16.280329 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.280445 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:18.244736 (+1964291us) spinlock_profiling.cc:243] Waited 7.97 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:18.422335 (+177599us) raft_consensus.cc:1685] finished
0709 13:05:18.422360 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.431052 (+  8692us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18335616,"thread_start_us":453263,"threads_started":1,"wal-append.queue_time_us":671764}
I20260709 13:05:18.299715 16291 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:18.434563  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:18.443883  8677 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 13:05:18.415932 16195 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:18.416855 16290 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.452963  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:18.480906  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:18.488165  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 339646us
I20260709 13:05:18.491955  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:18.493240  8680 raft_consensus.cc:2368] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20.
I20260709 13:05:18.499841  9387 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 24.
W20260709 13:05:18.501132  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.535970  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:05:18.506541 16282 log.cc:927] Time spent T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.375s	user 0.001s	sys 0.000s
I20260709 13:05:18.507762 16243 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:18.507118 16258 consensus_queue.cc:237] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.451682 16197 consensus_queue.cc:237] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.508723 16276 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:18.509709  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:18.545439  9389 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:18.444912 16219 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.516559  8647 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:18.431401 15476 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 13:05:18.524286  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:18.431183 16164 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:18.442880 15863 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:18.560756  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:18.528977  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.531669  8750 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:05:18.564889  8750 raft_consensus.cc:606] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:18.565135  8821 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:18.565464  8750 raft_consensus.cc:614] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:18.534843  8753 raft_consensus.cc:1240] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19. Ops: []
W20260709 13:05:18.542455  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.544256  8722 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:18.503168 16290 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.536002  8748 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:05:18.398864 16235 consensus_queue.cc:1059] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.103s
I20260709 13:05:18.530246 15949 raft_consensus.cc:3055] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:05:18.555632  8830 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.001s	sys 0.032s
I20260709 13:05:18.555635  8751 raft_consensus.cc:3060] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:18.557729  8681 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:18.527733 16291 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:18.447412 16286 consensus_queue.cc:1059] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.090s
W20260709 13:05:18.569178  8830 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.001s	sys 0.032s
I20260709 13:05:18.544862 16243 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.576969  8673 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:05:18.574512  8753 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:18.555773 16219 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:18.443038 16257 consensus_queue.cc:1059] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.112s
I20260709 13:05:18.585987  8677 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:18.576201  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:18.581686  8753 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:05:18.594712  8825 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:05:18.568856 15949 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:18.577663 16242 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:18.614216  8673 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:18.634358  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 118019us
W20260709 13:05:18.646898  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 368612us
W20260709 13:05:18.869050  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.592482 16235 consensus_queue.cc:1243] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 13:05:18.707002 16270 log.cc:927] Time spent T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:05:18.612251 16164 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:18.587599 16243 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.604592 16296 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:18.898964  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:18.580245 16008 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:18.863924  8821 raft_consensus.cc:3055] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:18.910416  8821 raft_consensus.cc:740] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:18.912194  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:18.913125  8756 raft_consensus.cc:2368] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18.
I20260709 13:05:18.883034 16164 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.635869  8825 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:18.866514  9382 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:18.866715  8751 raft_consensus.cc:2468] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:05:18.870954  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:18.873955  8680 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:05:18.874716  8750 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:18.883040  8826 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:05:18.886425  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 276588us
I20260709 13:05:18.929399  8673 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 21: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:18.887449  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2888) took 2530 ms. Trace:
I20260709 13:05:18.931084  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:18.932166  8827 raft_consensus.cc:2410] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:05:18.930670  8747 rpcz_store.cc:276] 0709 13:05:16.356629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.934630 (+578001us) service_pool.cc:224] Handling call
0709 13:05:16.939475 (+  4845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:16.947306 (+  7831us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:16.967183 (+ 19877us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.451382 (+1484199us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x7b18002c35c0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:05:18.886905 (+435523us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:18.886965 (+    60us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:18.887015 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.887274 (+   259us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.887389 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18075,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":30931,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":13722,"spinlock_wait_cycles":16454016,"thread_start_us":302274,"threads_started":1}
W20260709 13:05:18.908686 16095 consensus_meta.cc:287] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.650s	user 0.021s	sys 0.002s
W20260709 13:05:18.888748  8794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.939436  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:18.883939  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:18.907135  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:18.865658  9389 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
W20260709 13:05:18.923669  9384 consensus_meta.cc:287] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.606s	user 0.014s	sys 0.008s
I20260709 13:05:18.925889  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:18.888295  8647 leader_election.cc:400] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Vote denied by peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) with higher term. Message: Invalid argument: T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 22. Current term is 23.
I20260709 13:05:18.946434  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:18.896224 16298 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:05:18.928670 16295 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.277s	user 0.010s	sys 0.005s
I20260709 13:05:18.939558  8677 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:05:18.945535  8675 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 18: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:05:18.911350 16288 consensus_meta.cc:287] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.734s	user 0.019s	sys 0.007s
I20260709 13:05:18.947235  8826 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:18.946482  8828 raft_consensus.cc:2393] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 23: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:18.963903  8747 raft_consensus.cc:1275] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 13:05:18.964437  8747 pending_rounds.cc:85] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 13
I20260709 13:05:18.964676  8747 pending_rounds.cc:107] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 13:05:18.964954  8747 raft_consensus.cc:2889] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:18.953836  9384 raft_consensus.cc:2468] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 17.
I20260709 13:05:18.917872 15880 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:18.988279  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 100129us
I20260709 13:05:18.955750  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:18.956846  8751 raft_consensus.cc:3055] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:18.991286  8751 raft_consensus.cc:740] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:18.893147 16223 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:18.941721  8821 consensus_queue.cc:260] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:18.992133  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:18.992879  8825 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:18.993157  8677 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:18.967841  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:18.994269  8679 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:18.996676  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:18.997346  8828 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:18.991305  8680 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:18.927863 16008 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:18.928025 16136 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:18.998320  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:18.999903  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.000135  8827 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:05:19.002292  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:19.003981  8827 pending_rounds.cc:85] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Aborting all ops after (but not including) 10
I20260709 13:05:19.004298  8827 pending_rounds.cc:107] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:05:19.004521  8827 raft_consensus.cc:2889] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:19.003865  8825 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:19.009778  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:19.010708  8826 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:19.009670  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:19.002151  8751 consensus_queue.cc:260] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:19.023404  8672 raft_consensus.cc:2368] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20.
I20260709 13:05:19.024926  8647 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:19.029430  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:18.914340 16296 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:19.039602  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:19.053797 16230 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:19.056556  8751 raft_consensus.cc:3060] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:19.038481 16300 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:05:19.078291  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:18.987705 16288 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:19.000578 15880 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:19.143591  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.176596  8675 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:19.144291  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.044873 16284 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:19.044287 15926 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:19.007409 16298 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:19.167672  8822 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:19.169162  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:18.955001 15949 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:19.208534  8677 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:19.296494  8751 raft_consensus.cc:1275] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 13:05:19.256750  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.297294  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.261940  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.297271  8751 pending_rounds.cc:85] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 15
I20260709 13:05:19.055786 16277 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.114s
I20260709 13:05:19.206956  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:19.225663  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:19.299793  8828 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:19.299676  8825 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:19.301554  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:19.302205  8672 raft_consensus.cc:2393] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 15: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:19.302371  8828 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:19.157876 15970 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:19.274645  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1428) took 1052 ms. Trace:
W20260709 13:05:19.260118  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.209309 16308 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:05:19.288026  8680 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:19.298382  8677 raft_consensus.cc:2368] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19.
I20260709 13:05:19.314100  8828 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:19.301961  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:19.299031  8751 pending_rounds.cc:107] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 13:05:19.315366  8751 raft_consensus.cc:2889] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:19.229336  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:05:19.311785  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.315447  8825 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 23.
W20260709 13:05:19.316375  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.316264  8675 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:19.317392  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.318341  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.319509  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.288961 16228 consensus_meta.cc:287] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 1.384s	user 0.022s	sys 0.004s
I20260709 13:05:19.327728  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:19.328135  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:19.332365  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1430) took 1013 ms. Trace:
I20260709 13:05:19.333026  8751 rpcz_store.cc:276] 0709 13:05:18.319093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.928357 (+609264us) service_pool.cc:224] Handling call
0709 13:05:18.928763 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:19.331913 (+403150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.332259 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:19.335017  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:19.337687  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.338745  8677 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:19.343171  8828 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:19.344779  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:19.344696  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:19.352236  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:19.353066  8825 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:19.169281 16008 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:19.355548  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:19.356119  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:19.356978  8828 raft_consensus.cc:2393] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 21: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:19.357041  8827 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:19.359684  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:19.360333  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:19.360394  8826 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:19.341737 16243 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:19.376168  8825 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
W20260709 13:05:19.377554 16301 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
W20260709 13:05:19.378455 16309 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:05:19.368878 16230 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.379163 16288 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:19.354228 15949 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:19.398253 16241 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:19.399341 16311 log.cc:927] Time spent T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:05:19.400490 16296 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:19.379271 16303 log.cc:927] Time spent T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 13:05:19.379351  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:19.402181  9385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 1279) took 1172 ms. Trace:
I20260709 13:05:19.402591  9385 rpcz_store.cc:276] 0709 13:05:18.229329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.235365 (+  6036us) service_pool.cc:224] Handling call
0709 13:05:19.379218 (+1143853us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b5800004598. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4eee 0000000000375939 00007f32e74481e0 00007f32e75110c4 00007f32e7511131 00007f32e74bf0a0 00007f32e74be984 00007f32e0a396d4 00007f32e0a39662
0709 13:05:19.402080 (+ 22862us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41762560}
I20260709 13:05:19.383322  8751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116"
dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
mode: GRACEFUL
new_leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
 from {username='slave'} at 127.0.0.1:45676
I20260709 13:05:19.403998  8751 raft_consensus.cc:606] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Received request to transfer leadership to TS 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:19.404413  8751 raft_consensus.cc:614] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:05:19.404914 16219 consensus_meta.cc:287] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.800s	user 0.008s	sys 0.004s
I20260709 13:05:19.378515  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.405369 16219 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:19.410424 16298 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:19.325875 16228 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:19.393810  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:19.442315  8825 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 18: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:05:19.444722  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.391527  8675 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
W20260709 13:05:19.423753  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.439734  8826 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:19.416210  8830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2788) took 1723 ms. Trace:
I20260709 13:05:19.448729  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:19.447427  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:19.451733  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:19.451754  8825 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:19.450107  8830 rpcz_store.cc:276] 0709 13:05:17.693023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.769216 (+ 76193us) service_pool.cc:224] Handling call
0709 13:05:17.769753 (+   537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:17.781250 (+ 11497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:17.781286 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:17.903661 (+122375us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.576960 (+673299us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:18.577011 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.577096 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.415925 (+838829us) raft_consensus.cc:1685] finished
0709 13:05:19.415953 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.416134 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1189,"spinlock_wait_cycles":25199360,"thread_start_us":605306,"threads_started":1,"wal-append.queue_time_us":1137908}
I20260709 13:05:19.457078  8830 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:05:19.460484  8830 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:19.462316  8830 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:05:19.462951  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 27 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.458478  8826 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [16.11-18.12]
I20260709 13:05:19.463837  8672 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 27: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:19.460928  8825 raft_consensus.cc:1240] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:05:19.465580  8826 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:05:19.511386  8679 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:19.452358 16288 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.513924  8827 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:19.516168  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:19.526924  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:19.446640 16230 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:19.530809 16230 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:19.532124  8828 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:19.539743 16019 raft_consensus.cc:3055] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:19.540369 16219 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:19.543187  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2787) took 1862 ms. Trace:
I20260709 13:05:19.543893  8823 rpcz_store.cc:276] 0709 13:05:17.680464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.761614 (+ 81150us) service_pool.cc:224] Handling call
0709 13:05:17.776070 (+ 14456us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:17.788470 (+ 12400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:17.788509 (+    39us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:17.826195 (+ 37686us) log.cc:844] Serialized 78 byte log entry
0709 13:05:18.086940 (+260745us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:18.101732 (+ 14792us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.282896 (+181164us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.283325 (+   429us) log.cc:844] Serialized 14 byte log entry
0709 13:05:18.287154 (+  3829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.287266 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:18.524545 (+237279us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b18000c1f60. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1
0709 13:05:18.526422 (+  1877us) spinlock_profiling.cc:243] Waited 5.53 ms on lock 0x7b18000c1f60. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:05:19.530009 (+1003587us) spinlock_profiling.cc:243] Waited 5.81 ms on lock 0x7b18000c1f60. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:05:19.542769 (+ 12760us) raft_consensus.cc:1685] finished
0709 13:05:19.542804 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.543075 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57727,"prepare.queue_time_us":306722,"prepare.run_cpu_time_us":151,"prepare.run_wall_time_us":156,"raft.queue_time_us":70528,"raft.run_cpu_time_us":687,"raft.run_wall_time_us":41367,"spinlock_wait_cycles":117341440,"thread_start_us":333618,"threads_started":2,"wal-append.queue_time_us":460443}
I20260709 13:05:19.559772  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.557343  8827 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:05:19.540452 16019 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:19.536299 15880 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:19.569700  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:19.572202  8672 raft_consensus.cc:2368] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19.
I20260709 13:05:19.533833 16288 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:19.563367 16157 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.565140 16111 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.605942  8830 raft_consensus.cc:1275] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:05:19.614425  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:05:19.607651 16317 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:05:19.615880  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:19.617221  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:19.617583  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:19.626179  8679 raft_consensus.cc:2368] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18.
I20260709 13:05:19.626179  9385 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:19.617919  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:19.629518  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:19.614248 16019 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:19.604349 16157 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:19.614684 16023 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.607981 16111 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:19.634836  8677 raft_consensus.cc:3055] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:19.635998  8677 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:19.638177  8677 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:19.643697  8677 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:19.648473  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.651223  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.651492  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:05:19.653002 16256 log.cc:927] Time spent T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 2.483s	user 0.008s	sys 0.000s
I20260709 13:05:19.653664  8680 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:19.637007 16228 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:19.652067  8679 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:19.642590 15949 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.660986  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:19.661388  8749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2881) took 3311 ms (client timeout 1976 ms). Trace:
I20260709 13:05:19.661976  8679 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 23.
W20260709 13:05:19.661953  8749 rpcz_store.cc:269] 0709 13:05:16.349925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.884639 (+534714us) service_pool.cc:224] Handling call
0709 13:05:19.661141 (+2776502us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b540018d3d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:19.661307 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1040998144}
I20260709 13:05:19.638162 16023 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:19.669907  8793 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:19.671512 16296 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:19.674757  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.675645  8679 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:19.675001  8793 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:19.677274  9381 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 923) took 1614 ms (client timeout 1794 ms). Trace:
W20260709 13:05:19.679208  8645 leader_election.cc:336] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 2.047s (SENT)
I20260709 13:05:19.679692  8645 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:19.682967  9383 raft_consensus.cc:1240] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
W20260709 13:05:19.677723  9381 rpcz_store.cc:269] 0709 13:05:18.062757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.062922 (+   165us) service_pool.cc:224] Handling call
0709 13:05:19.677060 (+1614138us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54006275d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:19.677210 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380723968}
I20260709 13:05:19.684228  8677 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:05:19.685379 16290 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:19.686911  8677 pending_rounds.cc:85] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting all ops after (but not including) 10
I20260709 13:05:19.688066  8677 pending_rounds.cc:107] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:05:19.688455  8677 raft_consensus.cc:2889] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:19.663378 15949 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:19.690876  8793 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:19.692390 16286 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:19.688648 16228 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:19.693575  8793 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:19.696374  8793 leader_election.cc:400] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 19. Current term is 20.
I20260709 13:05:19.693972 16228 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:19.686730 16227 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:05:19.688939 15880 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:19.695423 16288 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:19.704159 16157 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:19.705044 16223 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:19.705617 16223 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:19.698637 16296 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:19.705082 15949 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:19.713646  8824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60424 (request call id 2780) took 2147 ms (client timeout 1893 ms). Trace:
W20260709 13:05:19.714207  8824 rpcz_store.cc:269] 0709 13:05:17.579844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.602064 (+ 22220us) service_pool.cc:224] Handling call
0709 13:05:19.713420 (+2111356us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b54002689d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:19.713567 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1156029184}
W20260709 13:05:19.715791  8754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2893) took 2035 ms (client timeout 2047 ms). Trace:
I20260709 13:05:19.716194  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:19.717502  8680 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:05:19.719682  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.719959  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:19.716208  8754 rpcz_store.cc:269] 0709 13:05:17.686983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.698723 (+ 11740us) service_pool.cc:224] Handling call
0709 13:05:19.715551 (+2016828us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54002fd5d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:19.715729 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":614035200}
I20260709 13:05:19.720577  8677 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:19.716761  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:19.721499  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:19.722409  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.723616  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:19.724143  8681 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:19.724419  8673 raft_consensus.cc:2393] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 17: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
W20260709 13:05:19.724481  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 50799.3us
I20260709 13:05:19.725705  8677 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:05:19.726778  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:19.725235  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:19.727383  8677 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:19.730037  8673 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:05:19.732949  8680 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 24.
W20260709 13:05:19.735524  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.736097  8679 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:05:19.739338  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:19.741217 16023 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:19.728026 16111 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:19.743250  8793 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:19.745503  8793 leader_election.cc:400] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19.
W20260709 13:05:19.751183  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.746876 16111 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:19.752327  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:19.749732  8647 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:19.753710  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:19.754284 16290 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:19.755205 16276 raft_consensus.cc:2764] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:05:19.740164 16284 consensus_queue.cc:1059] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:05:19.756143 16284 consensus_queue.cc:1243] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 0e9b0850145a4c66bfe983ffdfa93a57 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:05:19.756654 16284 raft_consensus.cc:3055] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:19.756980 16284 raft_consensus.cc:740] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:19.754544  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:19.758853  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.764140  8681 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:19.770236  8647 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:19.772526  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:19.757853 16284 consensus_queue.cc:260] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:19.766935 16276 raft_consensus.cc:2764] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:05:19.766750 16228 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:19.774036  8793 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:19.781117  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:19.777304 16220 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:19.777062 16219 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.783057  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:19.755193 16290 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.784029 16290 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:19.784155 16223 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:19.786695 16223 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:19.787719 16019 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:19.791090 15949 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:19.791622 16284 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:19.791858 15880 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:19.791649 15949 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.793603 15949 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:19.769176 16111 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:19.797187  8754 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:05:19.797955  9353 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:19.799661  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:19.800354  8754 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 22.
I20260709 13:05:19.800485 16234 raft_consensus.cc:2804] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:19.801043 16234 raft_consensus.cc:697] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:19.803238 16219 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:19.803788  8796 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:19.804320  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:19.805553 16219 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:19.803854  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:19.806301 16023 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:19.808607 16287 raft_consensus.cc:2804] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:19.809249 16287 raft_consensus.cc:697] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:19.810050  9383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34754 (request call id 1043) took 1580 ms. Trace:
I20260709 13:05:19.810269 16287 consensus_queue.cc:237] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:19.810482  9383 rpcz_store.cc:276] 0709 13:05:18.230022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.243363 (+ 13341us) service_pool.cc:224] Handling call
0709 13:05:19.678560 (+1435197us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.683444 (+  4884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.809828 (+126384us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54006275d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:19.809981 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3299324672}
I20260709 13:05:19.802049 16234 consensus_queue.cc:237] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:19.792479 16306 log.cc:927] Time spent T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.749s	user 0.004s	sys 0.000s
I20260709 13:05:19.823303  8794 leader_election.cc:400] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18.
I20260709 13:05:19.823760  8794 leader_election.cc:403] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 13:05:19.824944  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 52550.5us
I20260709 13:05:19.826578  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:19.815168  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:19.830444  8647 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:19.831346  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.849243  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:19.850435  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:19.850768  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:19.852758  8753 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:19.854529  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:19.856127  8793 leader_election.cc:400] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 18 election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19.
W20260709 13:05:19.857249  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.857249  8793 leader_election.cc:400] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18.
I20260709 13:05:19.858361  8793 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:19.842743 16220 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:19.843884 16228 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:19.862416  8647 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:19.863245  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:19.847549 16239 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18.
I20260709 13:05:19.875770  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:19.876645  8748 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 22: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:19.872301  8754 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:05:19.853165 16276 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:19.864799  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:19.880447  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:19.883584  8749 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:19.883898  8753 raft_consensus.cc:2410] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:05:19.828994 15476 consensus_queue.cc:1059] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:05:19.883620  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:19.886047  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 78347.1us
I20260709 13:05:19.855757 16232 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:19.887796  8753 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:19.846088 15880 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:05:19.889715 16287 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.858547 16286 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:19.891047  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:19.858556 16227 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:19.889815 16239 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:19.877950 16279 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.856650 16288 consensus_queue.cc:1048] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
W20260709 13:05:19.918534  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.871980 16298 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:19.917721  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:19.864779 16197 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:05:19.909714 16220 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:19.934731  9383 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:19.935079  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 72580.3us
W20260709 13:05:19.935546  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 77275.7us
I20260709 13:05:19.922602 15476 raft_consensus.cc:3055] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:19.918155 16276 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:19.937225  8793 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:19.940491  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:19.923837 16287 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:19.932840 16030 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:19.951104  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:19.937917 15476 raft_consensus.cc:740] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:05:19.951151  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 88861.9us
I20260709 13:05:19.954952  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:19.953552  8793 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:19.947309 16232 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:19.944958 16239 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:19.949635 16030 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.958045 16030 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:19.960599  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:19.961828  8822 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:05:19.963214 16328 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:05:19.970821  8824 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:05:19.972697 16051 consensus_queue.cc:1059] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:05:19.973045  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:19.975488 15476 consensus_queue.cc:260] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:19.967100 16297 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.434s	user 0.002s	sys 0.002s
I20260709 13:05:19.977034 15476 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:19.978452 16337 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.924453 16279 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:19.980441  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:19.951647 15880 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.983014  8793 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:19.984166  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:19.984920  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:19.987509  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.945329 16276 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:19.992879  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.981621 16051 consensus_queue.cc:1243] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:05:19.982868 16337 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:19.990115  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:19.998278  8678 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:19.982648 16284 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:20.001003  8647 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:20.003075 16228 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:20.002962 16241 consensus_queue.cc:1059] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:19.983594 16239 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:19.983743 16336 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:19.310596  8756 rpcz_store.cc:276] 0709 13:05:18.221982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.919863 (+697881us) service_pool.cc:224] Handling call
0709 13:05:18.947283 (+ 27420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.947529 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:19.046147 (+ 98618us) log.cc:844] Serialized 14 byte log entry
0709 13:05:19.274028 (+227881us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:19.274112 (+    84us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:19.274185 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.274286 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.274511 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":311506,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":1666,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":20812,"thread_start_us":325355,"threads_started":2}
I20260709 13:05:19.998348 16334 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:20.000149 15926 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:20.011188 16334 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.011673 16334 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:20.015462 16287 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:20.009140 16336 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.017652 16336 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:19.989742 15880 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.023070 16220 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:20.024761 16298 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:20.025090 15880 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:20.025344 16298 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:20.005229 16051 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:05:20.026081 16330 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:05:19.996883 16252 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
W20260709 13:05:20.060824  9388 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34746 (request call id 921) took 2040 ms (client timeout 1627 ms). Trace:
I20260709 13:05:20.005858  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:20.042960  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.997350 16234 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:20.061381  9388 rpcz_store.cc:269] 0709 13:05:18.020846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.020968 (+   122us) service_pool.cc:224] Handling call
0709 13:05:20.060532 (+2039564us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54004b64d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:20.060720 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":987375872}
I20260709 13:05:19.996528 16332 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:20.069293 15926 raft_consensus.cc:697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:20.004479 16232 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:20.026587 16298 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:20.082496  8795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.988505 16108 consensus_queue.cc:1059] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 13:05:20.080773  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:20.028596 16239 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.085188  8828 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 23.
W20260709 13:05:20.077525  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 71757.2us
I20260709 13:05:20.086934  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:20.092994  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 95644us
I20260709 13:05:20.087939 16239 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:05:20.142004  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 142504us
W20260709 13:05:20.143072  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.144971  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.073180 16290 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.149529  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:20.073578 15949 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.068629 16234 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:20.152503  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 65699.8us
I20260709 13:05:20.158576  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:20.159439  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:20.095441 16343 raft_consensus.cc:2764] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election decision vote started in defunct term 21: won
I20260709 13:05:20.166498  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:20.167578  8749 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 22.
W20260709 13:05:20.160746  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 169629us
I20260709 13:05:20.159688  8752 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:20.076427 16332 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:20.075855 15926 consensus_queue.cc:237] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.168790  8793 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:20.090256 16030 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:20.149693 16298 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:20.166553 16333 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.219s	user 0.000s	sys 0.002s
I20260709 13:05:20.162268 16287 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:20.176723  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.177201  8821 raft_consensus.cc:3060] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:20.153968 16232 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:05:20.191736 16335 raft_consensus.cc:3055] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:20.157015 15880 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:20.192207 16335 raft_consensus.cc:740] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:20.156306 16227 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:05:20.193225 16335 consensus_queue.cc:260] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.183317  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:20.161302 16345 raft_consensus.cc:3055] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:20.161688 16228 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:20.241976  8672 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:20.187613 16221 raft_consensus.cc:3055] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:20.190426  8752 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:20.182229  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:20.196745 16030 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:20.203531  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:20.194595 16286 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:20.199993  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:20.265409  8753 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:05:20.204996  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:20.199072  8676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1489) took 1868 ms. Trace:
I20260709 13:05:20.219748  8754 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:20.267403  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:20.222389  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:20.267818 16335 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:20.268044  8754 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:05:20.268225  8679 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:20.220768  8748 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:05:20.182940 16019 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:20.269878 16019 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.270308 16019 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:20.270535  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:20.271436  8748 raft_consensus.cc:2410] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:05:20.272272  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:20.255134  8678 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:20.272791  8748 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:20.276286  8747 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:05:20.277791  8747 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:05:20.222856  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.283661  8747 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:05:20.223238 16219 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.256279 16008 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:05:20.266291  8676 rpcz_store.cc:276] 0709 13:05:18.331070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.364898 (+ 33828us) service_pool.cc:224] Handling call
0709 13:05:18.397915 (+ 33017us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:18.400388 (+  2473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.400422 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.400922 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:05:19.822090 (+1421168us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b1800423fa0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e20990d1 00007f32e24b4a0f
0709 13:05:19.917425 (+ 95335us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:19.917492 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.917609 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:20.198733 (+281124us) raft_consensus.cc:1685] finished
0709 13:05:20.198756 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.199007 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3816,"raft.queue_time_us":2122,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":7573,"spinlock_wait_cycles":37132160,"thread_start_us":95045,"threads_started":1,"wal-append.queue_time_us":1527793}
I20260709 13:05:20.258174  9388 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:20.259188  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:20.287300  8753 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 24.
I20260709 13:05:20.243522 16296 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:05:20.295429  8752 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:20.296455  8753 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:20.231266 16345 raft_consensus.cc:740] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:20.315431  8821 raft_consensus.cc:1275] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 13:05:20.247779 16221 raft_consensus.cc:740] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:20.204901 16287 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:20.329413  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.333976  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 191598us
I20260709 13:05:20.256623 16223 consensus_queue.cc:1059] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 26.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 13:05:20.247721 16334 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.361289  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1425) took 2452 ms. Trace:
I20260709 13:05:20.361900  8755 rpcz_store.cc:276] 0709 13:05:17.908391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.220755 (+312364us) service_pool.cc:224] Handling call
0709 13:05:18.225143 (+  4388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.587677 (+362534us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:18.931822 (+344145us) log.cc:844] Serialized 14 byte log entry
0709 13:05:19.349413 (+417591us) log.cc:844] Serialized 14 byte log entry
0709 13:05:19.351156 (+  1743us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:19.351219 (+    63us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:19.351276 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.795889 (+444613us) spinlock_profiling.cc:243] Waited 6.36 ms on lock 0x7b18003e9fe0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:05:19.954066 (+158177us) spinlock_profiling.cc:243] Waited 8.27 ms on lock 0x7b18003e9fe0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1
0709 13:05:20.060677 (+106611us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b18003e9fe0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419
0709 13:05:20.360947 (+300270us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b54002ac454. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:20.361019 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.361204 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48351,"prepare.queue_time_us":673012,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":18665,"raft.run_cpu_time_us":539,"raft.run_wall_time_us":27898,"spinlock_wait_cycles":859286656,"thread_start_us":577188,"threads_started":2,"wal-append.queue_time_us":441149}
I20260709 13:05:20.365018  9352 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:20.359635 16052 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 13:05:20.363492 16030 consensus_queue.cc:1048] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:05:20.393198 16030 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:05:20.394559 16346 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:05:20.372555  8748 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:05:20.399982  8747 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:05:20.373399  8754 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 23.
I20260709 13:05:20.330754 16232 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:05:20.355914 16276 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:20.376768  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.378532 16205 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:20.322041 16219 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.383713 16345 consensus_queue.cc:260] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.405462 16345 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:20.405958 16095 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:20.372385  9388 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:20.385301 16290 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:20.406896 16332 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:20.407631 16228 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.413738 16284 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:20.416317 16219 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:20.416990 16219 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.418686  8680 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18. Ops: []
W20260709 13:05:20.422044  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 262828us
W20260709 13:05:20.369925 16347 log.cc:927] Time spent T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:05:20.425930 16350 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:20.427899  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:05:20.437062  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 351213us
W20260709 13:05:20.440064  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.442502  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 198376us
I20260709 13:05:20.447532  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:20.417482 16219 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:20.421387 16287 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:20.424744  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 139251us
I20260709 13:05:20.365725  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1543) took 1786 ms. Trace:
I20260709 13:05:20.450819  8821 rpcz_store.cc:276] 0709 13:05:18.579897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.580174 (+   277us) service_pool.cc:224] Handling call
0709 13:05:18.863562 (+283388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:20.172561 (+1308999us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b58006b93d0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24550aa 00007f32e24d7447 00007f32e24effe7 00007f32e24f17de 00007f32e24fbb49 00007f32e24fc416 00007f32e24f8d8f 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4
0709 13:05:20.356601 (+184040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.365647 (+  9046us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1854216064}
W20260709 13:05:20.425324  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.451964 16365 raft_consensus.cc:3055] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:05:20.452764 16365 raft_consensus.cc:740] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:20.448879  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:20.459205  8722 connection.cc:256] RPC call timeout handler was delayed by -1.5324569209999481s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2025s for a response
W20260709 13:05:20.484469  8722 connection.cc:256] RPC call timeout handler was delayed by -1.4865569209999876s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2076s for a response
I20260709 13:05:20.485021  8828 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:20.485803  8825 raft_consensus.cc:1275] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:05:20.482292  9382 raft_consensus.cc:1240] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: [17.13-19.14]
I20260709 13:05:20.489037  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:20.489708  8828 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:20.473065  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:05:20.456367  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 62031us
I20260709 13:05:20.501833  8825 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:05:20.502053  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:20.500520  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:20.505577  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:20.506652  8822 raft_consensus.cc:2393] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 21: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:20.505528  8719 leader_election.cc:400] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20.
I20260709 13:05:20.508502  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:20.508772  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:20.509467  8821 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:20.508543  8719 leader_election.cc:403] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:20.510865  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:20.512095  8825 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:20.513878  8823 raft_consensus.cc:1240] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:20.454056 16279 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:20.538558  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:20.539110  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:20.539335  8680 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:20.539741  8830 raft_consensus.cc:2376] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21. Re-sending same reply.
I20260709 13:05:20.515260  8828 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 23.
I20260709 13:05:20.515642  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:20.518270 16219 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:20.519132 16344 log.cc:927] Time spent T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.332s	user 0.000s	sys 0.004s
W20260709 13:05:20.520938 16362 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.054s	user 0.000s	sys 0.002s
I20260709 13:05:20.521483 15949 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:20.491181 16223 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:20.472082 16364 consensus_queue.cc:1059] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:05:20.522966 16356 consensus_queue.cc:1048] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 13:05:20.528786 15476 consensus_meta.cc:287] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.551s	user 0.014s	sys 0.000s
I20260709 13:05:20.501076 16157 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:05:20.591744  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:20.463132 16197 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.081s
W20260709 13:05:20.531960 16360 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:05:20.532502  9381 raft_consensus.cc:1240] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
W20260709 13:05:20.533209  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 61662.2us
I20260709 13:05:20.534979 16337 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:20.536568 16241 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:20.613663  8680 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:20.538300  9386 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 13:05:20.542017  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.541955  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:20.542553  9383 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:20.543286  8722 connection.cc:256] RPC call timeout handler was delayed by -1.401056920999963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2171s for a response
I20260709 13:05:20.624198  8828 raft_consensus.cc:2376] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19. Re-sending same reply.
W20260709 13:05:20.547227  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 177597us
I20260709 13:05:20.549000  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:20.550554  8793 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:20.554672  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:20.554917  8827 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:20.453969 16023 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 13:05:20.571907  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:20.574182  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:20.627766  8675 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 22.
I20260709 13:05:20.575582  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:20.629091  8825 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:20.624857  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:20.578548  8751 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:05:20.630086  8755 raft_consensus.cc:3060] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:05:20.579205  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:20.544718  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:20.631311  8822 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:20.481663 16205 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 13:05:20.636183  8752 raft_consensus.cc:2393] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 17: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:20.600436  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:05:20.636759  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 86270.7us
I20260709 13:05:20.637651  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:20.587285  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:20.603335  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:20.604049  8750 raft_consensus.cc:1275] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:20.639653  8793 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:20.538507 16234 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:20.537988 16334 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:20.454303 16221 consensus_queue.cc:260] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:20.619393  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.624282  8722 connection.cc:256] RPC call timeout handler was delayed by -1.1481569209998952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2452s for a response
W20260709 13:05:20.654323  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0884461249999049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
W20260709 13:05:20.654928  8722 connection.cc:256] RPC call timeout handler was delayed by -1.0380461249999371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1572s for a response
I20260709 13:05:20.545492  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:20.626426  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:20.655887  8722 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:20.656562  8830 raft_consensus.cc:2376] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22. Re-sending same reply.
I20260709 13:05:20.642984  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:20.661376  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.662339  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:20.663127  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:20.663591  8675 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 20: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:20.645150  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:20.664626  8825 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:20.656602  8828 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:20.657153  8722 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:20.666502  9353 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:20.663811  8827 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
W20260709 13:05:20.645754  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 180911us
I20260709 13:05:20.675493  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:20.501909 16365 consensus_queue.cc:260] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.676404  8825 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:20.603006 16223 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:20.685655  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 251663us
I20260709 13:05:20.611104 16019 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.640805 16364 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:05:20.652760 16059 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:20.653098 15949 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:20.662904 16286 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.643155 16219 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221) with higher term. Message: Invalid argument: T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20.
I20260709 13:05:20.658588 16258 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:20.681174 16221 raft_consensus.cc:3060] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:20.705639 16232 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:05:20.719453 16286 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.641377 16239 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:20.721766 16234 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:20.722266  8824 raft_consensus.cc:2393] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 24: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:20.722388 16234 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:20.722549 16239 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:20.722889 15949 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.723340 15949 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:20.724165  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:20.724880  8680 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:20.726126 16258 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.726606 16258 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:20.726997  8826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60438 (request call id 2817) took 1470 ms. Trace:
I20260709 13:05:20.722910  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:20.729257  8675 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:20.727438  8826 rpcz_store.cc:276] 0709 13:05:19.256148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.466687 (+210539us) service_pool.cc:224] Handling call
0709 13:05:19.467113 (+   426us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:19.468833 (+  1720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:19.468882 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:20.009046 (+540164us) spinlock_profiling.cc:243] Waited 6.28 ms on lock 0x7b2c000f2de8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e0df2a11 00007f32e0e02372 00007f32e2534a01 00007f32e24f943a 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2
0709 13:05:20.009755 (+   709us) log.cc:844] Serialized 52 byte log entry
0709 13:05:20.092610 (+ 82855us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:20.185168 (+ 92558us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.226097 (+ 40929us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.226599 (+   502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.226705 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:20.415523 (+188818us) spinlock_profiling.cc:243] Waited 4.49 ms on lock 0x7b48000cd158. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:05:20.417138 (+  1615us) spinlock_profiling.cc:243] Waited 3.75 ms on lock 0x7b18004c12c0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:05:20.518404 (+101266us) spinlock_profiling.cc:243] Waited 35.1 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:20.726706 (+208302us) raft_consensus.cc:1685] finished
0709 13:05:20.726733 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.726922 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":248794,"prepare.queue_time_us":148652,"prepare.run_cpu_time_us":155,"prepare.run_wall_time_us":5424,"raft.queue_time_us":41737,"raft.run_cpu_time_us":2730,"raft.run_wall_time_us":198675,"spinlock_wait_cycles":114141440,"thread_start_us":115824,"threads_started":2,"wal-append.queue_time_us":156787}
I20260709 13:05:20.735054  8793 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:20.736188 16052 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:20.736677 16052 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.737042 16052 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:20.716522 16223 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:20.740368  8680 raft_consensus.cc:2468] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 24.
I20260709 13:05:20.744074 16355 raft_consensus.cc:3055] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:20.735054  8722 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:20.745072 16336 consensus_meta.cc:287] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.727s	user 0.010s	sys 0.004s
W20260709 13:05:20.754585  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.663398 16252 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.723567 16234 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:20.702281 16359 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:20.670720 16157 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:05:20.759774  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.667042 16341 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:20.769237  8793 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:20.773729  8675 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:05:20.777848  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:20.799223  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:20.799678  8747 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:20.801726  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:20.802373  8676 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:20.788048  8755 raft_consensus.cc:2468] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
W20260709 13:05:20.790443  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 55512.3us
I20260709 13:05:20.702795  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:20.756138 16019 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:20.805514 16341 raft_consensus.cc:697] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:20.754601 16286 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:20.806262  8753 raft_consensus.cc:1275] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:05:20.693097 16298 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.806715 16341 consensus_queue.cc:237] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.807163 16221 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:20.807447 16298 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:20.808214 16221 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:20.808720 16221 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:20.806756  8753 pending_rounds.cc:85] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 14
I20260709 13:05:20.810089  8753 pending_rounds.cc:107] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:05:20.810329  8753 raft_consensus.cc:2889] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:20.705215 16059 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.810930 16221 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:20.811331 16059 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:20.715715 16365 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:05:20.757596 16355 raft_consensus.cc:740] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:20.813367  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:20.813321  8825 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 24.
W20260709 13:05:20.821511  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.829360  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.839924  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 137279us
W20260709 13:05:20.844038  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 338588us
I20260709 13:05:20.854250  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:20.810385 16298 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:20.818389  8676 raft_consensus.cc:2468] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:20.796567 16157 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.856081  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:20.798059 16361 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:20.813900 16239 raft_consensus.cc:2804] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:20.859324  8793 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:20.761162 16337 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:05:20.870599  9383 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:20.863974 16220 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:20.883695  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:20.818828 16221 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:05:20.752427 16052 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.797020 16336 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:20.869135  8722 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:20.845515 16350 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:05:20.904399  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:20.797736 16223 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.069s
I20260709 13:05:20.797247 16252 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.851794 15926 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:20.909435  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 264720us
I20260709 13:05:20.799484 15949 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.903779  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:20.980093  8827 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
W20260709 13:05:20.953294  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 104872us
I20260709 13:05:20.864437 16157 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:20.903394 16019 raft_consensus.cc:2804] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:20.820271 16342 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 13:05:21.033434  8673 raft_consensus.cc:1275] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:05:21.001322  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.994375  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 90068us
I20260709 13:05:21.049458  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:21.005632  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:21.052534  8829 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 22: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:21.052999  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:20.913436 16361 raft_consensus.cc:697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:05:21.065718  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 399317us
I20260709 13:05:21.024843  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:20.912927 16228 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:20.911619 16239 raft_consensus.cc:697] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:20.957655 16205 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.894088 16108 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.131s
I20260709 13:05:20.893558 16258 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:20.893931 16290 consensus_queue.cc:1048] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.131s
I20260709 13:05:21.076588 16095 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:21.040704 16296 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.718s	user 0.011s	sys 0.001s
I20260709 13:05:20.983227 16334 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.955466 16367 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:21.083540  8676 raft_consensus.cc:1275] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:05:21.068521  8674 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 23.
I20260709 13:05:21.079049 16357 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:20.911619 16220 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.028726 16277 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.987593 16374 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:21.024550 16298 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.102s
I20260709 13:05:21.071470 16228 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.088539 16337 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:21.081252 16019 raft_consensus.cc:697] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:20.986896 15926 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:21.139473  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 270389us
I20260709 13:05:21.141074  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:21.101956 16296 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:05:21.145569  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:21.131763 16286 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.132403 16348 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:21.143110 16337 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:21.222858 16337 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:21.140901 16095 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:21.103214 16357 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:21.139190 16228 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:21.087916 16205 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:21.167364 16333 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:05:21.082990 16355 consensus_queue.cc:260] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.106088 16334 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.170713 16298 consensus_queue.cc:1243] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:05:21.235631  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 186270us
W20260709 13:05:21.240789  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.108961 16367 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.158659 16359 raft_consensus.cc:697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:21.141901 16197 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 13:05:21.242920  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:05:21.249521  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.205574 16286 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.140678 16277 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.262313  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:21.263461  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 203663us
W20260709 13:05:21.263816  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.270341  8719 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:21.208330 16239 consensus_queue.cc:237] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.275067  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:21.249310 16348 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:21.282043 16252 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:21.282308 16359 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.283661 16220 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:21.269289 16052 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:21.290776 16108 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:21.264623 16290 consensus_queue.cc:1059] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:05:21.225214 16361 consensus_queue.cc:237] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.259387 16019 consensus_queue.cc:237] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:21.312057  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:21.293097 16354 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:05:21.289253 15949 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:21.310256 16371 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 13:05:21.310056 16370 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:05:21.293572 16387 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:21.293320 16377 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:21.319270 16258 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:21.321332 15476 raft_consensus.cc:3055] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:21.329872 16377 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.329582 16387 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:21.368278  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:21.369027  8677 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:05:21.313215 16389 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
W20260709 13:05:21.358831 16373 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:05:21.338907 16390 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:21.371101  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:21.360867 15476 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:05:21.388712 16388 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:05:21.388952 16205 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:21.388530 15476 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.390336 15476 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:21.336733 16348 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.366309 16377 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:21.400938 16030 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.401638 16030 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.369544 16387 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:21.404192 16372 log.cc:927] Time spent T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
W20260709 13:05:21.404429 16381 log.cc:927] Time spent T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:05:21.405678 16334 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:21.408330 15880 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.412665  8677 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 13:05:21.413916 16108 raft_consensus.cc:3055] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:21.414378 16108 raft_consensus.cc:740] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:21.415364 16340 consensus_queue.cc:1059] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:21.415285 16108 consensus_queue.cc:260] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.416647 16340 raft_consensus.cc:3055] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:05:21.416837 16108 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:21.417107 16340 raft_consensus.cc:740] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:05:21.388518 16059 consensus_meta.cc:287] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.000s
I20260709 13:05:21.419662 16059 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.421800 16340 consensus_queue.cc:260] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.422886 16357 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.424264 16357 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:21.427030 16277 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:21.372042 15926 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:21.359318 16336 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:21.430266 16030 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:21.431237  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:21.432052  8673 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
W20260709 13:05:21.432405 16394 log.cc:927] Time spent T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:05:21.433347  9352 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:21.434773 16030 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:21.435289 16030 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.435716 16030 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:21.438760 16390 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.441648 16390 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:21.379377  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1434) took 1307 ms. Trace:
I20260709 13:05:21.448577 16059 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:21.381295 16375 log.cc:927] Time spent T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
W20260709 13:05:21.381616 16365 consensus_meta.cc:287] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.561s	user 0.009s	sys 0.002s
I20260709 13:05:21.383200 16364 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:21.451890 16364 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.452423 16364 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:21.385349 16220 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.387231 16286 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:21.446880  8756 rpcz_store.cc:276] 0709 13:05:20.072499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.074786 (+  2287us) service_pool.cc:224] Handling call
0709 13:05:20.075402 (+   616us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:20.075993 (+   591us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:05:20.076382 (+   389us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.687751 (+611369us) log.cc:844] Serialized 14 byte log entry
0709 13:05:21.094215 (+406464us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:21.138291 (+ 44076us) log.cc:844] Serialized 52 byte log entry
0709 13:05:21.142366 (+  4075us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:21.142434 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.142542 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:21.379113 (+236571us) raft_consensus.cc:1685] finished
0709 13:05:21.379139 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:21.379305 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93367,"prepare.queue_time_us":47488,"prepare.run_cpu_time_us":152,"prepare.run_wall_time_us":5667,"raft.queue_time_us":119838,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":6010,"thread_start_us":917547,"threads_started":3,"wal-append.queue_time_us":233568}
I20260709 13:05:21.380405 16367 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:21.458304  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.482600 16220 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.485347  8647 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:21.485829 16220 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:21.486799 16336 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:21.487516 16336 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.490002 16336 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:21.491571 16335 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:21.492147 16335 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.497337  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:21.497556  8824 raft_consensus.cc:1240] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 22. Current term is 23. Ops: [20.13-22.14]
I20260709 13:05:21.497601  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:21.498082  8753 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:21.498543  8755 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 22: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:21.479620 16030 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.503990  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:21.506826  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:21.512341  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:21.513005  8830 raft_consensus.cc:2410] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 22 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:05:21.513337  8747 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:21.512393  8824 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:21.515713  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:21.516641  8751 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:05:21.519654  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:21.520704  8755 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 22.
I20260709 13:05:21.508742  8794 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:21.533239  8825 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:21.539172  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:21.540302  8821 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15.
I20260709 13:05:21.546648  8828 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:05:21.499580 16336 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.495496 16220 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:21.550652  8794 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:21.553556  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:21.554348  8756 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:21.455116 15880 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:21.563617 16380 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:05:21.557548 16336 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.569959 16336 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:21.572180 16030 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:21.573158  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:21.573837  8673 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:21.579577  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:21.582554  8673 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 18.
I20260709 13:05:21.584159  9352 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:21.585104 16023 raft_consensus.cc:2804] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:21.585635 16023 raft_consensus.cc:697] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:21.586637 16023 consensus_queue.cc:237] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.587522 16340 raft_consensus.cc:3060] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:21.588032  8754 raft_consensus.cc:3055] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:21.588428  8754 raft_consensus.cc:740] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:21.589262  8754 consensus_queue.cc:260] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.590353  8754 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:21.593854  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60438 (request call id 2838) took 1047 ms. Trace:
I20260709 13:05:21.594383  8829 rpcz_store.cc:276] 0709 13:05:20.546787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.547195 (+   408us) service_pool.cc:224] Handling call
0709 13:05:21.593583 (+1046388us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b54002689d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:21.593775 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941992192}
I20260709 13:05:21.549983 16296 raft_consensus.cc:2804] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:21.600229  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:21.554880 16220 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.522034 16157 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.621510  8751 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:21.628609  8756 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:21.626564  8754 raft_consensus.cc:1275] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:05:21.630208  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:21.641386  8753 raft_consensus.cc:2410] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:05:21.559957 16334 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:21.642709  8754 pending_rounds.cc:85] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 10
I20260709 13:05:21.549656 16286 consensus_queue.cc:1048] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 13:05:21.644794  8754 pending_rounds.cc:107] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:05:21.639850 16108 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:05:21.647075  9353 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.030s	sys 0.042s
I20260709 13:05:21.652220  8754 raft_consensus.cc:2889] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:21.652824  9353 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.030s	sys 0.042s
W20260709 13:05:21.655577  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 587314us
I20260709 13:05:21.658632  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:21.650557 16059 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.661185 16356 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:21.661257 16059 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.662475 16387 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:21.663189  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.663684  8672 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:21.664865 16387 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:21.650171 16399 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:21.665915 16399 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:21.666400 16399 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:21.667052  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:21.667802  8829 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:21.669395 16059 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:21.671487  8828 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:05:21.672791 16157 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:21.665653 16364 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.644493 16334 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.676640 16334 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:21.609473 16350 consensus_queue.cc:1059] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:05:21.676843 16364 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:21.675773 16343 consensus_queue.cc:1059] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:21.677578 16350 raft_consensus.cc:3055] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:21.677888 16343 consensus_queue.cc:1243] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:21.677981 16350 raft_consensus.cc:740] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:21.640488 16276 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:05:21.679617 16350 consensus_queue.cc:260] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.653564 16345 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:05:21.681392 16350 raft_consensus.cc:3060] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:21.685619  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:21.686533  8673 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:21.691244  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.637239 16220 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:21.641485 16296 raft_consensus.cc:697] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:21.107517  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:21.654685 16108 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:21.714323  8673 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:05:21.728646  9352 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:21.729676 16345 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.730458 16345 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.733652 16345 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:21.695858 15476 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:21.691040 16330 consensus_queue.cc:1048] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:05:21.759444  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:21.761008  8821 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:05:21.724776 16108 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:21.764241  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:21.707458 16258 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.657167 16377 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:21.772864  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.786911  8829 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:21.765434 16345 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:21.789047  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:21.800519  8680 raft_consensus.cc:3055] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:21.803210  8680 raft_consensus.cc:740] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:21.763531 16335 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:21.772785 16337 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.547s	user 0.014s	sys 0.000s
W20260709 13:05:21.818202  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.804412  8680 consensus_queue.cc:260] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.712832 16363 consensus_queue.cc:1048] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
W20260709 13:05:21.774699 16228 consensus_meta.cc:287] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.521s	user 0.012s	sys 0.001s
I20260709 13:05:21.825284  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:21.838136  8823 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:05:21.801921 16345 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.828215  8680 raft_consensus.cc:3060] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:21.815716 16343 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:21.772241 16258 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.763984 15476 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.819638 16336 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.822311 16052 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.805730  8747 raft_consensus.cc:2468] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 23.
W20260709 13:05:21.829260  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 100768us
I20260709 13:05:21.828066 16023 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:21.865111  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.880460  8681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 3233) took 1672 ms. Trace:
I20260709 13:05:21.881958  8681 rpcz_store.cc:276] 0709 13:05:20.179023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.179184 (+   161us) service_pool.cc:224] Handling call
0709 13:05:21.850888 (+1671704us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54002409d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:21.851057 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1511795584}
I20260709 13:05:21.843632 16345 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:21.821743 16337 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:21.889124  8821 raft_consensus.cc:1240] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:21.890245  8680 raft_consensus.cc:1275] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:21.791857 16296 consensus_queue.cc:237] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.894958  8680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 3244) took 1124 ms. Trace:
I20260709 13:05:21.896270  8823 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:05:21.855677 16343 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:21.926847  8680 rpcz_store.cc:276] 0709 13:05:20.770434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.772600 (+  2166us) service_pool.cc:224] Handling call
0709 13:05:20.773112 (+   512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.800364 (+1027252us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b540040f4a0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e2470dc0 00007f32e24fb1c0 00007f32e24fc231 00007f32e24f8d8f 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c
0709 13:05:21.894480 (+ 94116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.894879 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481407232}
I20260709 13:05:21.842919 16228 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.893064 15880 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:21.906221 16284 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
W20260709 13:05:21.966125  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.903719 16023 raft_consensus.cc:697] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:21.880460 16336 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:21.884521 16052 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:21.988948  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.997558  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:21.968163 16390 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.968583 16298 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:22.000638  9388 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 22: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:22.008026  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:22.020707 16411 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:05:22.022190 16364 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.023167 16364 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:22.025571 16336 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:22.025691 16364 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:22.026366 15880 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:22.027040 15880 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:22.028941 16258 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:22.033427  8796 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:22.057485  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:22.058354  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.059106 16228 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.059782 16023 consensus_queue.cc:237] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:22.013396 16298 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.068797 16337 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:22.030627 16387 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.069646 16377 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:22.070101 15926 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.996115 16277 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:05:22.023149 16404 log.cc:927] Time spent T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
W20260709 13:05:22.089241  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 207967us
I20260709 13:05:22.071638 16336 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:22.118899  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:22.130213  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:22.131139  9382 raft_consensus.cc:2468] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:22.030828 16343 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:22.125250 16356 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:22.132957  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:22.133337  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.144748  9383 raft_consensus.cc:2393] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 24: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:22.133714  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:22.133965 16419 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:22.134835 16364 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:05:22.136922 16334 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.101243 16288 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.012475 16390 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:22.153700  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.153689  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:22.161608  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:22.158744  9388 raft_consensus.cc:1240] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:05:22.165865  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:22.166118  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:22.166586  8680 raft_consensus.cc:3060] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:22.166179  8676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:22.167475  8681 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 19: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:22.168012  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
W20260709 13:05:22.176050  8645 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.696s (SENT) [suppressed 18 similar messages]
I20260709 13:05:22.177157  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:22.182392  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:22.189332  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:22.190683  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:22.191643  8677 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 22.
I20260709 13:05:22.193540  8680 raft_consensus.cc:2468] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:22.153420 16348 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.195122  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:22.195866  8680 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:22.195974  8679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:22.117837 16387 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.199841  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:22.130267 15926 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.127363 16277 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:22.209471  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:22.156282 16431 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.212325  9385 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:22.216517 16350 consensus_meta.cc:287] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.535s	user 0.015s	sys 0.000s
I20260709 13:05:22.217226 16350 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.217743 16350 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.211494 16348 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.210510  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:22.222420  8672 raft_consensus.cc:2410] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:05:22.212109  8680 raft_consensus.cc:2468] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 23.
W20260709 13:05:22.208566  8645 leader_election.cc:336] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.976s (SENT)
I20260709 13:05:22.180377 16423 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:05:22.265273  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:22.413398  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 183716us
I20260709 13:05:22.281627  8673 raft_consensus.cc:2410] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
W20260709 13:05:22.258301  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 1.11267e+06us
I20260709 13:05:22.420395  8679 raft_consensus.cc:3060] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:22.287328  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.259193 16221 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:05:22.432703  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.259029 16383 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.182s
W20260709 13:05:22.435672  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 227109us
I20260709 13:05:22.432744 16356 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:22.441429  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.172355 16288 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.267889  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3307) took 1493 ms. Trace:
I20260709 13:05:22.257444 16223 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:22.268744 16365 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:22.466513  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 421791us
I20260709 13:05:22.267112  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1437) took 1710 ms. Trace:
W20260709 13:05:22.260838  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 990426us
I20260709 13:05:22.475378  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:22.476043  8719 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:22.459810  8752 rpcz_store.cc:276] 0709 13:05:20.774318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.774564 (+   246us) service_pool.cc:224] Handling call
0709 13:05:20.775077 (+   513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:21.096426 (+321349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.096462 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:21.820454 (+723992us) spinlock_profiling.cc:243] Waited 4.31 ms on lock 0x7b2c000f0558. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e0df2a11 00007f32e0e02372 00007f32e2534a01 00007f32e24f943a 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2
0709 13:05:21.820926 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 13:05:21.971353 (+150427us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:21.991728 (+ 20375us) log.cc:844] Serialized 14 byte log entry
0709 13:05:22.267319 (+275591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.267430 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:22.267490 (+    60us) raft_consensus.cc:1685] finished
0709 13:05:22.267627 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.267800 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":290879,"prepare.queue_time_us":24930,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":309552,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":173,"spinlock_wait_cycles":9920256,"thread_start_us":413414,"threads_started":2,"wal-append.queue_time_us":167844}
W20260709 13:05:22.477018  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.467628  8722 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:22.414762 16426 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.218s	user 0.000s	sys 0.000s
I20260709 13:05:22.469615  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:22.485436  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:22.485937  8719 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:22.486092  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:22.472990  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:22.486943  8719 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:22.487793  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:22.488332  8719 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:22.458489  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:22.477126  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:22.477202  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:22.489626  8755 raft_consensus.cc:3055] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:22.489876  8753 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:22.490101  8755 raft_consensus.cc:740] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:05:22.453662 16220 consensus_meta.cc:287] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.732s	user 0.016s	sys 0.000s
I20260709 13:05:22.237110 16345 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.466703 16383 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.484241 16365 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:22.485061  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 53719us
I20260709 13:05:22.260612 16385 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:22.445429 15476 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:22.518711 16345 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.519495 16345 raft_consensus.cc:2804] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:22.519184 16383 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:22.519914 16345 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.520303 16345 raft_consensus.cc:3060] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:22.471359  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:22.475014  8748 rpcz_store.cc:276] 0709 13:05:20.556459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.556656 (+   197us) service_pool.cc:224] Handling call
0709 13:05:20.557273 (+   617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:20.573377 (+ 16104us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:05:20.616154 (+ 42777us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.912884 (+296730us) log.cc:844] Serialized 14 byte log entry
0709 13:05:21.320662 (+407778us) log.cc:844] Serialized 14 byte log entry
0709 13:05:21.407162 (+ 86500us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b180028eac0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c3f3 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd 000000000038f354 000000000038f199 00007f32df3ef83a
0709 13:05:22.257003 (+849841us) spinlock_profiling.cc:243] Waited 27.5 ms on lock 0x7b180028eac0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32e63cd58f 00007f32df40c571 00007f32df40e7c1 00007f32df40e779 00007f32df40e709 00007f32df40e6d1 00007f32df40d9cd
0709 13:05:22.263033 (+  6030us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:22.263657 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 13:05:22.263854 (+   197us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:22.263897 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.263976 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:22.266891 (+  2915us) raft_consensus.cc:1685] finished
0709 13:05:22.266914 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.267046 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1818519,"prepare.queue_time_us":42235,"prepare.run_cpu_time_us":216,"prepare.run_wall_time_us":13007,"raft.queue_time_us":2597413,"raft.run_cpu_time_us":567,"raft.run_wall_time_us":8570,"spinlock_wait_cycles":88545024,"thread_start_us":697786,"threads_started":2,"wal-append.queue_time_us":549735}
I20260709 13:05:22.490949  8755 consensus_queue.cc:260] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:22.491776 16415 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
I20260709 13:05:22.492141 16386 raft_consensus.cc:2764] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:05:22.493192 16396 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:22.493680 16359 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:22.494132 16008 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:22.474501 16384 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:22.496070  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:22.534861  8754 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:22.497948  8719 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:22.259603 16374 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:22.260268 16095 raft_consensus.cc:697] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:22.445371 16334 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:22.547533  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:22.548358  8754 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:22.507076  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:22.485146 16382 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:22.496090 16342 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:22.481036 16340 consensus_meta.cc:287] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.893s	user 0.012s	sys 0.004s
I20260709 13:05:22.468303 16223 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.556623  8679 raft_consensus.cc:2468] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:22.512990  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:22.505270 16228 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.501988 16220 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.506071 16252 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:05:22.517506 16298 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.523674  8753 raft_consensus.cc:2410] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:05:22.525893  8750 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:05:22.528463  9353 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:22.440225 16052 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:22.456894 16221 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:22.544358  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:22.546674  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.541518 16008 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
W20260709 13:05:22.553530  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46554.9us
I20260709 13:05:22.265539 16392 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:22.519703 16365 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:22.559796  8756 raft_consensus.cc:3055] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:22.584161  8756 raft_consensus.cc:740] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:22.421242 16431 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:22.542140 16357 consensus_meta.cc:287] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 1.117s	user 0.009s	sys 0.008s
I20260709 13:05:22.562893  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:22.590668  8748 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:22.557397 16374 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:22.443714 16356 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:22.513950 15926 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.450766 16423 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.575479  8722 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:22.582768  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 54418.8us
W20260709 13:05:22.550318 16361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.986s	user 0.030s	sys 0.023s
I20260709 13:05:22.513940  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:22.561497 16223 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:05:22.589783  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 110759us
I20260709 13:05:22.568409 16385 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:22.564173 16342 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.569712 16440 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:22.645258  8647 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:22.613574  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:22.627203  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51821.5us
I20260709 13:05:22.647377  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:22.572356 16228 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.582177 16008 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:22.584900 16428 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:22.642016  8676 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 22: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:05:22.605636 16361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.041s	user 0.030s	sys 0.023s
W20260709 13:05:22.611672  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.550931 16384 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:22.647460 16399 consensus_meta.cc:287] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.981s	user 0.014s	sys 0.000s
I20260709 13:05:22.641902 16374 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:05:22.778142  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 59283.4us
I20260709 13:05:22.676275 16221 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:22.781409 16432 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 13:05:22.781565 16433 consensus_queue.cc:1059] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:22.781935 16220 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.782588 16288 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:22.649399 16396 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:22.784811 16343 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:22.785252 16252 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:22.790953 16382 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:22.794399 16052 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.795059 16348 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.797250 16431 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:22.800599 16446 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
W20260709 13:05:22.727882 16108 consensus_meta.cc:287] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.955s	user 0.010s	sys 0.005s
I20260709 13:05:22.649663 16440 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:22.804672 16228 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.629707 16436 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.100s
I20260709 13:05:22.805214 16223 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.808045 16384 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:22.808722 16365 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.877812 16365 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.807783 16008 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.811807 16423 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:22.847455 16277 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:22.724849 16234 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.651532 16383 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:22.590669 16392 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:22.643189 16258 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.828011 16449 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.829131 16298 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:22.817973 16399 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.834998 16361 consensus_queue.cc:1059] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:05:22.850380  8722 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:05:22.893826  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 43585.8us
I20260709 13:05:22.648660 16095 consensus_queue.cc:237] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.826227 16342 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:22.830413 16357 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:05:22.883636 16228 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:22.846062 16252 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:22.876636 16223 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.880489 16355 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:22.815060 16356 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.880496 16365 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.866221 16350 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.861197 16052 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.896812  8829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1554) took 1082 ms. Trace:
W20260709 13:05:22.888728 16023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.055s	sys 0.042s
I20260709 13:05:22.897291 16449 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.902806 16008 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.911772 16052 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:22.912076 16449 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:22.899290  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 401398us
I20260709 13:05:22.902405 16252 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:22.901605  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:22.903640  8677 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:22.866097 16453 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:05:22.873003 16108 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:22.922083 16108 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:22.878331 16396 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.895362 16383 raft_consensus.cc:2764] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:05:22.896867 16298 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.907181  8829 rpcz_store.cc:276] 0709 13:05:21.814177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.814338 (+   161us) service_pool.cc:224] Handling call
0709 13:05:21.819719 (+  5381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:21.968678 (+148959us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.968727 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:21.971669 (+  2942us) log.cc:844] Serialized 26 byte log entry
0709 13:05:22.014602 (+ 42933us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:22.014685 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.014795 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:22.896463 (+881668us) raft_consensus.cc:1685] finished
0709 13:05:22.896500 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.896722 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7560,"raft.queue_time_us":34071,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":16754,"thread_start_us":173218,"threads_started":2,"wal-append.queue_time_us":510045}
I20260709 13:05:22.900034 16382 raft_consensus.cc:3055] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:05:22.918045 16023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.055s	sys 0.042s
I20260709 13:05:22.895206 16258 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:22.892707 16234 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:22.902334 16359 raft_consensus.cc:697] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:22.865511 16374 consensus_queue.cc:1048] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:05:22.925767  8722 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:05:22.929370  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.905064 16228 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:22.927934 16456 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:22.954778 16449 consensus_queue.cc:1059] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:22.955509  8722 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:22.955660 16350 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:22.956130 16383 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:22.927676 16382 raft_consensus.cc:740] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:22.960877 16399 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.929064 16453 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:05:22.962858 16277 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:22.971771 16258 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:22.972012  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:22.976368  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:22.980343  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:22.958967 16234 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:22.995867 16459 log.cc:927] Time spent T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 13:05:22.996253 16455 log.cc:927] Time spent T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:05:22.999044  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.001139  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.003682 16348 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:23.004710 16348 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:23.017627 16406 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.993s	user 0.002s	sys 0.008s
I20260709 13:05:23.019126  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:23.020413  8750 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:23.021828  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.022938  8678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1503) took 2061 ms. Trace:
I20260709 13:05:23.023279  8755 raft_consensus.cc:3060] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:23.023434  8678 rpcz_store.cc:276] 0709 13:05:20.961678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.961825 (+   147us) service_pool.cc:224] Handling call
0709 13:05:21.035438 (+ 73613us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:21.362078 (+326640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.362134 (+    56us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:21.373716 (+ 11582us) log.cc:844] Serialized 78 byte log entry
0709 13:05:21.580358 (+206642us) spinlock_profiling.cc:243] Waited 975 us on lock 0x7b48001002d8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:05:21.724614 (+144256us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:05:21.752273 (+ 27659us) log.cc:844] Serialized 14 byte log entry
0709 13:05:21.753309 (+  1036us) log.cc:844] Serialized 14 byte log entry
0709 13:05:21.754030 (+   721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.754142 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:23.016744 (+1262602us) spinlock_profiling.cc:243] Waited 7.01 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:23.022653 (+  5909us) raft_consensus.cc:1685] finished
0709 13:05:23.022680 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.022868 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":74571,"prepare.run_cpu_time_us":151,"prepare.run_wall_time_us":3196,"raft.queue_time_us":52299,"raft.run_cpu_time_us":2079,"raft.run_wall_time_us":496138,"spinlock_wait_cycles":25795584,"thread_start_us":509814,"threads_started":2,"wal-append.queue_time_us":449178}
I20260709 13:05:23.025295 16440 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:23.027987 16298 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.972163 16387 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:22.982306  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:22.983325  8753 raft_consensus.cc:1240] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:22.929101 16008 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 13:05:23.033370  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.050037 16456 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:23.040370  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:23.043357  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 67098.3us
I20260709 13:05:23.040796  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:22.986351 16374 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:23.049348  8754 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:05:23.052378  8753 raft_consensus.cc:2410] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:05:23.050673 16456 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:23.055909  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.059115  8755 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:05:23.059598  8755 pending_rounds.cc:85] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 12
I20260709 13:05:23.059900  8755 pending_rounds.cc:107] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:05:23.060184  8755 raft_consensus.cc:2889] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:23.060563  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:23.062520  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.063235  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.063798  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:22.976058 15880 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:23.064340  8753 raft_consensus.cc:2468] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:23.047734 16298 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.065277  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.071395  8749 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 27.
I20260709 13:05:23.065796  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.029945 16392 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:23.068984  8722 connection.cc:256] RPC call timeout handler was delayed by -1.19181805300002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 13:05:23.064743  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.043028 16383 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:22.989858 16343 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:05:23.076265  9385 raft_consensus.cc:1240] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 23. Current term is 24. Ops: []
I20260709 13:05:23.054560 16374 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.074143  8722 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:23.077736  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:23.066680 16019 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.499s	user 0.018s	sys 0.047s
I20260709 13:05:23.073920  8753 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:05:23.081012  9385 raft_consensus.cc:1240] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:23.065266 16456 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:23.049093 16345 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:23.080892 16457 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:23.084745  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:23.085275 16457 raft_consensus.cc:697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:05:23.086262 16420 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.437s	user 0.002s	sys 0.000s
I20260709 13:05:23.086436 16457 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:23.087262 16348 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:23.087538 16377 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:23.088019 16377 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:23.088902 16377 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:23.089973 16377 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:23.091049  8825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2825) took 1406 ms. Trace:
I20260709 13:05:23.090842 16383 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:23.091280 16228 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:23.091428  8825 rpcz_store.cc:276] 0709 13:05:21.684505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.684668 (+   163us) service_pool.cc:224] Handling call
0709 13:05:21.692547 (+  7879us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:21.696069 (+  3522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.696108 (+    39us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:21.696848 (+   740us) log.cc:844] Serialized 78 byte log entry
0709 13:05:21.887302 (+190454us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:21.961689 (+ 74387us) log.cc:844] Serialized 14 byte log entry
0709 13:05:21.967800 (+  6111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.967903 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:23.015375 (+1047472us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b48000cd158. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e7419b37 00007f32e74199ca 00007f32e751126f 00007f32e7510f37 00007f32e752555c 00007f32e7525508 00007f32e7525488 00007f32e7525443 00007f32e75246ec 00007f32e644ea96
0709 13:05:23.065855 (+ 50480us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b180038e3c0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3da2a6 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df40c522 00007f32df40e7c1
0709 13:05:23.085642 (+ 19787us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:23.090726 (+  5084us) raft_consensus.cc:1685] finished
0709 13:05:23.090752 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.090980 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45224,"prepare.queue_time_us":74419,"prepare.run_wall_time_us":61,"raft.queue_time_us":23663,"raft.run_cpu_time_us":27992,"raft.run_wall_time_us":1320115,"spinlock_wait_cycles":358968192,"thread_start_us":127685,"threads_started":1,"wal-append.queue_time_us":825221}
I20260709 13:05:23.095903  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:23.093519 16228 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.106979 16228 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:23.074795 16359 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:23.110713 16228 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:23.100096 16410 log.cc:927] Time spent T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.189s	user 0.001s	sys 0.010s
I20260709 13:05:23.121116  8675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1502) took 2160 ms. Trace:
I20260709 13:05:23.121426 16095 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.122103 16095 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:23.121584  8675 rpcz_store.cc:276] 0709 13:05:20.960926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.961059 (+   133us) service_pool.cc:224] Handling call
0709 13:05:20.961444 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:20.993386 (+ 31942us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:20.993439 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:21.565251 (+571812us) spinlock_profiling.cc:243] Waited 60.7 ms on lock 0x7b2c000014c8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e0df2a11 00007f32e0e02372 00007f32e2534a01 00007f32e24f943a 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2
0709 13:05:21.565832 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 13:05:21.758105 (+192273us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:21.758149 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.758223 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:23.099171 (+1340948us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:23.120885 (+ 21714us) raft_consensus.cc:1685] finished
0709 13:05:23.120909 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.121053 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31424,"raft.queue_time_us":32059,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":29986,"spinlock_wait_cycles":148598784,"thread_start_us":191120,"threads_started":1,"wal-append.queue_time_us":275843}
I20260709 13:05:23.071529 16382 consensus_queue.cc:260] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:23.079977 16019 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.515s	user 0.018s	sys 0.047s
I20260709 13:05:23.103644 16350 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.105791 16252 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:23.112536 16456 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:23.113193 16052 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:23.113548 16416 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 1.055s	user 0.005s	sys 0.014s
I20260709 13:05:23.113906 16258 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:05:23.116940  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.119166  8756 consensus_queue.cc:260] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:23.149200  8756 raft_consensus.cc:3060] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:23.076578 16298 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:23.080020 16374 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:23.128787  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:23.132097  8826 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:05:23.137166  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:23.119184 16345 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:23.150282  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.175294  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.157228 16252 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:23.160681 16418 log.cc:927] Time spent T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 1.150s	user 0.004s	sys 0.004s
I20260709 13:05:23.163353  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1555) took 1344 ms. Trace:
I20260709 13:05:23.164074  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.164374  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:23.214359  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.213992  8821 rpcz_store.cc:276] 0709 13:05:21.814604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.814759 (+   155us) service_pool.cc:224] Handling call
0709 13:05:21.836703 (+ 21944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.894007 (+ 57304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.158313 (+1264306us) spinlock_profiling.cc:243] Waited 52.1 ms on lock 0x7b540009ced4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:23.158540 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119851904}
I20260709 13:05:23.165727  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:23.169514  9354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.163383  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:23.210400  8830 raft_consensus.cc:2410] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 for term 23 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:05:23.215472  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.218806  8753 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:23.220721  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.222025  8749 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 21: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:23.226454  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:23.228705  8823 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:23.228638  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:23.145884 16008 consensus_queue.cc:1059] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.082s
I20260709 13:05:23.145597 16350 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:23.226848 16239 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.207s	user 0.011s	sys 0.016s
I20260709 13:05:23.232633  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.244396  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.245507  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.247192  8750 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:23.247509  8821 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 27.
I20260709 13:05:23.250017  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.250969  8750 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:05:23.251425 16390 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:23.252159  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.252429 16419 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.253161 16419 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:23.175163 16258 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:23.256207  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.163087 16052 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:23.247203  8823 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:23.261504  8828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60424 (request call id 2824) took 1580 ms. Trace:
I20260709 13:05:23.256501  9386 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:05:23.262176  8828 rpcz_store.cc:276] 0709 13:05:21.681441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.681617 (+   176us) service_pool.cc:224] Handling call
0709 13:05:21.682051 (+   434us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:21.682483 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.682516 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:21.683238 (+   722us) log.cc:844] Serialized 52 byte log entry
0709 13:05:21.825734 (+142496us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:21.825800 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.825918 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:23.112721 (+1286803us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:23.261080 (+148359us) raft_consensus.cc:1685] finished
0709 13:05:23.261116 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.261417 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12115,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":13712,"spinlock_wait_cycles":29214464,"thread_start_us":141984,"threads_started":1,"wal-append.queue_time_us":298762}
W20260709 13:05:23.248960 16239 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.227s	user 0.011s	sys 0.016s
I20260709 13:05:23.268733  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:23.270562  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:23.270607  8681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:23.275066  8675 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:23.275071  8680 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 14.
I20260709 13:05:23.276751  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:23.276995  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:23.277931  8680 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:23.279153  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:23.275380  8681 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:23.273876  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:23.280881  8821 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:23.278882  8680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:23.289767  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.306293  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:23.312469  8822 raft_consensus.cc:1275] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:05:23.314906  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.317626  8674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:23.317762  8828 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:23.280179  8675 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:23.318759  8674 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:23.322510  8824 raft_consensus.cc:1240] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 LEADER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 13:05:23.323599  8680 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 15.
I20260709 13:05:23.327162  8823 raft_consensus.cc:2393] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 15: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:23.330093  8678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:23.331307  8678 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:23.332017  8826 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:23.320143  8675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:23.333643  8675 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:23.281795 16423 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.347885  8681 raft_consensus.cc:2468] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 23.
I20260709 13:05:23.285118 16095 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:23.360384  8826 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
I20260709 13:05:23.361871  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.362942  8826 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 21: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:23.364043  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:23.367072  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.371122  8826 raft_consensus.cc:2393] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 23: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:23.373991  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:23.374933  8627 auto_rebalancer.cc:629] Failed to schedule move for tablet 8e9173dea5bf4398a5e138cb273df07c: Illegal state: Replica e814ed2013244fa6be42623ea5c0c7b9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 27 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:23.372483 16456 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:23.377704  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.379832  8826 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:23.380016  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.385913  8823 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
I20260709 13:05:23.390221  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:23.396950  8821 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
I20260709 13:05:23.409896  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.378780 16390 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.397430  8794 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:23.416638  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.416059 16059 consensus_queue.cc:1048] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:05:23.421746 16059 consensus_queue.cc:1243] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer 0e8c59a1f0394eae8ce33b99f10efadc log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:23.352031 16423 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:23.425381  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:23.427073 16470 log.cc:927] Time spent T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:05:23.412755 16367 consensus_queue.cc:1048] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.015s
I20260709 13:05:23.428196 16059 raft_consensus.cc:2804] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:23.418601 16390 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:23.431420 16469 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:05:23.435038  8821 raft_consensus.cc:2410] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:05:23.412299 15880 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:23.435616  8826 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:05:23.443123  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:23.447431  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.453703  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.445468 16298 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:23.458931  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:23.464506  8566 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7aff04f53d4845a6a4fa491fd2a06145 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:23.464758  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.450042 16059 raft_consensus.cc:697] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:23.466846  9388 raft_consensus.cc:1275] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:23.466846 16059 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:23.468264 16390 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:23.471395 16456 consensus_queue.cc:1048] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:23.474943 16052 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:23.476078  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.476881  8677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 3255) took 1306 ms. Trace:
I20260709 13:05:23.477363  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.479164  8827 raft_consensus.cc:2376] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19. Re-sending same reply.
I20260709 13:05:23.480568  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.477306  8677 rpcz_store.cc:276] 0709 13:05:22.169995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.192557 (+ 22562us) service_pool.cc:224] Handling call
0709 13:05:23.476669 (+1284112us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54001fa254. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:23.476820 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1594243456}
I20260709 13:05:23.481673  8827 raft_consensus.cc:2468] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:23.482698  8826 raft_consensus.cc:2376] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21. Re-sending same reply.
I20260709 13:05:23.483850  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:23.484138  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.482872  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.466749 16350 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:23.490958  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.492794  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.489586  8827 raft_consensus.cc:2393] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 22: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:23.494369  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.501317  8830 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:23.502789  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:23.503558  8830 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:23.503947 16423 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:23.483142 16258 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:23.511876 16419 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:23.515465  8826 raft_consensus.cc:2410] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:05:23.517244  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39024 (request call id 3326) took 1014 ms. Trace:
I20260709 13:05:23.517717 16220 raft_consensus.cc:3055] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:23.518200 16220 raft_consensus.cc:740] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:23.519199 16220 consensus_queue.cc:260] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:23.520632 16220 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:23.526522  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.527534  8826 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 15.
W20260709 13:05:23.527707 16385 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.876s	user 0.013s	sys 0.003s
I20260709 13:05:23.528272 16385 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:23.529156  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:23.529855  8826 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:23.532140  8673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 3251) took 1366 ms. Trace:
I20260709 13:05:23.532593  8673 rpcz_store.cc:276] 0709 13:05:22.166119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.167745 (+  1626us) service_pool.cc:224] Handling call
0709 13:05:23.531901 (+1364156us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54002a99d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:23.532071 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258551680}
I20260709 13:05:23.535040  9355 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:23.535717 16443 log.cc:927] Time spent T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.892s	user 0.008s	sys 0.007s
I20260709 13:05:23.536079 16427 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:23.536724 16427 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.537163 16427 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:23.538062  8830 raft_consensus.cc:2410] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:05:23.517725  8747 rpcz_store.cc:276] 0709 13:05:22.503485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.543914 (+ 40429us) service_pool.cc:224] Handling call
0709 13:05:23.517022 (+973108us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b54002ace54. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:23.517167 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1304502144}
W20260709 13:05:23.536278 16405 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 1.512s	user 0.007s	sys 0.010s
W20260709 13:05:23.542253  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.547384  9386 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34734 (request call id 1302) took 1512 ms. Trace:
I20260709 13:05:23.548080  9386 rpcz_store.cc:276] 0709 13:05:22.035362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.057073 (+ 21711us) service_pool.cc:224] Handling call
0709 13:05:23.547128 (+1490055us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54005b0754. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:23.547295 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2724618624}
I20260709 13:05:23.547576  8826 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:23.551345  8796 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:23.551793 16427 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:23.553243 16258 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:23.553750 16258 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.554040 16365 raft_consensus.cc:3055] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:05:23.554148 16258 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:23.558771 16365 raft_consensus.cc:740] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:23.559760  8827 raft_consensus.cc:1240] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:05:23.560020 16365 consensus_queue.cc:260] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:23.561687 16365 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:23.561836  8827 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:23.563371  8827 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:05:23.557869  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:23.584215  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:23.586122  8827 raft_consensus.cc:1240] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:05:23.589998  8677 raft_consensus.cc:1275] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:23.591303  8681 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:23.592684  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:23.593321  8673 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:23.585225  8672 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 24.
I20260709 13:05:23.558109 16427 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:23.564746  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:23.556777 16374 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:23.601712  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 189970us
I20260709 13:05:23.603540  8830 raft_consensus.cc:1240] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:05:23.603610  9386 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:05:23.604225  8793 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:23.605886  8677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:23.616247  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
W20260709 13:05:23.617516  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.619445  8672 raft_consensus.cc:1240] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:23.620398  8673 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 24.
I20260709 13:05:23.620904  8793 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:23.623358  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:23.578111 16258 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:23.627719  8793 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:23.631843  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:23.623997 16052 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:23.635599 16052 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:23.636144 16052 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.636616 16052 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:23.621806 16453 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:05:23.652562 16423 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:23.653318 16423 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.653769 16423 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:23.661306  8752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1447) took 1183 ms. Trace:
I20260709 13:05:23.662027  8752 rpcz_store.cc:276] 0709 13:05:22.478053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.478191 (+   138us) service_pool.cc:224] Handling call
0709 13:05:22.478520 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:22.522880 (+ 44360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:22.522918 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:22.523455 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:05:22.904168 (+380713us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:05:23.036767 (+132599us) log.cc:844] Serialized 14 byte log entry
0709 13:05:23.043537 (+  6770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.660603 (+617066us) spinlock_profiling.cc:243] Waited 43.9 ms on lock 0x7b54002fdfd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:23.660687 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:23.660769 (+    82us) raft_consensus.cc:1685] finished
0709 13:05:23.660943 (+   174us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.661205 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13396,"prepare.queue_time_us":132542,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":16189,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":14019,"spinlock_wait_cycles":101005696,"thread_start_us":464320,"threads_started":2,"wal-append.queue_time_us":360333}
I20260709 13:05:23.670538  8673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc"
I20260709 13:05:23.671384  8673 raft_consensus.cc:3060] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:05:23.671794  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 274436us
I20260709 13:05:23.677300  8672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" is_pre_election: true
I20260709 13:05:23.679406  8822 raft_consensus.cc:2393] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 16: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:23.676648  8794 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:23.629384 15880 raft_consensus.cc:2804] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:23.680668  8794 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:23.691329 16350 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:23.691902  8679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 3254) took 1522 ms. Trace:
I20260709 13:05:23.692433  8679 rpcz_store.cc:276] 0709 13:05:22.169785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.169924 (+   139us) service_pool.cc:224] Handling call
0709 13:05:23.691673 (+1521749us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b5400339fd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:23.691825 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514403712}
W20260709 13:05:23.699637 16468 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.358s	user 0.003s	sys 0.000s
I20260709 13:05:23.700462 16374 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:23.701519  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.692289 16350 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:23.700568 16258 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:23.710404  8676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53342 (request call id 3250) took 1544 ms (client timeout 1928 ms). Trace:
W20260709 13:05:23.710942  8676 rpcz_store.cc:269] 0709 13:05:22.165804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.165943 (+   139us) service_pool.cc:224] Handling call
0709 13:05:23.710204 (+1544261us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b5400299354. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:23.710349 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1090118528}
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 6
W20260709 13:05:23.723465  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.754307  8753 pending_rounds.cc:85] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 13
I20260709 13:05:23.754837  8753 pending_rounds.cc:107] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:05:23.755124  8753 raft_consensus.cc:2889] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:23.708344 16350 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:23.726332  8673 raft_consensus.cc:2468] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 16.
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:05:23.761287  7526 tablet_server.cc:179] TabletServer@127.7.89.129:0 shutting down...
W20260709 13:05:23.764657  8627 auto_rebalancer.cc:629] Failed to schedule move for tablet 53cce7f107584cc28999319a785023ef: Illegal state: Replica e814ed2013244fa6be42623ea5c0c7b9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:23.714486 16258 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:23.822857 16477 raft_consensus.cc:2804] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:05:23.868455  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 236699us
I20260709 13:05:23.871207  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:23.870777  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:23.906520  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 223624us
I20260709 13:05:23.907788  8794 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:23.918531  9355 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.773372 16258 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:23.944211  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.944680 16052 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:23.948974  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 78317us
W20260709 13:05:23.905102 16457 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.018s	sys 0.028s
W20260709 13:05:23.953224 16457 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.019s	sys 0.028s
I20260709 13:05:23.866056 16477 raft_consensus.cc:697] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:05:23.964402  8627 auto_rebalancer.cc:629] Failed to schedule move for tablet 614067c154284cfa9b7a01100f304cc9: Illegal state: Replica e814ed2013244fa6be42623ea5c0c7b9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:23.874825 16059 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:05:23.928426 16406 log.cc:927] Time spent T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
W20260709 13:05:23.920374 16345 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.003s	sys 0.024s
I20260709 13:05:23.969369  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55168 (request call id 1561) took 2130 ms. Trace:
I20260709 13:05:23.907768 16343 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:23.967538  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.964795  8793 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:23.999002 16486 consensus_queue.cc:1048] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:24.005683  8824 rpcz_store.cc:276] 0709 13:05:21.838861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.839045 (+   184us) service_pool.cc:224] Handling call
0709 13:05:21.839355 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:23.323216 (+1483861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.968984 (+645768us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54001937d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:23.969285 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3409764608}
I20260709 13:05:24.019610  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:24.025051  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:24.025864  8748 raft_consensus.cc:2376] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 23. Re-sending same reply.
I20260709 13:05:23.936717 16258 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:23.928300 15880 raft_consensus.cc:697] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 23 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:24.059923  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:24.060914  8751 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:24.025965  8752 raft_consensus.cc:2376] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 15. Re-sending same reply.
W20260709 13:05:24.016765 16345 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.004s	sys 0.025s
I20260709 13:05:24.084192  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:24.099002  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:24.109727  8751 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
I20260709 13:05:23.966996 16400 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.109s
I20260709 13:05:24.122347  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:24.127630  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.120386  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 212686us
I20260709 13:05:24.132836  8751 raft_consensus.cc:2393] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 21: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:24.133776  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:24.135614  8794 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:24.130800  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 165979us
W20260709 13:05:24.139213  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.144389  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:24.147264  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:24.148243  8752 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:24.137373  8793 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:24.108244 16410 log.cc:927] Time spent T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
W20260709 13:05:24.157876  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 187594us
W20260709 13:05:24.160998  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.044754 16343 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:24.193789  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:24.141373 16052 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:24.227458  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:24.239871  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:24.240958  8751 raft_consensus.cc:2410] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 13:05:24.248761  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:24.256904 16482 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:05:24.257972 16498 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:24.258657 16498 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.259114 16498 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:24.263113 16478 log.cc:927] Time spent T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 13:05:24.264448 16486 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:24.264966 16486 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:24.265017  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:24.265460 16486 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:24.266883 16343 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:24.267647 16483 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:24.268111 16483 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.268429 16483 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:24.221514 16052 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:24.249863  8751 raft_consensus.cc:2410] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:05:24.274300  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:24.275353  8748 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:24.220714 15880 consensus_queue.cc:237] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:24.279542  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:24.282537  8755 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:24.283469  8755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39032 (request call id 2923) took 1299 ms. Trace:
I20260709 13:05:24.283806  8755 rpcz_store.cc:276] 0709 13:05:22.984155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.281840 (+1297685us) service_pool.cc:224] Handling call
0709 13:05:24.282292 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:24.283144 (+   852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.283396 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:24.284533  9352 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:24.285537 16355 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:24.285957 16355 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:24.286477 16355 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:24.243894 16484 log.cc:927] Time spent T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.244s	user 0.002s	sys 0.000s
I20260709 13:05:24.286607  9352 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:24.249938 16481 raft_consensus.cc:2804] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:24.274447 16052 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.308566  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:24.300524 16479 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:05:24.308660  9352 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:24.329954  9352 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:24.345363  8751 raft_consensus.cc:1240] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 13:05:24.305699 16427 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:24.348768 16427 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:24.349292 16423 consensus_meta.cc:287] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.692s	user 0.014s	sys 0.000s
I20260709 13:05:24.349766 16423 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:24.320912 16197 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:24.325407 16052 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:24.324807 16481 raft_consensus.cc:697] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:24.367475  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:24.341091 16367 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:24.371804  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:24.381491  9385 raft_consensus.cc:2393] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 16: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:24.286660 16434 consensus_queue.cc:1048] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 13:05:24.284761 16496 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.087s
W20260709 13:05:24.399573  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.402225  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.388063  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:24.407270  8751 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:24.356547 16427 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:24.307534 16495 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:24.405366  8755 raft_consensus.cc:1240] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260709 13:05:24.403467  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 368423us
I20260709 13:05:24.372337 16376 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:24.412798  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 52955.8us
I20260709 13:05:24.198789 16477 consensus_queue.cc:237] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:24.396355 16220 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.425016  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:24.390434 16367 raft_consensus.cc:697] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:24.395666 16423 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:24.421170  9352 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:05:24.431689  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.415167 16495 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.450837  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:24.464923  8751 raft_consensus.cc:2410] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 25 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 13:05:24.423702 16197 consensus_queue.cc:1059] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:05:24.462846  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:24.469565  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:24.448207 16496 raft_consensus.cc:2804] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:24.472748  8749 raft_consensus.cc:2393] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:24.444635 16495 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:24.477093  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:24.433087 16481 consensus_queue.cc:237] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:24.430744 16220 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:24.485981  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 348643us
W20260709 13:05:24.484002  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 348438us
I20260709 13:05:24.487241  8794 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:24.488348  8749 raft_consensus.cc:1240] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:24.488437  8793 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:24.494922  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:24.482998 16496 raft_consensus.cc:697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:24.496403 16019 raft_consensus.cc:2804] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:24.493405  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:24.502370 16342 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:24.509805  8794 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:24.488061 16491 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:24.529944  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.494484 16489 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:05:24.535019  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:24.539307  8749 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:24.542882 16476 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:24.536540 16342 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:24.543774 16342 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:24.537611 16019 raft_consensus.cc:697] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 23 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:24.547813 16019 consensus_queue.cc:237] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:24.544723 16355 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:24.539486  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:24.556227  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.558641  8794 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:24.557893  8722 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:24.564122  9385 raft_consensus.cc:1240] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 24. Current term is 25. Ops: []
I20260709 13:05:24.568214  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:24.486291 16367 consensus_queue.cc:237] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:24.562961 16252 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:24.537990 16059 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:24.601126  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:24.603765  8823 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:24.601397  8753 raft_consensus.cc:2468] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:05:24.564580 16491 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:24.605170  8826 raft_consensus.cc:1240] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:24.568902 16239 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:24.631227  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.636125  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:24.645795  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.624732 16491 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:05:24.693073  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 63059.9us
W20260709 13:05:24.695936  8645 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.696s (SENT)
W20260709 13:05:24.700503  8645 leader_election.cc:336] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.730s (SENT)
I20260709 13:05:24.707546  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:24.714318  8645 leader_election.cc:336] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.910s (SENT)
I20260709 13:05:24.719722  8645 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:24.630860 16355 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:24.660223 16377 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.624374 16252 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:24.669668 16496 consensus_queue.cc:237] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:24.751413 16497 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.447s	user 0.002s	sys 0.003s
I20260709 13:05:24.753522 16486 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:24.734202 16491 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:24.758878 16355 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:24.759552 16355 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:24.761929 16355 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:24.750245 16252 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:24.767153 16252 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:24.767948 16252 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.768424 16252 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:24.771147  8573 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43478 (request call id 259) took 30835 ms (client timeout 15000 ms). Trace:
W20260709 13:05:24.739293  8645 leader_election.cc:336] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Timed out: RequestConsensusVote RPC to 127.7.89.130:42321 timed out after 1.804s (SENT)
W20260709 13:05:24.742945 16505 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.061s	user 0.002s	sys 0.000s
I20260709 13:05:24.780372  8756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1459) took 1026 ms. Trace:
I20260709 13:05:24.778628  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:24.780959  8756 rpcz_store.cc:276] 0709 13:05:23.753888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.762451 (+  8563us) service_pool.cc:224] Handling call
0709 13:05:23.762891 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:23.824260 (+ 61369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:23.824301 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:23.866597 (+ 42296us) log.cc:844] Serialized 26 byte log entry
0709 13:05:24.146114 (+279517us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:24.146192 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.146293 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:24.750716 (+604423us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:24.772685 (+ 21969us) raft_consensus.cc:1685] finished
0709 13:05:24.772719 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:24.780271 (+  7552us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60914,"raft.queue_time_us":61846,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":9034,"spinlock_wait_cycles":36359040,"thread_start_us":190968,"threads_started":1,"wal-append.queue_time_us":362871}
I20260709 13:05:24.772548 16377 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:24.796450 16377 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:24.789729 16498 consensus_meta.cc:287] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.530s	user 0.006s	sys 0.004s
I20260709 13:05:24.741463 16345 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:24.741009 16350 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:24.777072 16483 consensus_meta.cc:287] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.002s
I20260709 13:05:24.802227 16483 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:24.806401 16483 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:24.811069 16498 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:24.814111  8822 raft_consensus.cc:1240] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:24.817636 16498 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:24.826925 16486 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:24.812695  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.830124 16499 log.cc:927] Time spent T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
I20260709 13:05:24.791268 16495 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:24.835645 16220 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:24.856695  8823 raft_consensus.cc:2410] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:05:24.837145 16052 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:24.774868 16252 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:24.883482  8824 raft_consensus.cc:1240] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 23 LEADER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 22. Current term is 23. Ops: []
I20260709 13:05:24.838811 16382 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:24.849103  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 109841us
W20260709 13:05:24.879493  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 413535us
I20260709 13:05:24.850131 16343 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:05:24.894522  9353 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.956s	user 0.023s	sys 0.039s
W20260709 13:05:24.895270  9353 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.023s	sys 0.040s
W20260709 13:05:24.895754  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 957790us
I20260709 13:05:24.917371  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:24.918357  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:24.918716  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:24.888311 16220 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:24.918856  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:24.917291  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:24.928340  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:24.928226  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:24.929776  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:24.931208  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 373358us
I20260709 13:05:24.932018  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:24.932757  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:24.935952  8722 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:24.931409  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:24.945432  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:24.952384  9383 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:24.954988 16252 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.955623 16252 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:24.957870 16252 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:24.959271 16396 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:24.960317  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:24.962435  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:24.958359 16252 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:24.948995 16343 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.932243 16220 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:24.974750  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:24.982372  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 380411us
I20260709 13:05:24.972795 16396 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:24.983703  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:24.977931 16239 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:24.972431 16252 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:24.974474 16343 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:24.982004 16350 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:25.010900 16258 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.935s	user 0.016s	sys 0.001s
I20260709 13:05:24.991016 16052 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:25.013177 16434 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.015576 16330 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:25.013378 16377 raft_consensus.cc:493] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:25.017772 16400 raft_consensus.cc:3055] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:25.032437 16400 raft_consensus.cc:740] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:25.013728 16491 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.029979 16434 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.047894 16434 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:25.031917 16350 consensus_queue.cc:1059] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:05:25.051206 16350 raft_consensus.cc:3055] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:25.051731 16350 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:25.052753 16350 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.054267 16350 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:25.053086 16343 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:25.042768 16400 consensus_queue.cc:260] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.056155 16400 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.039438 16342 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.034914 16377 raft_consensus.cc:515] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.059828 16342 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:25.061094 16377 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:25.038880 16345 raft_consensus.cc:2764] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:05:25.064416 16491 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:24.771847  8573 rpcz_store.cc:269] 0709 13:04:53.935355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.017265 (+10081910us) service_pool.cc:224] Handling call
0709 13:05:04.540923 (+523658us) spinlock_profiling.cc:243] Waited 6.51 ms on lock 0x7b54002d7dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2463090 00007f32e2501f3a 00007f32e82b81cc 00007f32e83b637e 00007f32e68772a4 00007f32e6877232 00007f32e6877161 00007f32e68770dc 00007f32e6876392 00007f32e083448c 00007f32e08338c6
0709 13:05:24.770865 (+20229942us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:11.056144 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:11.143815 (+ 87671us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:11.146887 (+  3072us) write_op.cc:435] Acquired schema lock
0709 13:05:11.151603 (+  4716us) tablet.cc:662] Decoding operations
0709 13:05:11.212267 (+ 60664us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:11.212371 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:11.212400 (+    29us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:05:11.228505 (+ 16105us) tablet.cc:701] Row locks acquired
0709 13:05:11.228533 (+    28us) write_op.cc:260] PREPARE: finished
0709 13:05:11.228736 (+   203us) write_op.cc:270] Start()
0709 13:05:11.228897 (+   161us) write_op.cc:276] Timestamp: P: 1783602311228670 usec, L: 0
0709 13:05:11.228923 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:05:11.229874 (+   951us) log.cc:844] Serialized 9786 byte log entry
0709 13:05:11.707745 (+477871us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:12.142942 (+435197us) op_driver.cc:464] REPLICATION: finished
0709 13:05:12.679413 (+536471us) write_op.cc:301] APPLY: starting
0709 13:05:12.682277 (+  2864us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:13.368252 (+685975us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b1800487ba0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e22380ec 00007f32e2284c0a 00007f32e22844f7 00007f32e2284490 00007f32e223bf48 00007f32e5f501bb 00007f32e5f50532 00007f32e5f97b93 00007f32e5e44de3
0709 13:05:15.012197 (+1643945us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:15.063230 (+ 51033us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:16.002962 (+939732us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:16.003675 (+   713us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=0,mrs_lookups=0
0709 13:05:16.003761 (+    86us) write_op.cc:312] APPLY: finished
0709 13:05:16.131345 (+127584us) log.cc:844] Serialized 220 byte log entry
0709 13:05:16.518175 (+386830us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:16.554962 (+ 36787us) write_op.cc:454] Released schema lock
0709 13:05:16.597211 (+ 42249us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:16.739054 (+141843us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:16.740809 (+  1755us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2314,"reported_tablets":50,"spinlock_wait_cycles":14963968,"child_traces":[["op",{"apply.queue_time_us":496301,"apply.run_cpu_time_us":84481,"apply.run_wall_time_us":4038889,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108527,"num_ops":25,"prepare.queue_time_us":380436,"prepare.run_cpu_time_us":32604,"prepare.run_wall_time_us":301326,"raft.queue_time_us":120785,"raft.run_cpu_time_us":77021,"raft.run_wall_time_us":419828,"replication_time_us":913826,"spinlock_wait_cycles":114920576,"thread_start_us":1095237,"threads_started":5,"wal-append.queue_time_us":680402,"wal-append.run_cpu_time_us":23498,"wal-append.run_wall_time_us":3007053}]]}
W20260709 13:05:25.120743  8573 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43482 (request call id 233) took 17571 ms (client timeout 15000 ms). Trace:
W20260709 13:05:25.133816  8794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.018s	sys 0.012s
W20260709 13:05:25.134757  8794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.018s	sys 0.013s
W20260709 13:05:25.135641  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 520609us
I20260709 13:05:25.159224  8754 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
W20260709 13:05:25.172204  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 188412us
W20260709 13:05:25.188149  8754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39032 (request call id 2936) took 2131 ms (client timeout 1910 ms). Trace:
I20260709 13:05:25.188220  8647 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:25.127028  8573 rpcz_store.cc:269] 0709 13:05:07.549703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.119977 (+17570274us) service_pool.cc:211] Skipping call since client already timed out
0709 13:05:25.120370 (+   393us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:05:25.211643  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:25.213721  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:25.225559  9383 raft_consensus.cc:1275] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
I20260709 13:05:25.225610  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:25.226859  9383 pending_rounds.cc:85] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting all ops after (but not including) 14
I20260709 13:05:25.227362  9383 pending_rounds.cc:107] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 13:05:25.227903  9383 raft_consensus.cc:2889] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:25.189178  8754 rpcz_store.cc:269] 0709 13:05:23.056726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.065559 (+  8833us) service_pool.cc:224] Handling call
0709 13:05:25.187638 (+2122079us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b540034b054. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24fd2fa 00007f32e74beb17 00007f32e0a396d4 00007f32e0a39662 00007f32e0a39591 00007f32e0a3950c 00007f32e0a387c2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:25.187953 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4798019328}
I20260709 13:05:25.234329  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.235219  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.235725  8829 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:25.235415  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.236979  8822 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 21: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:25.234411  8824 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:25.239500  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.240389  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.240980  8823 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:05:25.241575 16019 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:25.241600  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:25.242146  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:25.022049 16258 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.242727 16239 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:25.243646  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:25.242722 16095 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.027751 16330 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:05:25.279590  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 66202.1us
I20260709 13:05:25.299299 16019 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.299736 16396 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:25.309151 16019 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:25.316670 16510 sasl_common.cc:323] Time spent creating new SASL server: real 0.258s	user 0.000s	sys 0.002s
I20260709 13:05:25.316510 16330 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.298493  9384 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:25.319543  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:25.321796 16252 leader_election.cc:290] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:25.330166  8826 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:25.344409  8720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.009s	sys 0.025s
I20260709 13:05:25.344555  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:25.352031  8720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.010s	sys 0.025s
I20260709 13:05:25.322403 16382 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:25.359155  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:25.362145  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47496.7us
I20260709 13:05:25.364647  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:25.364446  8566 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.020s	sys 0.003s
I20260709 13:05:25.300670 16239 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.323510 16095 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.362434 16345 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:05:25.376849  8566 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.021s	sys 0.003s
I20260709 13:05:25.375166 16343 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:05:25.379865 16343 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.380407 16343 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:05:25.384364 16239 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:25.385349 16258 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:25.394740  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:25.378993 16345 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:25.402194  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 96938.3us
I20260709 13:05:25.364610 16483 consensus_queue.cc:1048] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.103s
I20260709 13:05:25.415221  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.416005  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.401983 16345 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:25.356163 16396 consensus_queue.cc:1059] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:05:25.429824  8823 raft_consensus.cc:2368] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16.
I20260709 13:05:25.438016  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:25.439240  8827 raft_consensus.cc:2393] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:25.421020 16526 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.440903 16526 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.442919 16512 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:25.444577 16526 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:25.444514 16512 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.445650 16526 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.435360  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.447031 16523 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:25.447721  8829 raft_consensus.cc:2393] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:25.419907  8822 raft_consensus.cc:2368] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20.
W20260709 13:05:25.418706  8721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.010s	sys 0.035s
I20260709 13:05:25.447574 16523 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.376268 16252 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
W20260709 13:05:25.452639  8721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.010s	sys 0.036s
W20260709 13:05:25.449479 16477 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.037s	sys 0.034s
W20260709 13:05:25.453831  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 61002.2us
W20260709 13:05:25.448446  8644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.020s	sys 0.030s
I20260709 13:05:25.435729 16382 consensus_queue.cc:1059] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:05:25.458491 16220 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:25.434437  8824 raft_consensus.cc:2376] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22. Re-sending same reply.
W20260709 13:05:25.465418  8644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.021s	sys 0.030s
I20260709 13:05:25.462747  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:25.449676 16523 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.421617 16330 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:25.449975 16481 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.031s	sys 0.053s
W20260709 13:05:25.464964 16477 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.037s	sys 0.034s
I20260709 13:05:25.435616 16396 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.446486 16526 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.461853 16095 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:25.477634  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.477705 16220 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:25.478114  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:25.478533  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.479628 16519 consensus_queue.cc:1059] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:05:25.484750  8829 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 19. Current term is 20. Ops: []
W20260709 13:05:25.485772  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.486150  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.486836 16512 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:25.507071 16252 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:25.508528 16252 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:25.509516  8822 raft_consensus.cc:2376] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16. Re-sending same reply.
I20260709 13:05:25.488536 16495 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:25.491380 16382 raft_consensus.cc:3055] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:25.490911 16396 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.495345  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:25.472177  8672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.015s	sys 0.023s
W20260709 13:05:25.520790  8672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.016s	sys 0.024s
W20260709 13:05:25.521920  9353 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.522420  8672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1532) took 1812 ms. Trace:
W20260709 13:05:25.471843 16481 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.032s	sys 0.053s
W20260709 13:05:25.507380  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.511246  8826 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:05:25.504086  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.507622  8825 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 19.
I20260709 13:05:25.517256  8827 raft_consensus.cc:2376] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22. Re-sending same reply.
I20260709 13:05:25.498888 16525 raft_consensus.cc:3055] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:25.528838 16525 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:25.530265 16525 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.532357 16525 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:25.537828  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.509828 16252 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.518018 16382 raft_consensus.cc:740] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:05:25.546355  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 75870.4us
I20260709 13:05:25.547371  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.552320 16350 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:25.553503 16350 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.556409 16519 consensus_queue.cc:1059] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:25.557972 16095 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.558739 16095 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.561527 16095 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:25.563464 16382 consensus_queue.cc:260] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.565132 16220 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.565608 16382 raft_consensus.cc:3060] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:25.535840 16483 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:25.581126  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 645927us
I20260709 13:05:25.582677 16396 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:25.584794 16529 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:25.586089 16529 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.589694 16529 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:25.591084  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.593988 16483 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.591580  8827 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:25.524295  8672 rpcz_store.cc:276] 0709 13:05:23.709485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.709638 (+   153us) service_pool.cc:224] Handling call
0709 13:05:23.710040 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.710285 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:05:23.720097 (+  9812us) log.cc:844] Serialized 14 byte log entry
0709 13:05:24.791984 (+1071887us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b1800325060. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e20a14bc 00007f32df3ef48b 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a
0709 13:05:25.521545 (+729561us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:25.521695 (+   150us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:05:25.521846 (+   151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.521979 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:25.522242 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57569,"prepare.queue_time_us":9613,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":75,"raft.queue_time_us":36286,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"spinlock_wait_cycles":3233664,"thread_start_us":957905,"threads_started":2,"wal-append.queue_time_us":536076,"wal-append.run_cpu_time_us":1444,"wal-append.run_wall_time_us":1033206}
I20260709 13:05:25.602928 16252 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:25.627588  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.636036 16350 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:25.640625 16396 consensus_queue.cc:1059] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:25.641923 16396 consensus_queue.cc:1243] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:05:25.566205 16220 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.647040 16519 raft_consensus.cc:493] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:25.648262 16519 raft_consensus.cc:515] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.651619 16519 leader_election.cc:290] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:25.653744 16526 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:25.666147 16522 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.225s	user 0.004s	sys 0.001s
I20260709 13:05:25.666877 16095 raft_consensus.cc:3055] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:25.669364 16258 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:25.670195 16423 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:25.670949 16514 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:25.679068  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 188784us
I20260709 13:05:25.752478 16530 raft_consensus.cc:3055] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:25.753232 16530 raft_consensus.cc:740] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:25.765640 15880 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:05:25.767185  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 96572us
W20260709 13:05:25.771629  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 199989us
I20260709 13:05:25.707839 16095 raft_consensus.cc:740] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:25.742448 16495 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:05:25.779940  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 183859us
I20260709 13:05:25.704465 16350 consensus_queue.cc:1048] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.070s
I20260709 13:05:25.783397  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.790550 16095 consensus_queue.cc:260] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.792393 16095 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:25.809391  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 202689us
I20260709 13:05:25.809571  9355 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:25.810230 16400 consensus_meta.cc:287] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.753s	user 0.011s	sys 0.005s
I20260709 13:05:25.766602 15880 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.788348 16423 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.833087 16400 consensus_queue.cc:1059] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:25.840332 16532 consensus_queue.cc:1059] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:05:25.841346 16234 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:25.841598 16532 raft_consensus.cc:3055] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:25.841847 16234 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:25.777477 16258 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.810526 16220 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:25.853003 16533 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:25.834604 16423 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:25.849079 16532 raft_consensus.cc:740] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:25.858739 16533 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.894965 16532 consensus_queue.cc:260] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.896719 16532 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:25.897774 16533 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:25.867741 16234 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.899958 16536 consensus_queue.cc:1059] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:25.901302 16234 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.901736 16350 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:05:25.872581  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:25.867770  8645 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:25.905257  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:25.905812  8647 leader_election.cc:400] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16.
I20260709 13:05:25.873971 16345 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.908383 16345 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:25.874948 16495 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.911305 16495 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:25.911365 16220 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:25.788883 16514 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.912045  9383 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:25.912317 16220 raft_consensus.cc:697] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:25.849642 16523 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.914870 16220 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.916599 16523 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:25.917449 16523 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:25.884125  8814 tablet_service.cc:1558] Processing DeleteTablet for tablet 364ad2097a2341cdb96752b9a370f9a9 with delete_type TABLET_DATA_TOMBSTONED (TS e814ed2013244fa6be42623ea5c0c7b9 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:37810
W20260709 13:05:25.918825  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 52337.4us
I20260709 13:05:25.892580 16258 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:25.835075 16332 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:25.874449 16343 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.924310 16332 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.924837 16332 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:25.914196 16514 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:25.939829 16533 consensus_queue.cc:1059] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:05:25.941370 16526 raft_consensus.cc:3055] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:05:25.942068  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 85438.6us
I20260709 13:05:25.906366  8647 leader_election.cc:403] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:25.778887 16530 consensus_queue.cc:260] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:25.943876 16526 raft_consensus.cc:740] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:25.947312 16523 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16.
I20260709 13:05:25.947201  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:25.947155 16526 consensus_queue.cc:260] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.948334  8754 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 23.
I20260709 13:05:25.948575 16526 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:25.949577 16525 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.949755  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:25.950639 16525 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:25.951339  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:25.949826  8647 leader_election.cc:400] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20.
I20260709 13:05:25.959523  8647 leader_election.cc:403] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:05:25.960055  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.950683  8748 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:25.966928  8755 raft_consensus.cc:2393] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:05:25.964885  9352 leader_election.cc:336] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:25.968559  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:25.926394 16343 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 28 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:25.963389  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:25.970737  9352 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:25.978873  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:25.985420  8756 raft_consensus.cc:2410] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 13:05:25.987413  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.990185  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:25.977389 16530 raft_consensus.cc:3060] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:26.012192  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:26.007148  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:25.984382 16523 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 20
W20260709 13:05:26.017328  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.015116  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.021162  8824 raft_consensus.cc:2393] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 20: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:26.026437  8647 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:25.989826 16496 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:26.025220  9352 leader_election.cc:336] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.031105  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:26.028860  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:26.026366  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:26.033340  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.045581  8756 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:26.049274  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:26.050941  9352 leader_election.cc:336] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.052434  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:26.052846 16030 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:26.053562 16030 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:26.056578 16030 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:26.063138  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:26.063575 16536 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:26.064334  8825 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:26.064435 16536 raft_consensus.cc:697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:05:26.071961 16517 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.632s	user 0.003s	sys 0.002s
I20260709 13:05:26.031741 16343 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:05:26.082151  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.091315  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:26.091352  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:26.091742  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:26.106119  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.044660  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:26.108553  8752 raft_consensus.cc:2410] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 13:05:26.120283  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 76284.2us
W20260709 13:05:26.121598  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.117312  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:26.117957  8824 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
W20260709 13:05:26.125311  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.114370 16496 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:05:26.126809 16496 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:26.131160  8827 raft_consensus.cc:1240] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:26.131422  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:26.132495  8756 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:26.147264 16483 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:26.066048 16536 consensus_queue.cc:237] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:26.179765  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 46717us
I20260709 13:05:26.180572  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:26.187541  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:26.189000  8750 raft_consensus.cc:1240] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:26.190193  8747 raft_consensus.cc:3060] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:26.191215  8750 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:05:26.192196  8719 leader_election.cc:336] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.192515  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:26.192684  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.199455  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:26.200448  8823 raft_consensus.cc:2410] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:05:26.179337 16095 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.201349  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:26.203980  8748 raft_consensus.cc:2468] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:26.204726  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:26.205185  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 82808.2us
I20260709 13:05:26.207132  9389 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 17.
I20260709 13:05:26.187384 16512 raft_consensus.cc:3055] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:26.215380  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:26.215802 16512 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:26.217007 16512 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.207015  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.211058  8755 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:26.226451 16512 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:26.256892 16542 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:26.258217  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:26.209035 16095 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.219321 16332 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:26.210798  9352 leader_election.cc:336] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.228508  8753 raft_consensus.cc:1275] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:05:26.259232  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 67062.2us
W20260709 13:05:26.288780  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 81056.1us
W20260709 13:05:26.222841  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 317673us
I20260709 13:05:26.227331  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9ddea0a887446f910d38f8302e8172" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:26.272789  8748 raft_consensus.cc:1275] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 13:05:26.278355  8747 raft_consensus.cc:2468] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:26.284349 16523 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857) with higher term. Message: Invalid argument: T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20.
I20260709 13:05:26.287354  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:26.227236  8749 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:26.295389  8750 raft_consensus.cc:1275] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 13:05:26.289896  8748 pending_rounds.cc:85] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 11
I20260709 13:05:26.297544  8748 pending_rounds.cc:107] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 13:05:26.297907  8748 raft_consensus.cc:2889] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:26.299039  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:26.300104  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:26.289798  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.298005  8750 pending_rounds.cc:85] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 12
I20260709 13:05:26.305635  8750 pending_rounds.cc:107] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 13:05:26.306578  8750 raft_consensus.cc:2889] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:26.258553 16252 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:05:26.302805  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:26.303241  8719 leader_election.cc:336] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.291828 16396 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:26.321303  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 119559us
W20260709 13:05:26.309376  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.323165  8722 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:26.323832  8756 raft_consensus.cc:2393] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 24: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:26.305526 16423 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.329378  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:26.330211 16526 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:26.290478 16529 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:26.338789  8747 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:26.340303 16095 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:26.305015 16542 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:26.323890  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:26.330693  8796 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:26.322996  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.347199 16543 tablet_replica.cc:333] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:05:26.348852 16423 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.351455  8748 pending_rounds.cc:85] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 11
I20260709 13:05:26.371636  8748 pending_rounds.cc:107] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:05:26.372080  8748 raft_consensus.cc:2889] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:26.324304 16496 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:26.375344  8747 raft_consensus.cc:2468] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 16.
I20260709 13:05:26.353108 16332 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:26.355000 16530 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:26.354887  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:26.383872  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 271937us
I20260709 13:05:26.384429  8823 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:26.324950 16396 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.356057  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:26.356184 16252 consensus_queue.cc:1059] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:05:26.343179 16526 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:26.365087  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 67695.5us
I20260709 13:05:26.357164  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:26.367738  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.368854  8755 raft_consensus.cc:2468] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 22.
I20260709 13:05:26.324739 16343 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:26.352869  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b50b38d6e5314ca190c9a14a5e0f655d" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:26.382951  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 93688.2us
I20260709 13:05:26.391294  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:26.401196  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:26.402000  8755 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 17.
I20260709 13:05:26.403429  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:26.403650 16052 raft_consensus.cc:2764] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:05:26.404215  8756 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 24.
I20260709 13:05:26.384068 16530 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.408715  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:26.413204  8749 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:26.395157 16526 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:26.421788  8749 raft_consensus.cc:1275] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:05:26.415274 16345 raft_consensus.cc:3055] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:26.424371  8749 pending_rounds.cc:85] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 14
I20260709 13:05:26.415517  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:26.428574  8749 pending_rounds.cc:107] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:05:26.428947  8749 raft_consensus.cc:2889] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:26.416059 16343 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.417732  8722 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.410766 16530 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:26.419291 16542 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:26.411020  8719 leader_election.cc:336] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.423834  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:26.424144  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:26.441468  8755 raft_consensus.cc:2410] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
W20260709 13:05:26.424782  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.424782  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:26.429003  9385 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:26.454667  9353 leader_election.cc:304] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.457600 16507 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:26.436707  8753 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 27.
W20260709 13:05:26.465716  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 48106.2us
I20260709 13:05:26.466967  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.468214  8647 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:26.459439 16512 raft_consensus.cc:2749] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:26.427989 16345 raft_consensus.cc:740] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:26.461472  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.479485  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:26.481930  8754 raft_consensus.cc:2393] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 20: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:26.475911  8722 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:26.483222  8722 leader_election.cc:304] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.484128  8756 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:26.491391 16519 raft_consensus.cc:2749] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:26.435191 16350 consensus_queue.cc:1059] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:05:26.502830 16544 raft_consensus.cc:3055] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:05:26.503438 16544 raft_consensus.cc:740] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:26.503664 16345 consensus_queue.cc:260] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.505024 16345 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:26.505210  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 94247.1us
I20260709 13:05:26.504813 16544 consensus_queue.cc:260] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.507490 16544 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:26.504745  8827 raft_consensus.cc:1240] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:26.444871 16549 consensus_queue.cc:1059] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 13:05:26.511417 16549 raft_consensus.cc:3055] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:26.511824 16549 raft_consensus.cc:740] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:26.465387 16252 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.513862 16252 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.514933 16533 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:26.515420 16533 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.515813 16533 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:26.518462  8796 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:26.519675 16498 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:26.520172 16498 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:26.520582 16498 raft_consensus.cc:3060] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:26.472056 16477 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.521538  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.521762 16477 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.522589 16532 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:26.523394  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 49430.5us
I20260709 13:05:26.523552 16514 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:26.524142 16514 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.524346 16477 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:26.524606 16514 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:26.526868 16423 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:26.527186  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:26.527503 16548 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:26.527992  8826 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:26.477970 16095 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:26.530359  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:26.533012  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.506142  8719 leader_election.cc:336] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.534873  9353 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.539827  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:26.534938  8719 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:26.512820 16549 consensus_queue.cc:260] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.525460  8794 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:26.545109  8747 raft_consensus.cc:2468] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 25.
I20260709 13:05:26.539839  8823 raft_consensus.cc:2468] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:05:26.546952  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:26.546849 16519 consensus_queue.cc:1059] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:26.547876 16095 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:26.549332 16095 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.549782 16095 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:26.551687  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.557057  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.477886 16536 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:26.555917  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:26.563846  8751 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
W20260709 13:05:26.592907 16554 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
W20260709 13:05:26.598438  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.598977 16536 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:26.600225  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:26.600538 16536 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.601639 16350 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:26.602519 16350 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:05:26.602923  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 170004us
I20260709 13:05:26.603063 16534 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:26.603888  8645 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.604832 16548 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:26.606894 16549 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:26.547719 16498 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:26.612040  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.612995 16498 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:26.614962  8747 raft_consensus.cc:1275] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 13:05:26.618377  8747 pending_rounds.cc:85] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 15
W20260709 13:05:26.618891  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 47610.4us
I20260709 13:05:26.619844 16052 consensus_queue.cc:1059] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:26.624763  8747 pending_rounds.cc:107] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 13:05:26.625397  8747 raft_consensus.cc:2889] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:26.606304  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.640677 16533 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:26.642019  8794 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:26.644893 16483 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 13:05:26.649991  8751 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:26.583460 16423 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:26.651693  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:26.654597  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.674932  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 50989.2us
I20260709 13:05:26.677596 16498 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:26.678686 16498 raft_consensus.cc:697] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:26.571684 16514 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.691339  8794 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:26.696159  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 89942.8us
W20260709 13:05:26.701007  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 49430.4us
I20260709 13:05:26.711719  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.695819 16350 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:26.722127 16514 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:26.723342 16514 raft_consensus.cc:2804] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:26.700460 16544 raft_consensus.cc:2804] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:05:26.733451  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 227240us
I20260709 13:05:26.680622 16498 consensus_queue.cc:237] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:26.735365  8719 leader_election.cc:336] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.746861  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.723954 16514 raft_consensus.cc:697] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:26.713737 15880 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:26.749279  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.735857 16544 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.730031 16533 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:26.757937  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 46330.1us
I20260709 13:05:26.746299 16350 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:26.762605 16542 raft_consensus.cc:2804] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:26.767195 16477 raft_consensus.cc:3055] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:26.770190 16514 consensus_queue.cc:237] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.725289 16530 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.776973  8647 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:05:26.782240  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 96315.2us
I20260709 13:05:26.759328 16519 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.757619 16544 raft_consensus.cc:3060] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:26.787202  8627 auto_rebalancer.cc:629] Failed to schedule move for tablet 1148edfb8a5a41fdaced6489dfd22b63: Illegal state: Replica 0e9b0850145a4c66bfe983ffdfa93a57 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:26.788370 16525 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:26.776378 16542 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:26.791036 16542 raft_consensus.cc:3060] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:26.793349 16530 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:26.795687 16526 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.798223 16526 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:26.799371 16252 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:26.799985 16252 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.800453 16252 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:26.801975 16548 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:26.804989  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 45951.6us
I20260709 13:05:26.756476 16423 raft_consensus.cc:697] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:26.783270  8645 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.785372 16533 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:26.817095  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.780886 16477 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:26.830161 16548 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:26.830390 16350 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:26.840863 16350 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.841614 16350 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:26.830420  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 68715us
I20260709 13:05:26.844676  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:26.846917  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 111607us
W20260709 13:05:26.848161  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.838267 16396 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:26.850069 16574 consensus_queue.cc:1048] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:26.851121  8794 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:26.854975 16483 consensus_queue.cc:1048] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:05:26.856097  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:26.866588  8823 raft_consensus.cc:3060] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:26.867098  8719 leader_election.cc:336] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.867698  8719 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:26.855559 16477 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.864470  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:26.869423  8824 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:26.847386 16345 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:26.872646 16576 raft_consensus.cc:2804] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:26.824954 16519 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.885679  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:26.886837  8826 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 27.
I20260709 13:05:26.875406 16345 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.874797 16548 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:05:26.896912  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 45867.4us
I20260709 13:05:26.885618 16576 raft_consensus.cc:697] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:26.891989 16396 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:26.904749  8719 leader_election.cc:336] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:26.905236  8719 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:26.902951  8794 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:26.907529 16519 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:26.903033 16477 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:26.909286 16423 consensus_queue.cc:237] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.911680 16534 raft_consensus.cc:3055] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:26.911978 16252 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:26.912179 16534 raft_consensus.cc:740] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:26.913187 16534 consensus_queue.cc:260] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.909714 16548 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:26.916239  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 280686us
I20260709 13:05:26.898933 16345 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:26.923207 16544 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:26.924082 16548 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:26.925801 16544 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:26.927762  9355 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:26.936559  8722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.943099  8823 raft_consensus.cc:2468] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
W20260709 13:05:26.974779  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.976464  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 75542.3us
I20260709 13:05:26.914474 16534 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:26.985910  8794 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:27.020216  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.029212  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.044018  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.911689 16350 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:26.956873 16519 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:27.053658  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 125978us
W20260709 13:05:27.068480  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 85352.9us
I20260709 13:05:27.069528  8794 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:27.073511  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 595922us
W20260709 13:05:27.076733  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.051934 16532 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.050181 16332 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:27.133002  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:27.096154  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 191415us
W20260709 13:05:27.143114  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 73670.8us
I20260709 13:05:27.075920 16483 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:27.073930 16030 raft_consensus.cc:2804] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:05:27.148479  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 223075us
I20260709 13:05:27.148723  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:27.095777 16530 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.094679 16574 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:27.160163 16582 raft_consensus.cc:2804] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:05:27.165149  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.167577 16532 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.160238 16575 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:27.168824 16575 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:27.170171 16532 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:27.171217 16576 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.171783 16575 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:27.175942  8719 leader_election.cc:336] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.176774  8719 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:27.158792 16095 consensus_meta.cc:287] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.608s	user 0.007s	sys 0.009s
I20260709 13:05:27.150044 16483 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:27.186916 16483 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:27.186733 16542 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.166090 16530 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.193140 16530 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:27.174770 16574 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:27.194729 16095 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.197278 16095 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:27.199240  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.201722  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:27.211387  9381 raft_consensus.cc:3060] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:27.184154 16578 log.cc:927] Time spent T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:05:27.211881  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.156749 16544 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:27.214455 16574 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:27.234871  8827 raft_consensus.cc:1275] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:27.264883  8824 raft_consensus.cc:2468] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:27.223604 16512 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:05:27.236752  9352 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 115 similar messages]
I20260709 13:05:27.218387 16542 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:27.284500 16529 raft_consensus.cc:3055] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:27.285221 16529 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:27.289921  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:27.292095  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
W20260709 13:05:27.294312  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 57591.7us
I20260709 13:05:27.297264  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:27.303550  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.309569  8722 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:27.313791  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 169732us
I20260709 13:05:27.311378  8722 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:27.323717  8794 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:27.327633  8794 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:27.330888  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 166353us
W20260709 13:05:27.331881  8719 leader_election.cc:336] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.332409  8719 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:27.330737  8794 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:27.335454  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 240879us
I20260709 13:05:27.318336 16534 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:05:27.267043 16544 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:27.327160 16568 raft_consensus.cc:2804] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:27.329252 16519 raft_consensus.cc:2804] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:27.335283 16515 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:05:27.333470 16512 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.330267 15880 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:27.328282 16582 raft_consensus.cc:697] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 22 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:27.330250 16030 raft_consensus.cc:697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
W20260709 13:05:27.375159  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 159893us
I20260709 13:05:27.376412  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.378548  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.350803 16534 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.286563 16529 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.380623  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:27.387282  8755 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:27.352589 16530 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:05:27.384646  8752 raft_consensus.cc:2410] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 for term 23 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
W20260709 13:05:27.397873  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 66035.5us
W20260709 13:05:27.398959  8719 leader_election.cc:336] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.399500  8719 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:27.400981  8719 leader_election.cc:336] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.401458  8719 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:27.399235 16030 consensus_queue.cc:237] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:27.373443 16515 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.406527  9387 raft_consensus.cc:2468] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:27.372593 16568 raft_consensus.cc:697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:27.410006  8748 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:27.409075  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:27.416581  9352 leader_election.cc:336] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.379427 16519 raft_consensus.cc:697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
W20260709 13:05:27.440872 16477 consensus_meta.cc:287] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.532s	user 0.012s	sys 0.000s
I20260709 13:05:27.444339 16582 consensus_queue.cc:237] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.380704 16534 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:05:27.424638 16543 raft_consensus.cc:2243] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:27.431030 16544 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:27.409380 16252 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:05:27.454126  9352 leader_election.cc:336] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.454730  9352 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:27.462543  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.463402 16571 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:27.464007 16571 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:27.464464 16571 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:27.467072  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:27.468000  9384 raft_consensus.cc:2393] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:27.377383 16512 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.470309 16512 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:27.471015  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.403790 16332 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.408764 16533 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.471915 16533 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.473045 16396 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:27.473589 16396 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:27.436709 16568 consensus_queue.cc:237] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.475986 16396 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:27.410089 16515 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:05:27.440371 16519 consensus_queue.cc:237] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.380889 16514 consensus_queue.cc:1048] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:05:27.471871 16332 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:27.420939  8719 leader_election.cc:336] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.492870  8719 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:27.503038 16586 log.cc:927] Time spent T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.106s	user 0.003s	sys 0.000s
I20260709 13:05:27.505239 16571 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.474339 16533 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:27.512902 16332 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:27.516306 16571 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:27.527938  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.538368  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.544528  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.586923  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 245381us
W20260709 13:05:27.597282  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.488883  8755 raft_consensus.cc:2468] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 24.
I20260709 13:05:27.621299  9381 raft_consensus.cc:2468] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 18.
I20260709 13:05:27.619073  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:27.629552  8796 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:27.632314  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:27.639413  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.641357  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.565659 16533 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:27.590553 16512 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.585580 16544 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.566107 16592 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:27.590508 16529 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:27.627677 16345 consensus_meta.cc:287] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.579s	user 0.013s	sys 0.000s
I20260709 13:05:27.626226 16573 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:27.562237 16483 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:27.646693 16498 raft_consensus.cc:2804] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:27.671831  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:27.678188  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 257227us
I20260709 13:05:27.655388 16549 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.682271  9381 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: [19.15-19.15]
W20260709 13:05:27.682804  9352 leader_election.cc:336] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:27.684939  8826 raft_consensus.cc:1240] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 24. Current term is 25. Ops: []
I20260709 13:05:27.656402 16512 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.659761 16345 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:27.699847  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:27.667058 16544 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:27.705106  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 269720us
I20260709 13:05:27.704810  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:27.700493 16543 raft_consensus.cc:2272] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:27.703661 16542 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.743422 16498 raft_consensus.cc:697] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:27.743745 16542 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.744606 16498 consensus_queue.cc:237] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:27.699098 16576 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:27.747270 16597 log.cc:927] Time spent T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:05:27.747381 16576 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.750312 16576 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:27.711572 15880 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:27.728413  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:27.729254 16345 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:27.732632  9353 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:27.736233 16595 raft_consensus.cc:2764] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:05:27.699815 16599 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.750748 16544 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:27.726866  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:27.755779  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 44221.3us
W20260709 13:05:27.770045  8793 leader_election.cc:336] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.576s (SENT)
I20260709 13:05:27.770546  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:27.746302 16542 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:27.762435  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.698100 16549 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:27.755246 16512 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:27.773313  8719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.778990  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.779791  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.780552  8749 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:05:27.783514  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.783806  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.784768  8755 raft_consensus.cc:2368] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20.
I20260709 13:05:27.785007  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.785688  8748 raft_consensus.cc:2376] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22. Re-sending same reply.
I20260709 13:05:27.787154  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.789186  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:27.790642  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:27.791518  8752 raft_consensus.cc:2410] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:05:27.771970 16515 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.792466  8748 raft_consensus.cc:2410] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
W20260709 13:05:27.792819 16514 raft_consensus.cc:670] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopped
I20260709 13:05:27.794024 16515 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 28 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:27.795110  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:27.795761  8754 raft_consensus.cc:2376] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22. Re-sending same reply.
I20260709 13:05:27.796728 16549 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:27.797361 16544 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.797788  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.798002 16544 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:27.800338 16544 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:27.801229  8722 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:27.802809 16512 raft_consensus.cc:2804] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:27.803392 16512 raft_consensus.cc:697] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:27.804414 16483 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 25 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:27.794137  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:27.805785  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.806757 16599 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.809369 16599 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:27.811317 16557 raft_consensus.cc:2804] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:05:27.812072 16557 raft_consensus.cc:697] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:27.812726 16543 ts_tablet_manager.cc:1916] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:27.813397  8827 raft_consensus.cc:1240] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 7aff04f53d4845a6a4fa491fd2a06145 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:27.813181 16557 consensus_queue.cc:237] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:27.815279 16507 consensus_queue.cc:1048] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:27.816025 16234 consensus_queue.cc:1059] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:05:27.817162 16234 raft_consensus.cc:3055] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:05:27.817533 16234 raft_consensus.cc:740] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:27.819125  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:27.819541  8821 raft_consensus.cc:1240] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:27.820307  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:27.820811  9352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.821614  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:27.821993  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:27.802434 16534 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.822526  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.823061  8823 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 28: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:05:27.823297  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.826740  8755 raft_consensus.cc:2410] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:05:27.830591  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:27.829012  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 96536us
I20260709 13:05:27.833307  8821 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 22 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:27.833546  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.833920  9353 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:27.836272  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:27.764292 16030 consensus_queue.cc:1048] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
W20260709 13:05:27.841275  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 71264.9us
I20260709 13:05:27.838558  8825 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 13:05:27.848116  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:27.850101  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:27.853734 16607 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.102s	user 0.001s	sys 0.004s
I20260709 13:05:27.868660  8755 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:27.856909 16575 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:27.878450  8821 raft_consensus.cc:1240] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:27.836213 16330 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:27.887185 16573 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.887382  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:27.888172  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:27.888332  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:27.888173  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:27.893568  8825 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:27.894906  8828 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:27.892084  8829 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:27.892747  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:27.890805 16496 consensus_queue.cc:1048] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 13:05:27.898787  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:27.899497  8830 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:27.832610 16332 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 13:05:27.898288  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:27.818531 16234 consensus_queue.cc:260] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:27.804476 16512 consensus_queue.cc:237] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:27.904014  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.901075  8825 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:27.905341  8755 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:27.905913  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:27.907984  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:27.909806  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.907423 16534 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 28 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:27.913133  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:27.916035  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:27.917196  8826 raft_consensus.cc:2393] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 22: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:27.920099  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:27.921882  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:27.919124  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:27.924747  8793 leader_election.cc:336] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.669s (SENT)
I20260709 13:05:27.903786 16532 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.922817  8828 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 16.
I20260709 13:05:27.936419  8826 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:05:27.937167  8793 leader_election.cc:336] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Timed out: RequestConsensusVote RPC to 127.7.89.129:43221 timed out after 1.921s (SENT)
I20260709 13:05:27.938139  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:27.943722  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:27.945520 16608 log.cc:927] Time spent T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.215s	user 0.002s	sys 0.001s
I20260709 13:05:27.896714 16573 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:27.946949  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.957144  9385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:27.958138  8826 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:05:27.959451  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:27.960475  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:27.962586  8821 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:27.963711  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:27.961174  8828 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 23.
I20260709 13:05:27.927738 16534 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:27.958268  9384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:27.967046  8825 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:27.967813 16544 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:27.933835 16529 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:27.962939  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:27.979266  9384 raft_consensus.cc:2393] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 18: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:27.935873 16234 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:27.968734  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:27.980737  9389 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:05:27.983600  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.985297  8755 raft_consensus.cc:1240] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 13:05:27.982517  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:27.987112  8794 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:27.988226  8825 raft_consensus.cc:1240] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:27.988608  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.979033 16529 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.000308  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:27.951052 16587 consensus_queue.cc:1059] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:05:27.954869 16532 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:28.003402  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:28.004081  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:28.005702  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:28.006978  8748 raft_consensus.cc:2410] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:05:27.991825 16515 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:28.008414  8755 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 28: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:28.010246  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:27.993906 16614 raft_consensus.cc:2804] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:28.012326  8755 raft_consensus.cc:2468] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 24.
I20260709 13:05:28.001528 16529 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:28.013808  8750 raft_consensus.cc:2393] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:27.981603 16534 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:28.017007 16605 log.cc:927] Time spent T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.269s	user 0.001s	sys 0.002s
I20260709 13:05:28.014518 16549 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:28.038440 16595 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.039134 16595 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:28.041594 16595 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:28.021142  8823 raft_consensus.cc:1240] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20. Ops: [18.14-19.15]
I20260709 13:05:28.022049 16614 raft_consensus.cc:697] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:05:28.022848  8577 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43482 (request call id 228) took 137265 ms (client timeout 15000 ms). Trace:
I20260709 13:05:28.010241 16515 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:28.028309  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:28.004695 16604 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.145s	user 0.010s	sys 0.000s
I20260709 13:05:28.029485  8755 raft_consensus.cc:1240] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:28.031219  8752 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 16.
I20260709 13:05:28.032192  8830 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:28.022928 16563 raft_consensus.cc:3055] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:28.050423  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:28.052749  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:28.054030  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:28.047575  8577 rpcz_store.cc:269] 0709 13:03:10.765432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.826894 (+ 61462us) service_pool.cc:224] Handling call
0709 13:05:28.008588 (+137181694us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2810636,"reported_tablets":46}
I20260709 13:05:28.054114  8748 raft_consensus.cc:2393] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 21: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:28.061614  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:28.059029 16614 consensus_queue.cc:237] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.067323  8750 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:28.051699 16515 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:28.064533  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:28.069741  8755 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:05:28.052255 16563 raft_consensus.cc:740] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:28.051554 16595 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:28.071790  8748 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 13:05:28.074579  8748 pending_rounds.cc:85] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 13
I20260709 13:05:28.075038  8748 pending_rounds.cc:107] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 13:05:28.075366  8748 raft_consensus.cc:2889] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:28.093667  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:28.100517  8793 leader_election.cc:336] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:28.100078  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:28.055786 16549 consensus_queue.cc:1059] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
W20260709 13:05:28.122359  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 48708.7us
I20260709 13:05:28.101558  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:28.101907  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:28.108039  8753 raft_consensus.cc:1240] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:28.101365  8830 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 21.
I20260709 13:05:28.150489  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:28.070487 16532 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:28.154117 16616 consensus_queue.cc:1059] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:05:28.155167 16549 consensus_queue.cc:1243] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 22.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:05:28.148562  8750 raft_consensus.cc:2468] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:28.155884  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:28.170847  8645 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:28.171234  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:28.171289  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:28.173326  8748 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 24.
I20260709 13:05:28.175572  8750 raft_consensus.cc:2393] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 25: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:28.178195 16548 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:28.178453 16345 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:28.179347 16563 consensus_queue.cc:260] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:28.180483 16549 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:28.181084 16563 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:28.183195  8645 leader_election.cc:400] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Vote denied by peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321) with higher term. Message: Invalid argument: T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 19. Current term is 20.
I20260709 13:05:28.185739  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:28.186583  8753 raft_consensus.cc:2468] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:28.187668 16575 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:28.188920  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 88442.2us
W20260709 13:05:28.189764  8793 leader_election.cc:336] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:28.190181  8793 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:28.191045 16599 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:28.192551  9386 raft_consensus.cc:1275] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:05:28.192647  8793 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:28.193367  8793 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:28.194296 16599 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:28.196165  9387 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:28.181035  8755 raft_consensus.cc:2410] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 13:05:28.196707  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:28.197510  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:28.200459 16599 consensus_queue.cc:1048] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:28.203315 16534 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:28.202242 16599 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 13:05:28.150920 16544 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:28.209089  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:28.211796 16606 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.276s	user 0.004s	sys 0.009s
I20260709 13:05:28.212598  9355 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:28.213826  9387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:28.215350 16507 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:28.215904 16507 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.216037 16573 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:28.216377 16507 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:28.217844 16350 raft_consensus.cc:3055] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:28.218266 16350 raft_consensus.cc:740] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:28.219233 16350 consensus_queue.cc:260] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.220604 16350 raft_consensus.cc:3060] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:28.163542 16612 log.cc:927] Time spent T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.251s	user 0.000s	sys 0.002s
I20260709 13:05:28.149698 16595 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:28.224373  8793 leader_election.cc:336] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:28.185410  8645 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:28.234400  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 59338.5us
W20260709 13:05:28.229357  8793 leader_election.cc:336] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:28.245306  9382 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 25.
I20260709 13:05:28.258811  8647 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:28.228389 16534 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.265091  8722 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:28.270856  8722 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:28.266954 16544 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:28.274241  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:28.259147 16532 raft_consensus.cc:3055] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 LEADER]: Stepping down as leader of term 23
W20260709 13:05:28.278355  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 93051.2us
I20260709 13:05:28.258046 16573 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.275484 16542 consensus_queue.cc:1059] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:05:28.282908 16544 consensus_queue.cc:1059] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:28.267669 16549 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:28.274452 16512 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:28.273577 16616 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:28.262391 16548 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 13:05:28.271574 16568 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:28.272800 16534 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:28.279850 16532 raft_consensus.cc:740] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:28.308813 16573 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.311383  8645 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:28.275578 16533 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:05:28.317546  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.310863 16542 consensus_queue.cc:1243] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer e814ed2013244fa6be42623ea5c0c7b9 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:05:28.328016  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.339697  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:28.348909  8823 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 28: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:28.356024  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:28.356446  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:28.357399  8829 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
I20260709 13:05:28.296901 16549 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.359409  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:28.301416 16512 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.377588  8825 raft_consensus.cc:1275] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:05:28.384877  8821 raft_consensus.cc:1275] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:05:28.323532 16595 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:28.346767 16599 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.344995 16533 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.300091 16529 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:28.376403 16512 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:28.362097 16549 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:28.359310 16542 raft_consensus.cc:3055] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:05:28.399477  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.396642 16533 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:28.324261 16568 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:28.418433  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 151722us
I20260709 13:05:28.428674  8647 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:28.400509 16542 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:05:28.435752  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 161610us
I20260709 13:05:28.444041  8722 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:28.447022  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:28.460660  8822 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:05:28.461890  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.396374 16599 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.419554 16532 consensus_queue.cc:260] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.480521 16548 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.481247 16548 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.484411 16345 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:28.485281 16345 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:28.490373 16542 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.492681 16568 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:28.498515  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.502681 16573 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:28.503015 16542 raft_consensus.cc:3060] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:28.510334 16529 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:28.486253 16515 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:28.499212  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 187895us
W20260709 13:05:28.517696 16609 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.762s	user 0.000s	sys 0.007s
I20260709 13:05:28.520057 16568 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:28.517547 16631 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:28.518743 16507 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:28.552516  8645 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:28.541108 16345 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.577940  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:28.545239 16627 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:28.571867 16544 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:28.594419  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.594887 16635 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:28.595939 16635 consensus_queue.cc:1059] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:28.596984 16635 consensus_queue.cc:1243] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer 7aff04f53d4845a6a4fa491fd2a06145 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:05:28.597615 16635 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:05:28.580292 16633 raft_consensus.cc:3055] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:05:28.611281  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 167480us
W20260709 13:05:28.613570  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.584419 16631 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:05:28.618108  8720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.590770 16627 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.621114  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:28.625407  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:28.626274  8747 raft_consensus.cc:3060] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:28.626453  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 133035us
I20260709 13:05:28.627374  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:28.628682  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 76332.9us
I20260709 13:05:28.629364  8645 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:28.632017  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:28.588903 16345 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.611107 16633 raft_consensus.cc:740] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:28.638142  8751 raft_consensus.cc:1275] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:28.610934 16350 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:28.627646 16634 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.644758  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:28.628199 16548 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:28.647504  8756 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:28.630715 16627 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:28.663079  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:28.663414 16632 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:28.664880 16633 consensus_queue.cc:260] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:28.667100 16631 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.655081 16532 raft_consensus.cc:3060] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:28.668916  8570 catalog_manager.cc:5697] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.673254 16573 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:28.675114 16507 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:28.675940  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 46634.9us
I20260709 13:05:28.676812  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:28.679935 16637 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:05:28.680822 16512 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.681675 16616 raft_consensus.cc:3055] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:05:28.682253 16616 raft_consensus.cc:740] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:28.649505  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:28.654027 16515 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:28.655062 16636 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:28.655287  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:28.655594 16634 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:28.658241 16345 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:28.659361 16533 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.720543 16533 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:28.615356 16544 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:28.661607 16543 ts_tablet_manager.cc:1929] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.10
I20260709 13:05:28.683707 16512 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:28.683849  8796 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:28.685119 16633 raft_consensus.cc:3060] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:28.685863 16549 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.687916  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:28.687582  8570 catalog_manager.cc:5697] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.690214  9382 raft_consensus.cc:1240] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:05:28.690572  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:28.751296  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 53903.8us
I20260709 13:05:28.690786  8753 raft_consensus.cc:1275] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:05:28.691210 16548 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:28.695057 16642 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.756978 16642 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.759569 16642 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:28.704360 16350 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:28.715312  8751 raft_consensus.cc:2393] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 28: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:28.761361  8751 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:05:28.735775 16614 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.003s	sys 0.004s
I20260709 13:05:28.705010 16534 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:28.718278 16634 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:28.721396  8748 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:05:28.715925 16636 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.674325 16573 consensus_queue.cc:1048] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.032s
W20260709 13:05:28.641963  8719 leader_election.cc:336] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:28.891064  8719 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:28.766000 16614 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.006s	sys 0.004s
W20260709 13:05:28.884444  8721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:28.883366  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 243592us
I20260709 13:05:28.887485  8824 raft_consensus.cc:1275] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:28.766279 16543 log.cc:1199] T 364ad2097a2341cdb96752b9a370f9a9 P e814ed2013244fa6be42623ea5c0c7b9: Deleting WAL directory at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0/minicluster-data/ts-2-root/wals/364ad2097a2341cdb96752b9a370f9a9
I20260709 13:05:28.877693  8747 raft_consensus.cc:2468] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 25.
W20260709 13:05:28.763090  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 79363.9us
W20260709 13:05:28.902204  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 260229us
W20260709 13:05:28.903141  8719 leader_election.cc:336] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:28.903587  8719 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:28.904469 16533 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:28.885067 16542 consensus_queue.cc:1059] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 13:05:28.883051 16345 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:28.906596 16630 raft_consensus.cc:3055] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:28.907050 16630 raft_consensus.cc:740] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:28.907078 16345 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:28.892517  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 202010us
I20260709 13:05:28.889344  8752 raft_consensus.cc:1275] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
I20260709 13:05:28.882973 16548 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:28.910785  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:28.916409  8747 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
I20260709 13:05:28.888818 16350 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:28.926504  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:28.930917  8751 raft_consensus.cc:1275] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:05:28.931247  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:28.934701  8823 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:28.883234 16654 raft_consensus.cc:2804] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:05:28.884511 16636 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:28.933931  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:28.936316  8752 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:28.908237 16630 consensus_queue.cc:260] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:28.933430  8827 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:28.950526 16630 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:28.938524  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:28.939245  8719 leader_election.cc:336] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:28.939443  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:28.939038  8570 catalog_manager.cc:5697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.940073  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:28.940547 16534 consensus_queue.cc:1048] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:28.940725  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:28.941076 16549 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:28.768083 16616 consensus_queue.cc:260] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.941352  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:28.942137  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:28.942423 16599 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:28.942741 16548 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:28.944571 16234 consensus_queue.cc:1048] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:28.946106 16573 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:28.947091  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:28.947866  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:28.762053 16627 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:28.948612 16632 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:28.942843  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:28.954721  8822 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:28.954523  8719 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:28.956530  8646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.958885  8828 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
W20260709 13:05:28.964943  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55347.3us
I20260709 13:05:28.985504 16345 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:28.985924 16654 raft_consensus.cc:697] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:28.987149 16654 consensus_queue.cc:237] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:28.995615  9389 raft_consensus.cc:1275] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:29.001570  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:28.980980  8722 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:29.025591  8750 raft_consensus.cc:2393] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 17: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
W20260709 13:05:29.036561  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 55675.3us
I20260709 13:05:29.036195  8823 raft_consensus.cc:2468] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 23.
W20260709 13:05:29.042966  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 103724us
W20260709 13:05:29.044662  8719 leader_election.cc:336] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.045301  8719 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:28.979197 16533 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.982285 16543 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37810 (request call id 0) took 3088 ms. Trace:
I20260709 13:05:29.046674  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:29.051518  8752 raft_consensus.cc:2468] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:28.991705 16645 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:29.052749  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 52449.4us
W20260709 13:05:29.070570  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 191309us
I20260709 13:05:29.048797 16529 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:29.049100 16616 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:29.075582  8719 leader_election.cc:336] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:29.077658 16646 log.cc:927] Time spent T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:05:29.091361  8645 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.087421  8570 catalog_manager.cc:5697] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:29.067414 16642 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:05:29.086897  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.056839 16544 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:29.093971  8794 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:29.105551  8755 raft_consensus.cc:1275] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:05:29.069031 16573 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:29.113471  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:29.114215  8719 leader_election.cc:336] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.089635 16529 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.058733 16533 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.114583  8719 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.103330 16634 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:29.119637 16529 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:29.153411 16534 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:29.156409  8719 leader_election.cc:336] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.120612 16635 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:29.183804  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 113720us
W20260709 13:05:29.193388  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 154621us
I20260709 13:05:29.194209  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:29.089933 16645 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:29.186378  8719 leader_election.cc:336] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.096470 16575 consensus_queue.cc:1048] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:05:29.118862 16653 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:05:29.196722  8647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:29.196216  8719 leader_election.cc:336] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.198138  8719 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.198589  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:29.199517  8719 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.199990  8719 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.200433  8645 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.201921  8645 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.191767  8822 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
W20260709 13:05:29.205163  8719 leader_election.cc:336] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.196960  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.205662  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:29.207038  8794 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:29.176100 16634 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:29.210186  8719 leader_election.cc:336] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.210054  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:29.212014  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.210728  8719 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.197453 16653 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.203910 16345 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:29.206030 16573 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:29.179965 16571 consensus_queue.cc:1048] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:05:29.236963 16549 raft_consensus.cc:2804] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:29.234069 16634 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:29.222491 16534 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:29.208819 16632 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:29.235473 16515 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:29.206516 16649 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:05:29.280196 16573 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:29.235011 16630 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:05:29.264190 16549 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.282662 16549 raft_consensus.cc:3060] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:29.240916 16653 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:29.284920 16664 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:29.269361 16655 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:05:29.268990  8570 catalog_manager.cc:5697] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:29.211941 16599 consensus_queue.cc:1048] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:05:29.279176 16663 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:29.262315 16548 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:29.280570 16515 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.055557 16543 rpcz_store.cc:276] 0709 13:05:25.883372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.883743 (+   371us) service_pool.cc:224] Handling call
0709 13:05:26.321305 (+437562us) ts_tablet_manager.cc:1151] Deleting tablet 364ad2097a2341cdb96752b9a370f9a9
0709 13:05:26.342695 (+ 21390us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:05:28.285273 (+1942578us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:28.661428 (+376155us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:28.971501 (+310073us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":968715,"tablet-delete.queue_time_us":380250,"thread_start_us":224461,"threads_started":1}
I20260709 13:05:29.287546  8645 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.315392  8645 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:29.287328  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 78178.6us
I20260709 13:05:29.212824 16544 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:29.236197 16614 consensus_queue.cc:1048] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:05:29.298418 16515 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:29.282387 16533 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:29.319262 16548 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:29.325807 16645 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:29.328691 16350 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:29.299541 16627 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:29.331673  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 119840us
I20260709 13:05:29.336994  8645 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:29.343631  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 133450us
I20260709 13:05:29.348201  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:29.349327  8822 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:29.349828 16533 consensus_queue.cc:1048] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:29.350095  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:29.350769 16532 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:05:29.345093 16657 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.253s	user 0.000s	sys 0.003s
I20260709 13:05:29.353122  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:29.355372  8829 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 22. Current term is 23. Ops: [20.14-22.15]
W20260709 13:05:29.355722  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 364212us
W20260709 13:05:29.356549  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:29.357439  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:29.361418  8644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:29.340155 16661 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.106s	user 0.003s	sys 0.000s
I20260709 13:05:29.368932  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:29.367971  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:29.340214 16630 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:29.375638  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:29.385512  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:29.386569  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:29.371428 16632 raft_consensus.cc:3060] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:29.399065 16642 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:29.399049 16645 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.402190 16668 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:29.398950  8570 catalog_manager.cc:5697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:29.402833 16668 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.403306 16636 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.402782 16645 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.403707 16631 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.406734  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:29.388877 16544 raft_consensus.cc:2804] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:29.411908  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:29.414950  8752 raft_consensus.cc:2393] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 20: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:29.412801  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:29.376930 16648 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:29.368086 16548 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:05:29.384713 16350 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:05:29.404663 16533 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.404435  8570 catalog_manager.cc:5697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 11 to 16, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 16 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:29.413285 16633 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.410763 16642 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.426192  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:29.427773 16630 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.441569 16533 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.445724  8749 raft_consensus.cc:2468] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:29.446528  8755 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:29.409677 16542 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.453913  8747 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:29.443288 16633 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.426265 16544 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:29.461714  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.437386 16644 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:29.472711  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:29.474324  9383 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:29.457549 16648 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.425661 16631 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.473840 16544 raft_consensus.cc:3060] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:29.488119 16345 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:29.501332 16645 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:29.501860 16668 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:29.502334  8755 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 25.
I20260709 13:05:29.503253 16631 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:29.503185 16633 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:29.504590 16680 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.504774 16635 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:29.506189  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.505695  8570 catalog_manager.cc:5697] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 7 to 17. New cstate: current_term: 17 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:29.527454  8570 catalog_manager.cc:5697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 11 to 13, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 13 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:29.529712 16653 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.498222 16350 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:29.509092 16549 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:29.511240 16656 log.cc:927] Time spent T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.326s	user 0.002s	sys 0.000s
I20260709 13:05:29.515514  8722 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:29.547863  8722 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:29.514410 16665 log.cc:927] Time spent T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:05:29.478294 16542 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:29.560415  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 54292.1us
I20260709 13:05:29.520834  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:29.521329  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:29.479737 16630 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:29.565078 16631 consensus_queue.cc:1059] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:05:29.565757 16670 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.220s	user 0.002s	sys 0.000s
I20260709 13:05:29.566136  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:29.566850 16630 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:29.560506 16549 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:29.566521 16350 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:29.570144 16515 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.576891  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:29.577122 16515 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:29.577409  8794 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:29.577633  8827 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:29.561892 16680 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.578991 16651 raft_consensus.cc:2804] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:29.524371  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:29.486968 16648 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.531934  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:29.532984 16659 log.cc:927] Time spent T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.213s	user 0.000s	sys 0.005s
I20260709 13:05:29.533277 16636 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:29.534832 16642 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:29.539927  8566 catalog_manager.cc:5028] TS e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): tablet 364ad2097a2341cdb96752b9a370f9a9 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:05:29.506915 16533 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:29.562997 16568 raft_consensus.cc:2804] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:29.563400 16645 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:29.564643 16542 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:29.580843  9383 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:29.583048  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:29.616926  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:29.584544  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:29.618131  8824 raft_consensus.cc:3060] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:29.581508  8828 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
I20260709 13:05:29.587967  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:29.598150  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 45477us
I20260709 13:05:29.599047  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:29.566144 16653 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:29.587719 16651 raft_consensus.cc:697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
W20260709 13:05:29.637252  8566 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 97348.4us
I20260709 13:05:29.615368  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:29.570071 16634 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.632216  8753 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:29.615368  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:29.640677  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:29.596863 16345 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:29.663091  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:29.668582  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 78450.7us
I20260709 13:05:29.680302  8751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35702 (request call id 1480) took 1054 ms. Trace:
I20260709 13:05:29.638530 16645 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:29.644002 16568 raft_consensus.cc:697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:29.687285  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:29.703347  8645 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:29.709134  8627 auto_rebalancer.cc:629] Failed to schedule move for tablet 9c86a8d1b46a4dad9149c50458be45d2: Illegal state: Replica 0e9b0850145a4c66bfe983ffdfa93a57 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:29.681190  8751 rpcz_store.cc:276] 0709 13:05:28.625652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.982478 (+356826us) service_pool.cc:224] Handling call
0709 13:05:29.015533 (+ 33055us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.015846 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:29.056793 (+ 40947us) log.cc:844] Serialized 14 byte log entry
0709 13:05:29.606161 (+549368us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b1800286300. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df3eee07 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df40bc02 00007f32e209320b 00007f32e2098d23 00007f32e209923a 00007f32e2503a2a
0709 13:05:29.679916 (+ 73755us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:29.679973 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:29.680022 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.680088 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.680228 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5323,"prepare.queue_time_us":47625,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"spinlock_wait_cycles":9236736,"thread_start_us":68023,"threads_started":1}
I20260709 13:05:29.710834  8824 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:29.716482  8754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3385) took 1085 ms. Trace:
I20260709 13:05:29.720983  8645 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:29.722776  8645 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.717111  8754 rpcz_store.cc:276] 0709 13:05:28.620031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.620202 (+   171us) service_pool.cc:224] Handling call
0709 13:05:28.636553 (+ 16351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:28.721176 (+ 84623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:28.721212 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:28.721808 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 13:05:29.046136 (+324328us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:29.046214 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.046343 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.496254 (+449911us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b4400022b4c. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df332c85 00007f32df3336a2 00007f32e20a51db 00007f32e20a0aaf 00007f32e209486e 00007f32e2093bb6 00007f32e20937d8 00007f32e209d221 00007f32e209d1d9 00007f32e209d169
0709 13:05:29.695697 (+199443us) raft_consensus.cc:1685] finished
0709 13:05:29.695733 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.704931 (+  9198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37216,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":67163392,"thread_start_us":335370,"threads_started":2,"wal-append.queue_time_us":492688}
I20260709 13:05:29.732730  8645 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.741055  8645 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.704854 16681 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:29.709411 16542 raft_consensus.cc:2804] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:29.715344 16548 raft_consensus.cc:3060] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:29.753012 16549 raft_consensus.cc:2804] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:29.785465 16549 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.786136 16549 raft_consensus.cc:3060] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:29.771457  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:29.731315 16679 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:29.768463 16542 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.792032  8824 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 22: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:29.734045 16532 raft_consensus.cc:3060] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:29.797627  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 90006.4us
I20260709 13:05:29.748718 16616 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:05:29.806017  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:29.830211  8754 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
I20260709 13:05:29.807150 16632 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:29.813216 16681 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:29.793860 16680 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:29.749271 16350 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 13:05:29.798933 16542 raft_consensus.cc:3060] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:29.833675  9353 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.842911  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:29.848417  8627 auto_rebalancer.cc:629] Failed to schedule move for tablet ae08d8f5271e475baa4344843405b254: Illegal state: Replica e814ed2013244fa6be42623ea5c0c7b9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:29.849328  8645 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:29.847668 16681 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:29.805373 16568 consensus_queue.cc:237] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.801172 16645 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:29.867237  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:29.880597  8719 leader_election.cc:336] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.881140  8719 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:29.848302 16688 consensus_queue.cc:1048] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 13:05:29.883764  8753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3395) took 1260 ms. Trace:
W20260709 13:05:29.881904  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 48373.1us
W20260709 13:05:29.886304  8719 consensus_peers.cc:597] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:29.871553 16682 log.cc:927] Time spent T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:05:29.884310  8753 rpcz_store.cc:276] 0709 13:05:28.623576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.755649 (+132073us) service_pool.cc:224] Handling call
0709 13:05:29.619497 (+863848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.643484 (+ 23987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.883528 (+240044us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b54002ac450. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f88ad 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6 00007f32e083326f
0709 13:05:29.883692 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1985584896}
W20260709 13:05:29.848419 16529 consensus_meta.cc:287] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.641s	user 0.007s	sys 0.007s
I20260709 13:05:29.889305  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:29.894826  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:29.843478 16616 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
W20260709 13:05:29.896818  8719 leader_election.cc:336] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.880055 16631 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:29.901573 16681 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:29.797019 16651 consensus_queue.cc:237] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.887297 16534 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:29.919703 16632 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.920459 16632 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:29.923062  8749 raft_consensus.cc:1240] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 13:05:29.909013  8719 leader_election.cc:336] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:29.927552  8719 leader_election.cc:336] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.929481  8719 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:29.924379 16632 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:29.934166 16534 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:29.937976  8754 raft_consensus.cc:1240] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:05:29.907181 16631 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:29.947504  8719 leader_election.cc:336] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.929162 16688 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:05:29.950124  8719 leader_election.cc:336] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:29.954737  8719 leader_election.cc:336] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:29.958524  8719 leader_election.cc:336] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:29.962136  7526 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 8706)
I20260709 13:05:29.973881  8722 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:29.975775  8722 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:29.977435 16645 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:29.977919 16645 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.978296 16645 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:29.979018  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:29.979278 16534 raft_consensus.cc:2804] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:29.979974 16534 raft_consensus.cc:697] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:29.980549  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:29.982753  9381 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:05:29.905325 16529 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:29.985620  8793 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.948649 16631 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:29.988551  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:29.993640  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 106102us
I20260709 13:05:29.947855 16345 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:29.996318  8719 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:29.955729 16688 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:30.007644 16345 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:30.009083  8719 leader_election.cc:336] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.009616  8719 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:30.013229  8719 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.013707  8719 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.022359  8751 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 22: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:29.997238 16681 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:30.028395  8752 raft_consensus.cc:1275] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 13:05:30.029353  8719 leader_election.cc:336] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:30.030463  8719 leader_election.cc:336] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.030908  8719 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.031621  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:30.032532  8749 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:30.040444 16664 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:30.008538 16688 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:30.043571  8719 leader_election.cc:336] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.053959  8719 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.055985  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:30.057893  8719 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:30.032961 16544 consensus_meta.cc:287] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.003s
I20260709 13:05:29.999766 16571 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:05:30.061081 16533 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:30.020757 16573 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:29.981000 16534 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:30.088145 16700 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:05:30.008351 16345 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:29.993387 16616 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:30.064285 16627 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:05:30.130462  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:30.131031  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:30.132023  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:30.137122  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 129118us
W20260709 13:05:30.138288  8793 leader_election.cc:336] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.106405 16681 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.135079  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:30.114140 16632 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:30.152541  8793 leader_election.cc:336] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.133356 16679 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.132601 16630 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:30.162926  8793 leader_election.cc:336] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.158936  8749 raft_consensus.cc:2468] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:30.092317 16544 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:30.167097  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:30.175345  8793 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.171497  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:30.175943  8793 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:30.186571 16705 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:05:30.164381 16681 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:30.189004 16706 log.cc:927] Time spent T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:05:30.189075 16681 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:30.192723 16544 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:30.193468 16544 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:30.192721 16681 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.194458 16681 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:30.177320 16630 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:30.196615 16702 log.cc:927] Time spent T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:05:30.197100 16630 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:30.197204 16548 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:05:30.186452 16703 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 13:05:30.201122 16684 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.425s	user 0.004s	sys 0.000s
I20260709 13:05:30.203887  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:30.205178  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:30.205953  8756 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 21: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:30.167469 16632 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:30.208101 16571 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:30.208581  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:30.208849 16632 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:30.209129 16708 log.cc:927] Time spent T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 13:05:30.226959  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 51650.6us
I20260709 13:05:30.186691 16529 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:30.154709 16549 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:30.239735  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:30.228540  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:30.247839 16632 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.194103 16544 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:30.255157  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:30.270596  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:30.272629  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:30.276467  8793 leader_election.cc:336] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.279624  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:30.286569  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:30.287705  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:30.288409  8827 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:30.288504  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:30.288738  8824 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:30.289955  8830 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 22. Current term is 23. Ops: [20.14-22.15]
I20260709 13:05:30.275120  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:30.296167  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:30.296882  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:30.301928  8793 leader_election.cc:336] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.302536  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:30.305011  8793 leader_election.cc:336] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.297474 16719 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:30.307529 16676 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:30.307960  8753 raft_consensus.cc:1240] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 LEADER]: Rejecting Update request from peer 0e8c59a1f0394eae8ce33b99f10efadc for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:05:30.309052 16630 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.309700 16630 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:30.306504  8793 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:30.273737 16632 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:30.295900 16515 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:30.295482 16548 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:30.169869 16679 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:30.307715 16719 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:30.304239 16532 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:30.325809  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:30.327152  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:30.331070 16549 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:30.331811 16666 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:30.334355 16544 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:30.328099  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:30.314296 16345 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:30.337843  8793 leader_election.cc:336] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:30.346247  8793 leader_election.cc:336] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.325613 16614 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:05:30.347232  8793 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.349092  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:30.351166  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:30.355131  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:30.347669  8793 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:30.362730 16719 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:30.363466 16651 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:30.368076 16679 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:30.361001  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:30.316952 16542 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:30.372665  8793 leader_election.cc:336] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.373370  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:30.345755 16549 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:05:30.381964  8793 leader_election.cc:336] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.382887  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:30.383162  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:30.383359  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:30.384490  8793 leader_election.cc:336] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.360575 16571 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
W20260709 13:05:30.385404  8793 leader_election.cc:336] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:30.391634  8793 leader_election.cc:336] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.392678  8793 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:30.394673  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:30.395262  8793 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.395624  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:30.396323 16515 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:05:30.396683 16630 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:30.396878  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:30.386197 16651 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:30.398080  8793 leader_election.cc:336] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:30.399334  8793 leader_election.cc:336] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.399701  8793 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:30.401093  8793 leader_election.cc:336] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.401475  8793 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:30.386368 16549 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:30.403157  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:30.404500 16614 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:30.404755  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:30.408066 16549 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:30.408272 16534 raft_consensus.cc:493] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:30.408823 16635 raft_consensus.cc:3055] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:30.409443 16635 raft_consensus.cc:740] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:30.410459 16635 consensus_queue.cc:260] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:30.403103 16515 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:05:30.411634 16635 raft_consensus.cc:3060] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:30.412319  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:30.413625 16542 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:30.397819 16052 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:30.398136 16681 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:30.408965 16534 raft_consensus.cc:515] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:30.417407 16534 leader_election.cc:290] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:30.419418 16632 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:30.419831  9386 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:05:30.421222  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:30.410490  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:30.424847  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:30.425731  8822 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:30.429453  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:30.433636 16568 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:30.444793 16634 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:30.445230 16720 log.cc:927] Time spent T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:05:30.392333 16571 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:30.448688 16571 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:30.449035  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:30.427316 16631 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:30.449934  8822 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 22: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:30.451731  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295d39c61dd3489e98d9393a2da9c062" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:30.452095 16631 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:30.452966  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:30.453608  8821 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:30.454226  8645 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.462229  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.469956  8822 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:05:30.475963  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:30.475176  8722 leader_election.cc:304] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.468138 16631 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:30.481349  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:30.489327  8824 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:30.487668 16632 consensus_queue.cc:1059] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:05:30.467929 16635 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:30.490298 16681 consensus_queue.cc:1059] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:30.490981 16632 raft_consensus.cc:3055] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:30.491194 16549 raft_consensus.cc:3055] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:05:30.491400 16632 raft_consensus.cc:740] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:30.491636 16549 raft_consensus.cc:740] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:30.492322 16632 consensus_queue.cc:260] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:30.494225 16632 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:30.495169 16573 raft_consensus.cc:2749] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:30.487406  8719 leader_election.cc:336] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.492488 16549 consensus_queue.cc:260] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:30.492231 16645 consensus_meta.cc:287] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.000s
W20260709 13:05:30.498898  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:30.485625  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:30.500852 16529 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:05:30.500852  8719 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.511543  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.510409  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:30.525563 16648 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:30.526723  8824 raft_consensus.cc:2468] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:30.527631  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:30.478626 16350 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:05:30.534374  8722 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:30.547504  8794 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:30.549387  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 48574us
W20260709 13:05:30.550290  8719 leader_election.cc:336] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.551350  8719 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:30.552594  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:30.557142  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:30.558919  8719 leader_election.cc:336] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.510403 16681 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:05:30.557848  8793 consensus_peers.cc:597] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:30.561712  9382 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 22: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:30.530184 16542 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:05:30.568077  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:30.564024 16483 raft_consensus.cc:2804] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:30.549818 16549 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:30.573136  8719 leader_election.cc:336] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.581010  8821 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
W20260709 13:05:30.588550  8719 leader_election.cc:336] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.589044  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:30.590378  8719 leader_election.cc:336] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:30.591630  8719 leader_election.cc:336] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.592041  8719 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.557052 16664 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:05:30.593703  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:30.599042  8719 leader_election.cc:336] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.508046 16645 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:30.580545 16350 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:30.597422  8645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.567706 16573 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:30.557389 16515 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.042s
I20260709 13:05:30.599516  8719 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.568650 16648 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.608399 16648 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:30.610697 16648 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:30.592391  8793 leader_election.cc:336] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:30.612804  8793 leader_election.cc:336] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:30.614486  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:30.616120  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:30.617116  8793 leader_election.cc:336] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.617775  8793 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:30.619068  8627 auto_rebalancer.cc:629] Failed to schedule move for tablet 614067c154284cfa9b7a01100f304cc9: Illegal state: Replica e814ed2013244fa6be42623ea5c0c7b9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
W20260709 13:05:30.619628  8793 leader_election.cc:336] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.620817 16514 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:30.567898 16529 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:05:30.592901 16634 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:30.599521 16483 raft_consensus.cc:697] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:30.625473 16483 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:30.631090 16645 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:30.597822 16668 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:30.634572  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:30.637990  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:30.638950  8821 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:05:30.635414  8719 leader_election.cc:336] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.657238  8719 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.671347  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:30.654105 16722 log.cc:927] Time spent T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 13:05:30.672315  9382 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
W20260709 13:05:30.675917  8719 leader_election.cc:336] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.680199  8719 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:30.682554  8719 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:30.683894  8719 leader_election.cc:336] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.684787  8719 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.686353  8821 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 23.
I20260709 13:05:30.654765 16668 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:30.689999  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:30.632484 16350 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:30.698364  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:30.675233 16557 raft_consensus.cc:493] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.675844 16573 raft_consensus.cc:697] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:30.699445  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
W20260709 13:05:30.703315  8627 auto_rebalancer.cc:629] Failed to schedule move for tablet 3c997cc62c984d6e8962403382a451bb: Illegal state: Replica 0e9b0850145a4c66bfe983ffdfa93a57 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:30.704284  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:30.704952  8752 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 19.
W20260709 13:05:30.698045  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 150576us
I20260709 13:05:30.701294  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:30.685945 16534 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:30.688541 16664 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:30.713526  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:30.697003 16634 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:30.718919  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:30.719635  8749 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:30.718905  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:30.717147  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:30.723712  8645 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.728406  8645 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:30.692641 16571 consensus_queue.cc:1048] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.028s
I20260709 13:05:30.710872 16648 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:05:30.728184 16688 consensus_meta.cc:287] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: Time spent flushing consensus metadata: real 0.660s	user 0.015s	sys 0.000s
I20260709 13:05:30.710076 16557 raft_consensus.cc:515] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:30.733163 16681 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:30.745536 16726 log.cc:927] Time spent T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:05:30.742486  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39024 (request call id 3407) took 1150 ms. Trace:
I20260709 13:05:30.757159  8647 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.769030  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:30.761194  8749 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:30.768054  8748 rpcz_store.cc:276] 0709 13:05:29.592114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.639201 (+ 47087us) service_pool.cc:224] Handling call
0709 13:05:29.645099 (+  5898us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:29.667808 (+ 22709us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:29.765871 (+ 98063us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b18000725e0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32df386c0a 00007f32e8ac6219 00007f32df40ea40 00007f32df4094e4 00007f32df409455 00007f32e5ef13c7 00007f32e25039a7 00007f32e25130bf 00007f32e2513038
0709 13:05:30.436844 (+670973us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b18000725e0. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e82d4de1 00007f32e82e15b1 00007f32e20a14bc 00007f32df3ef48b 00007f32e8249ab9 00007f32df40b69b 00007f32df40993f 00007f32df409455 00007f32e5ef13c7 00007f32e25039a7 00007f32e25130bf
0709 13:05:30.441380 (+  4536us) log.cc:844] Serialized 14 byte log entry
0709 13:05:30.574361 (+132981us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:30.575108 (+   747us) log.cc:844] Serialized 52 byte log entry
0709 13:05:30.575260 (+   152us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:30.575291 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.575364 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:30.742234 (+166870us) raft_consensus.cc:1685] finished
0709 13:05:30.742258 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.742411 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36764,"prepare.queue_time_us":127944,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":25991,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":4967,"spinlock_wait_cycles":19747328,"thread_start_us":149319,"threads_started":2,"wal-append.queue_time_us":178608}
I20260709 13:05:30.736994 16542 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:30.774034  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:30.774955  8719 leader_election.cc:336] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.777653  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.749383 16614 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:30.782842  8719 leader_election.cc:336] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:30.752600 16648 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.785970  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:30.785298  8719 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.788327  8645 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:30.789390  8647 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.791262  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.776052 16542 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.794085  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:30.806437  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:30.781553 16631 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:30.784502 16680 raft_consensus.cc:2804] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:30.857851  8821 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
W20260709 13:05:30.864882  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 82061.1us
I20260709 13:05:30.798157  8749 raft_consensus.cc:1275] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:05:30.790385 16648 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:30.789673 16679 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:30.796759 16719 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:30.789767 16630 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:30.793516 16548 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:30.767629 16688 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:30.854624 16666 consensus_meta.cc:287] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.522s	user 0.014s	sys 0.000s
I20260709 13:05:30.857002 16542 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:30.787638 16635 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.085s
I20260709 13:05:30.843904 16532 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:30.847944 16616 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:30.951144 16703 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:05:30.951014 16616 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:30.953773 16616 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:30.843922 16718 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:30.871935 16544 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:30.874341 16680 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:30.885401 16733 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:05:30.875597 16568 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:05:30.783938 16573 consensus_queue.cc:237] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:30.948632 16557 leader_election.cc:290] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:30.962749  7526 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 8706)
I20260709 13:05:30.890542 16735 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:30.963335  9386 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:30.965025 16680 raft_consensus.cc:3060] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:30.968466 16718 raft_consensus.cc:697] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:30.895116 16666 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:30.995824  8722 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:31.002300  8822 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:05:31.007926  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:31.010079 16688 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:31.075490  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 79748.8us
I20260709 13:05:31.013906 16542 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:31.029245  8830 raft_consensus.cc:1275] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:05:31.080906  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
W20260709 13:05:31.030718  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:31.082667  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 52029.5us
I20260709 13:05:31.026948 16681 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:31.075151 16630 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.075861 16549 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:31.079896 16689 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:31.083436 16663 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:31.079020 16350 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:31.108285  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293a9dab5be3479a90d5803585682a00" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:31.075670 16735 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:31.114389  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:31.115510  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 450363us
I20260709 13:05:31.088675 16627 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
W20260709 13:05:31.116801  9352 leader_election.cc:336] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:31.123660  9352 leader_election.cc:336] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.124099  9352 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.094818 16745 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:31.093266 16664 consensus_queue.cc:1048] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:05:31.134567  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:31.112509 16350 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.101819 16549 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:31.110374 16663 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:31.103636 16689 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:31.100602 16630 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:31.125870 16635 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:31.152145  9355 leader_election.cc:304] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.149250 16687 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:31.167595 16234 raft_consensus.cc:2749] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:05:31.169551 16635 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:31.171154 16663 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:31.131502 16648 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:31.187505  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:31.177850  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.184286 16689 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:31.185051 16681 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.165114 16756 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.200757  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:31.168529 16630 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:31.194657 16735 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:31.199797 16738 consensus_queue.cc:1048] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:05:31.158354 16350 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:31.189407 16755 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:31.200508 16751 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:05:31.143615 16745 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:31.126276 16718 consensus_queue.cc:237] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:31.200165 16234 raft_consensus.cc:2749] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:31.195874 16681 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:31.237540 16758 log.cc:927] Time spent T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:05:31.230660 16752 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.276422 16752 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:31.277462  8755 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:31.212356 16756 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:31.279493 16350 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:31.280642 16756 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:31.231086 16754 log.cc:927] Time spent T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:05:31.229657 16755 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:31.288507 16752 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:31.289785 16755 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:31.294037 16681 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:31.247689 16757 log.cc:927] Time spent T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:05:31.221604 16630 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.282248 16745 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:31.312011 16681 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.329340  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:31.341394  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:31.335484 16549 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:31.331672 16681 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:31.313215 16630 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:31.369244  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:31.371620  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:31.379603  8793 leader_election.cc:336] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.393901  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:31.378145  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:31.383194  8719 leader_election.cc:336] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.379158  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:31.378736  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:31.408414  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:31.406592  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:31.411191  8752 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:05:31.413967  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:31.414331  8647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.032s	sys 0.011s
W20260709 13:05:31.415252  8647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.033s	sys 0.012s
W20260709 13:05:31.416113  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 622073us
I20260709 13:05:31.417102  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:31.385000 16760 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:05:31.385852 16759 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:05:31.418380  8647 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.381186  8748 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:05:31.407706 16746 log.cc:927] Time spent T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.465s	user 0.001s	sys 0.002s
I20260709 13:05:31.397441 16549 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.423523  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.441344 16542 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:31.443264 16542 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:31.422145  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:31.452569  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.452868  8752 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:05:31.453996 16542 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:31.454954 16635 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:31.455832  8647 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.456784 16635 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:31.460065  9355 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.409911 16681 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:31.467764 16234 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:31.469523  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:31.415390 16582 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:31.473847  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:31.475745  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:31.475745  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:31.476518  8826 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:31.478143  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:31.437769 16755 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:31.489202  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:31.420451 16718 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:31.496865  9381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:31.502223  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:31.498637  8647 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.506300  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:31.507663  8647 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.511620  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:31.512315 16681 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:31.523356  8826 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:05:31.524376  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:31.525954  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.529446  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:31.530575  8647 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:31.461578 16549 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:31.466035 16680 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:31.541159  8647 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:31.538419  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:31.525604 16681 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:31.502053 16755 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:31.546430  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.528374 16635 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:31.548671  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:31.535445 16350 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:31.550661  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:31.502787 16735 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:05:31.538302 16645 raft_consensus.cc:493] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:31.545644 16630 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:31.549263 16718 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:31.586825  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:31.588325 16549 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:31.589360 16635 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:31.589524  8645 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.593731  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:31.595633 16635 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:31.596164 16756 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:31.597671 16756 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:31.601372 16750 log.cc:927] Time spent T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.428s	user 0.000s	sys 0.004s
I20260709 13:05:31.601649 16756 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:31.602494 16630 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:31.591629 16549 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:31.564013 16681 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:31.616160  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:31.648689  8753 raft_consensus.cc:2393] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 19: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:31.617848  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:31.571730 16645 raft_consensus.cc:515] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:31.554090 16755 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:31.629575  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 484278us
W20260709 13:05:31.666760  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:31.676422  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:31.677598  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:31.670217  9352 leader_election.cc:336] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.682425  9352 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.666289 16718 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:31.693796 16707 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:05:31.732860  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:31.733300  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 235449us
W20260709 13:05:31.734962  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 57404.8us
W20260709 13:05:31.735247  8793 leader_election.cc:336] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:31.746428  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 180157us
I20260709 13:05:31.749300  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:31.751578  8796 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:31.747985  8719 leader_election.cc:336] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:31.761319  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 214981us
I20260709 13:05:31.757769  8719 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.766990  8647 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:31.768573  9386 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34734 (request call id 1367) took 1073 ms. Trace:
I20260709 13:05:31.773370  8647 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:31.774860  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 90886.4us
I20260709 13:05:31.763576 16483 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:31.768992  9386 rpcz_store.cc:276] 0709 13:05:30.690356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.690532 (+   176us) service_pool.cc:224] Handling call
0709 13:05:30.702759 (+ 12227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:31.028446 (+325687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.763808 (+735362us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b540061b554. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2462571 00007f32e24f9e01 00007f32e24f883e 00007f32e74bd419 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c 00007f32e08338c6
0709 13:05:31.763993 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":598744960}
I20260709 13:05:31.760748 16680 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:31.761034 16548 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:31.782732  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 217348us
I20260709 13:05:31.765995 16634 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:31.785535  8793 leader_election.cc:336] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:31.769429 16549 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.776810 16542 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:31.819620 16645 leader_election.cc:290] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:31.821440  8794 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:31.786556 16663 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:31.784231 16681 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:31.754415 16718 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:31.838550  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 66030.3us
W20260709 13:05:31.838459  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 69666.1us
I20260709 13:05:31.816084 16634 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:31.797614 16548 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:31.835405 16483 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:31.870461  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 122506us
I20260709 13:05:31.822506 16549 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:31.912609 16775 log.cc:927] Time spent T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:05:31.894891 16630 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.910466 16755 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:31.942728 16648 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:31.946141 16735 consensus_queue.cc:1059] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:05:31.948135 16783 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:05:31.948639 16783 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:31.949623 16783 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:31.953152  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:31.959358  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:31.960354  9389 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:31.930014 16649 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:31.979468  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:31.983093  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:31.926298 16630 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:31.992914  8756 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:05:32.003397  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 137300us
W20260709 13:05:32.001204  8719 leader_election.cc:336] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:32.005400  8719 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:32.010049 16768 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260709 13:05:32.010555 16769 log.cc:927] Time spent T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:05:32.018738  7526 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 8706)
I20260709 13:05:32.031149 16634 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:31.986861 16649 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.031997 16549 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:32.046358  9389 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 23.
I20260709 13:05:32.046255  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.055723  8824 raft_consensus.cc:1275] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:05:32.061911 16630 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:32.043066 16645 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:32.035619 16648 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:32.064525 16582 consensus_meta.cc:287] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: Time spent flushing consensus metadata: real 0.569s	user 0.011s	sys 0.004s
I20260709 13:05:32.065088 16582 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:32.063946 16648 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.046902 16548 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:32.068302  8825 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:32.097085  8647 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145; no voters: 
I20260709 13:05:32.098559 16548 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:32.099239 16548 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:32.100296 16548 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:32.105770  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:32.106940 16549 raft_consensus.cc:493] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.107547 16549 raft_consensus.cc:515] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.109233 16783 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:32.109895  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:32.110078 16755 raft_consensus.cc:493] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.110841  8756 raft_consensus.cc:2410] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:05:32.110565 16755 raft_consensus.cc:515] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:32.112725 16755 leader_election.cc:290] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:32.120028  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41d15a8ebfe8480d9861c1285faaf7b5" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:32.109756 16549 leader_election.cc:290] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:32.136924  8719 consensus_peers.cc:597] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:32.070386  8824 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:05:32.076077 16649 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:32.069656 16790 consensus_queue.cc:1048] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:32.090724  8825 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 23.
I20260709 13:05:32.133870  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:32.124975  8722 leader_election.cc:304] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:32.148819  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.150003  8821 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:32.156083  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:32.154898  9352 leader_election.cc:336] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:32.158080  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:32.158703  9352 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:32.163596  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:32.154182 16634 raft_consensus.cc:2749] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:32.141746 16771 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:32.155838 16529 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.028s	sys 0.065s
I20260709 13:05:32.156359 16680 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.136619 16630 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:05:32.175336 16542 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:32.201081 16529 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.028s	sys 0.066s
I20260709 13:05:32.218739  8826 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:05:32.223919  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:32.225184  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 68321.1us
I20260709 13:05:32.228978  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.230209  8826 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 20.
I20260709 13:05:32.231352  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13d1c43d45594a8d9dc9c717fd6e4ca7" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.244197  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae08d8f5271e475baa4344843405b254" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:32.244990 16630 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.245636  8825 raft_consensus.cc:2393] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 21: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:32.187911 16771 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:32.251406  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 67403.8us
W20260709 13:05:32.252458  9352 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 150 similar messages]
I20260709 13:05:32.255215  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.257078 16648 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:32.260833  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:32.261622  8751 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 28.
I20260709 13:05:32.262773  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:32.263526  8754 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 23.
I20260709 13:05:32.245697 16630 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.267783  8824 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:32.270443 16630 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:32.274101  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:32.275028  8747 raft_consensus.cc:2393] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 23: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:32.250188 16771 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:32.279407 16529 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:32.280233 16529 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:32.280949  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.286536  8645 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:05:32.286906  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 60660.7us
I20260709 13:05:32.288149  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:32.288762  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:32.289083  8747 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:32.178728 16704 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:32.290786  8645 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
W20260709 13:05:32.299705  8719 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:32.333184  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:32.211404 16680 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:32.296967 16542 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:05:32.298835 16630 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:32.359992  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.302739 16771 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:32.353786  8824 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:05:32.366426  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 82875.8us
I20260709 13:05:32.356072 16542 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.367300 16718 raft_consensus.cc:493] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.364427 16630 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.365008 16771 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.352883 16791 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:32.385861  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:32.345600 16534 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:05:32.405802  9386 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 21: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:32.370685 16529 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:32.395539 16799 log.cc:927] Time spent T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:05:32.432035  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.406575 16798 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.433626 16798 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.384485 16542 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:05:32.378765 16668 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:32.387058 16630 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:32.405509 16791 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.404906 16771 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:05:32.466144  8793 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:32.387244 16718 raft_consensus.cc:515] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.458853 16791 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:32.477329 16798 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:32.485014  8821 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:32.462646 16680 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:32.493654  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:32.459894 16668 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:32.523413 16794 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 13:05:32.524746  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 159747us
W20260709 13:05:32.526018 16800 log.cc:927] Time spent T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:05:32.533594 16718 leader_election.cc:290] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:32.535596 16718 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.536252 16718 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:32.541298 16718 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:32.543105  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:32.534361 16542 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.533866 16668 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:32.554653 16542 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 29 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:32.578526  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:32.580349  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:32.583217  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c963cd681ab4f45a5c339f9fd6e2687" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.585198  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:32.587452  8647 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:32.590042  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:32.588234 16542 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:05:32.593572  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:32.574681 16614 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.590987 16718 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:32.602159 16718 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.602689 16718 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:32.602413 16614 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:32.603403  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:32.603876  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 153941us
I20260709 13:05:32.601421  8647 leader_election.cc:304] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:32.612880  8647 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:32.593284 16680 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:32.591353 16755 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:32.602706 16796 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:32.603685 16668 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:32.631871  8822 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:05:32.646175  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:32.659965  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:32.660733  8749 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:05:32.662132  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:32.638093 16668 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:32.613130 16549 raft_consensus.cc:2749] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:32.677448  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:32.684126  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:32.644784 16796 raft_consensus.cc:3060] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:32.648370 16681 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.704464  8747 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:05:32.706266  8749 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 29.
I20260709 13:05:32.675246 16668 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:32.690632 16797 log.cc:927] Time spent T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.215s	user 0.002s	sys 0.002s
I20260709 13:05:32.647218 16648 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 13:05:32.713083  8645 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:32.720096  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.720551  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.721192  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.721030  8830 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 28.
I20260709 13:05:32.724201  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.724347  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:32.724648  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4f0aa5ad2ef496d9a459b6742346f0a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.724948  8830 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:05:32.724354  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.702129 16654 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:32.732230  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:32.750591  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 48095.9us
I20260709 13:05:32.721294  8826 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 17.
I20260709 13:05:32.738865  8645 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:32.752604 16680 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:32.753767 16614 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:32.710297 16681 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.739928  8824 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:05:32.741348  8647 leader_election.cc:304] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:32.725137 16549 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:05:32.764528  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:32.758128 16791 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.769552  8647 leader_election.cc:304] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:32.766315 16680 raft_consensus.cc:2749] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:32.804430  8830 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 29.
W20260709 13:05:32.787575  8793 leader_election.cc:336] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:32.799154  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:32.716931 16798 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
W20260709 13:05:32.835476  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 47841.3us
I20260709 13:05:32.760774 16796 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:32.779769 16755 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:32.867781  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 96424.9us
I20260709 13:05:32.760725 16630 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.873034 16630 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:32.873756 16630 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.875229 16796 raft_consensus.cc:2749] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:32.874253 16630 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.876291 16771 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.876909 16649 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.877697 16649 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.893289  8754 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:05:32.776713 16787 raft_consensus.cc:493] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:32.895217 16787 raft_consensus.cc:515] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:32.909593 16787 leader_election.cc:290] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:32.893819 16771 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:32.910597 16681 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:32.911036  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:32.911805  8750 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 22.
I20260709 13:05:32.906148 16549 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 29 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:32.913754 16791 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:32.892676  8571 catalog_manager.cc:5697] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:32.919371 16681 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.920676 16681 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.921365 16783 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:32.921526 16718 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:32.923108 16681 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:32.924005  8794 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:05:32.942169  8793 consensus_peers.cc:597] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:32.914871 16549 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.13, Last appended by leader: 13, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:32.971846  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:32.974321  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 204795us
I20260709 13:05:32.982220  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:32.943217  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 89735us
I20260709 13:05:32.935304 16783 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:33.004184 16654 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:33.013947  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:33.014734  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:33.015327  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.928138 16630 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:33.008621  8829 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:05:33.008734  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.941541 16718 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:33.020365  7526 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 8706)
I20260709 13:05:33.020459  8828 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 23.
W20260709 13:05:33.012501  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 127296us
I20260709 13:05:33.017128  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:32.945495 16649 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:33.009208  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:33.023612  8824 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:33.028015  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:33.029070  8825 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:33.006474 16783 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.035152 16783 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:33.012502 16814 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.005108 16668 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:33.013437 16771 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:33.041706  8824 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
I20260709 13:05:33.042531 16630 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.043146 16718 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:33.055631  8719 leader_election.cc:336] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:33.056353  8719 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:33.031328 16680 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.062721  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:33.068907  8749 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 21: Already voted for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in this term.
I20260709 13:05:33.061076  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.061354  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:33.059036 16573 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:33.064169 16630 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:33.065032 16668 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Requested vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:33.065555  8752 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Rejecting Update request from peer e814ed2013244fa6be42623ea5c0c7b9 for earlier term 20. Current term is 21. Ops: []
I20260709 13:05:33.066968  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:33.067673  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.036088 16811 raft_consensus.cc:493] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.071046  8755 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:33.070204  8753 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 22.
I20260709 13:05:33.075943  8754 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:33.078532  9383 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
W20260709 13:05:33.093129  8796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.093783 16651 raft_consensus.cc:2955] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:05:33.098126  8825 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 21.
I20260709 13:05:33.053745 16755 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.027s
I20260709 13:05:33.099118  8645 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:33.100258  8645 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:33.101385 16798 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:05:33.101905 16798 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.058552 16771 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:33.051304 16814 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:33.113075  8647 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:33.070988 16680 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:33.133473  8755 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 24.
I20260709 13:05:33.138916  8645 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:33.104555 16718 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:33.139313 16549 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:33.149866 16549 raft_consensus.cc:493] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:33.151412 16549 raft_consensus.cc:515] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:33.110641 16798 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:33.109970 16815 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.119119 16681 raft_consensus.cc:493] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:33.120524 16548 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.064s
I20260709 13:05:33.185825 16681 raft_consensus.cc:515] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:33.188392 16681 leader_election.cc:290] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:33.188653 16680 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:33.190050 16630 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:33.191224 16771 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:33.196118  8754 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c59a1f0394eae8ce33b99f10efadc in term 18.
W20260709 13:05:33.193737  8719 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:33.202054  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.162555 16812 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:33.102726 16811 raft_consensus.cc:515] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:33.205637  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.209040  8755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.211891  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.204602  8645 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 
I20260709 13:05:33.154424 16796 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:05:33.217474 16771 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:33.217628 16812 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:33.243674 16812 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:33.244648 16814 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:33.221560 16811 leader_election.cc:290] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:33.240567  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.242529 16771 raft_consensus.cc:697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:33.248783 16771 consensus_queue.cc:237] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:33.264070  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 133616us
I20260709 13:05:33.175971 16815 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:33.262518 16817 log.cc:927] Time spent T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:05:33.268551  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:33.267783 16822 log.cc:927] Time spent T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:05:33.266140  8647 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:33.271488  8796 consensus_peers.cc:597] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53cce7f107584cc28999319a785023ef. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:33.274260  8751 raft_consensus.cc:2955] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:05:33.277748 16815 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:33.278604 16828 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:33.279462 16828 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 LEADER]: Becoming Leader. State: Replica: 0e8c59a1f0394eae8ce33b99f10efadc, State: Running, Role: LEADER
I20260709 13:05:33.280463 16828 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:33.282943  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 281507us
W20260709 13:05:33.284091  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.285851  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.305457 16783 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:33.309471  8827 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 23: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
I20260709 13:05:33.310541 16783 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:33.318526  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:33.319000  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:33.324819  9388 raft_consensus.cc:2393] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 21: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:05:33.329268  8571 catalog_manager.cc:5697] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 15 to 16, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 16 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:33.323915  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.341611  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "396dbfb60a3644b9b28c39c3dd36b322" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:33.269351 16549 leader_election.cc:290] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:33.361179  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:33.356146  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:33.361764  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:33.368510  8751 raft_consensus.cc:3060] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:05:33.369100  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:33.378638  8571 catalog_manager.cc:5697] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 15 to 19, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 19 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:33.385633  8826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
W20260709 13:05:33.387183  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.386739  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:33.388207  8825 raft_consensus.cc:2393] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e9b0850145a4c66bfe983ffdfa93a57 in current term 21: Already voted for candidate 0e8c59a1f0394eae8ce33b99f10efadc in this term.
I20260709 13:05:33.381795  8571 catalog_manager.cc:5697] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:33.390309  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:33.391959 16798 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:33.398707  8647 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:33.400132  8647 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:33.402748  8751 raft_consensus.cc:2468] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 23.
I20260709 13:05:33.402281  8571 catalog_manager.cc:5697] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 18 to 21, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 21 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:33.406015  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:33.361296 16549 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.394405 16798 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:33.409487  8722 leader_election.cc:304] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:33.373548  8570 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40096 (request call id 144) took 58652 ms (client timeout 15000 ms). Trace:
I20260709 13:05:33.413296  8748 raft_consensus.cc:2393] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e8c59a1f0394eae8ce33b99f10efadc in current term 23: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:05:33.415555  8793 leader_election.cc:336] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:33.416119  8793 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:33.417259  8794 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:33.428292  8826 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:33.429669  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95fdf24d372f40b8877c2792102f34cf" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:33.431819  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e078746b204711a1e15f221c16e31a" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:33.433974  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05559e602d4b4ea3986701fa621706a0" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:33.406956 16827 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:33.431690  8827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:33.415223 16796 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:05:33.420135  8571 catalog_manager.cc:5697] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:33.444485  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:33.447993  8571 catalog_manager.cc:5697] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 13 to 14, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 14 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:33.453811  8828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:33.454128  8571 catalog_manager.cc:5697] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:33.456503  8571 catalog_manager.cc:5697] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 11 to 14, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 14 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:33.458468  8571 catalog_manager.cc:5697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 16 to 17, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 17 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:33.417879 16549 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:33.463114 16549 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:33.431499 16783 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:05:33.424175 16787 raft_consensus.cc:2749] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:33.479529  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:33.483068  8722 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:33.483418  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:33.435832 16052 raft_consensus.cc:2804] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:05:33.504602 16052 raft_consensus.cc:697] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:33.506201 16052 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:33.491922 16793 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:05:33.499684 16787 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:33.498227 16557 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.523059  8647 leader_election.cc:304] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:33.532678  8647 leader_election.cc:304] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:33.539812  8647 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:33.541083 16549 raft_consensus.cc:2749] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:33.541452  8647 leader_election.cc:304] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:33.542150 16549 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:33.542326 16771 raft_consensus.cc:2749] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:33.557230  8647 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:33.544245 16796 raft_consensus.cc:2749] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:33.574872  8830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:33.576972  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:33.577271 16798 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:33.578022 16798 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:33.413035  8570 rpcz_store.cc:269] 0709 13:04:34.721650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.721981 (+   331us) service_pool.cc:224] Handling call
0709 13:04:34.996469 (+274488us) spinlock_profiling.cc:243] Waited 5.24 ms on lock 0x7b5400002490. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d50b2 00007f32e834ea89 00007f32e82e6da6 00007f32e82b8088 00007f32e83b637e 00007f32e68772a4 00007f32e6877232 00007f32e6877161 00007f32e68770dc
0709 13:05:33.373416 (+58376947us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:30.207466 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:30.323722 (+116256us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:30.323785 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:05:30.323807 (+    22us) tablet.cc:662] Decoding operations
0709 13:05:30.364396 (+ 40589us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:30.364516 (+   120us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:30.364548 (+    32us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:05:30.365544 (+   996us) tablet.cc:701] Row locks acquired
0709 13:05:30.365597 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:05:30.365814 (+   217us) write_op.cc:270] Start()
0709 13:05:30.366001 (+   187us) write_op.cc:276] Timestamp: P: 1783602330365737 usec, L: 0
0709 13:05:30.366020 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:05:30.393277 (+ 27257us) log.cc:844] Serialized 3574 byte log entry
0709 13:05:30.719109 (+325832us) op_driver.cc:464] REPLICATION: finished
0709 13:05:30.972350 (+253241us) write_op.cc:301] APPLY: starting
0709 13:05:30.972474 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:31.101034 (+128560us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:31.107442 (+  6408us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:31.337094 (+229652us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:31.351032 (+ 13938us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 13:05:31.351096 (+    64us) write_op.cc:312] APPLY: finished
0709 13:05:31.351578 (+   482us) log.cc:844] Serialized 89 byte log entry
0709 13:05:31.352743 (+  1165us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:31.352931 (+   188us) write_op.cc:454] Released schema lock
0709 13:05:31.353018 (+    87us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:31.359285 (+  6267us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:31.359334 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1061,"reported_tablets":28,"spinlock_wait_cycles":12061568,"child_traces":[["op",{"apply.queue_time_us":245531,"apply.run_cpu_time_us":20180,"apply.run_wall_time_us":386878,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41711,"num_ops":9,"prepare.queue_time_us":165160,"prepare.run_cpu_time_us":22011,"prepare.run_wall_time_us":301352,"raft.queue_time_us":152864,"raft.run_cpu_time_us":15964,"raft.run_wall_time_us":187299,"replication_time_us":332554,"thread_start_us":290801,"threads_started":4,"wal-append.queue_time_us":92190,"wal-append.run_cpu_time_us":48643,"wal-append.run_wall_time_us":1859652}]]}
I20260709 13:05:33.548903 16787 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:33.544386 16557 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:33.600682 16838 log.cc:927] Time spent T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.110s	user 0.002s	sys 0.000s
I20260709 13:05:33.614488 16796 raft_consensus.cc:493] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:33.613631 16798 raft_consensus.cc:493] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.674889 16812 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.678930 16812 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:33.683079 16812 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:33.684341 16796 raft_consensus.cc:515] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:33.688458 16806 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:05:33.683425 16798 raft_consensus.cc:515] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:33.672240 16549 raft_consensus.cc:493] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.697774 16549 raft_consensus.cc:515] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:33.703796  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.711195 16796 leader_election.cc:290] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:33.708900 16787 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:33.735244 16549 leader_election.cc:290] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:33.739845  8752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.752928  8756 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:05:33.746809  8748 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 29 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 29 index: 14. (index mismatch)
I20260709 13:05:33.748309  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.748688  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.758698  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 26 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:33.759349  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f81d7aa5ed04c68b603d272f24760e2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 26 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.761078  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01533e2a5204b1ebccac1133e495542" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:33.770289  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 215291us
I20260709 13:05:33.708170 16812 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:33.787067  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 43745.6us
I20260709 13:05:33.806346  8824 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 29 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 29 index: 14. (index mismatch)
I20260709 13:05:33.804499 16557 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:33.827504  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.827371  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:33.829115  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.828327  8747 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:05:33.833570 16828 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.834508 16828 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:33.836328 16798 leader_election.cc:290] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:33.836867  8824 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:05:33.839901  8793 leader_election.cc:336] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:33.840451  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:33.830183  8747 pending_rounds.cc:85] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 10
I20260709 13:05:33.844321  8747 pending_rounds.cc:107] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:05:33.841025  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:33.845185  8747 raft_consensus.cc:2889] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:33.844326  8829 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 21.
W20260709 13:05:33.832515  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.848510  8645 leader_election.cc:304] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:33.853510 16549 raft_consensus.cc:493] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.853461 16827 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:33.854287 16549 raft_consensus.cc:515] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:33.795126 16812 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:33.846935 16796 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:05:33.859477 16549 leader_election.cc:290] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:33.860682 16648 raft_consensus.cc:2749] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:05:33.861198 16848 ts_tablet_manager.cc:933] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:33.863583  8645 leader_election.cc:304] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:33.864624 16771 raft_consensus.cc:2749] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:33.851756 16735 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:33.866484 16771 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:05:33.860760 16850 log.cc:927] Time spent T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:05:33.869875 16812 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:33.871960  8824 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:05:33.880982  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:33.876098  8754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.876838  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.877123  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.880215  8750 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:05:33.875293  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:33.890573  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
W20260709 13:05:33.906203  8793 leader_election.cc:336] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:33.887063  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:33.907204  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 49904.1us
W20260709 13:05:33.907898  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.907732  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:33.915025  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3796428796f4d6c8fd8e9ff2897d116" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:33.927666  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.896096 16693 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:33.939097  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c31a55ecab4a17acb00373b55d6f1d" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:33.944231  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:33.910130 16648 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:33.946324  8821 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:05:33.956841  8647 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:33.935045  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.903118 16796 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:05:33.938370 16828 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:33.941933 16693 raft_consensus.cc:493] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:33.965790  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:33.970048  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:33.960342 16812 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:33.966471 16693 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:33.994347  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:33.995594 16849 log.cc:927] Time spent T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:05:33.960661 16848 tablet_copy_client.cc:323] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:05:34.014290  8647 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:34.021106  7526 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 8706)
I20260709 13:05:34.036105  8840 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet 53cce7f107584cc28999319a785023ef from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
I20260709 13:05:34.036976  8840 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet 53cce7f107584cc28999319a785023ef from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-53cce7f107584cc28999319a785023ef
I20260709 13:05:34.021507 16796 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:05:34.045434 16648 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:05:34.055277  8627 auto_rebalancer.cc:629] Failed to schedule move for tablet 3524354c9db64c0c9b532bef2a0ceb6f: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:34.034859  8647 leader_election.cc:304] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:34.060410  8647 leader_election.cc:304] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:34.061324 16648 raft_consensus.cc:2749] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:34.062052 16648 raft_consensus.cc:2749] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:34.062474  8751 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:05:34.064560  8647 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:34.080467  8823 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Refusing update from remote peer 0e8c59a1f0394eae8ce33b99f10efadc: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:05:34.031680 16828 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:34.077513 16648 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:05:34.073800 16549 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
W20260709 13:05:34.160183  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 48211.4us
W20260709 13:05:34.173174  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 92790.4us
I20260709 13:05:34.161480 16549 raft_consensus.cc:493] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.207028 16648 raft_consensus.cc:493] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:34.206099  8722 consensus_peers.cc:597] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 304eb26fa48d4971bffbe6266846243b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:34.220847  8823 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:05:34.169298 16693 raft_consensus.cc:515] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:34.245566 16856 log.cc:927] Time spent T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:05:34.247915  8840 tablet_copy_source_session.cc:215] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:34.238149 16814 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.263666 16814 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:34.266130 16814 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:34.242998 16771 raft_consensus.cc:493] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.267306 16771 raft_consensus.cc:515] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:34.269491 16771 leader_election.cc:290] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:34.241345 16834 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:05:34.283043 16848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53cce7f107584cc28999319a785023ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:34.289290 16693 leader_election.cc:290] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:34.207713 16648 raft_consensus.cc:515] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:34.292672 16648 leader_election.cc:290] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:34.206542 16549 raft_consensus.cc:515] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:34.295488 16549 leader_election.cc:290] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:34.296285  9386 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 21. Current term is 22. Ops: []
I20260709 13:05:34.302026  8825 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:05:34.305573  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 77883.6us
I20260709 13:05:34.305944  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:34.306596  8825 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:34.314437 16798 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:34.339505 16787 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:05:34.341219 16663 raft_consensus.cc:2955] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 LEADER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
W20260709 13:05:34.356199  8719 consensus_peers.cc:597] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:34.361079 16848 tablet_copy_client.cc:806] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:34.361992 16848 tablet_copy_client.cc:670] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:34.367633  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:34.368731  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:34.374907  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:34.375221  8825 raft_consensus.cc:2468] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 22.
I20260709 13:05:34.376024  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:34.373119  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:34.376616  9355 leader_election.cc:304] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:34.388952  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:34.389145  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c86a8d1b46a4dad9149c50458be45d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:34.379925  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8611228be8dd4f08b2b609e202974a6a" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:34.393793  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142891c077a34ba289adf90a99aee612" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:34.394842  8756 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:05:34.415512  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c390d8b5bb47c9a7d7a841a4fe135e" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:34.415562 16848 tablet_copy_client.cc:538] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:34.407069 16801 raft_consensus.cc:2804] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:34.449934  8823 raft_consensus.cc:1240] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Rejecting Update request from peer 0e9b0850145a4c66bfe983ffdfa93a57 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:05:34.459031 16791 consensus_queue.cc:1059] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:05:34.468333 16791 raft_consensus.cc:3055] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:05:34.468825 16791 raft_consensus.cc:740] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:34.457928 16801 raft_consensus.cc:697] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:34.469744 16791 consensus_queue.cc:260] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:34.470484 16801 consensus_queue.cc:237] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:34.503155  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:34.528363 16798 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.529350 16798 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:34.532742 16798 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:34.535269  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 51815.1us
W20260709 13:05:34.562209  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 48513.5us
I20260709 13:05:34.508731 16791 raft_consensus.cc:3060] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:34.558789 16771 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:34.576689  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:34.575932  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 156639us
I20260709 13:05:34.577900  8645 leader_election.cc:304] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:34.580125  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 71703.3us
I20260709 13:05:34.594537  8645 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:34.567492 16582 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:05:34.584306 16771 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:34.615643 16771 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:34.622430  8577 catalog_manager.cc:5697] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 19 to 24. New cstate: current_term: 24 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:34.625212 16869 log.cc:927] Time spent T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:05:34.624711  8577 catalog_manager.cc:5697] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.639762 16873 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:34.626569  8577 catalog_manager.cc:5697] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 16 to 20, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 20 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.632050 16872 raft_consensus.cc:2749] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:34.648813  8645 leader_election.cc:304] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:34.650781  8645 leader_election.cc:304] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:34.652563  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 45340.3us
I20260709 13:05:34.659781  8577 catalog_manager.cc:5697] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.661898  8577 catalog_manager.cc:5697] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 15 to 19, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 19 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:34.664371  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:34.665411  8719 consensus_peers.cc:597] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:34.666256  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:34.666844  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:34.652231 16812 raft_consensus.cc:2749] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:34.668110 16866 raft_consensus.cc:2749] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:34.692054  8829 raft_consensus.cc:2955] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:05:34.703347  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca09750f1b44e018d62c21aeb12a929" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:34.706127  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:34.712636  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:34.726027  8577 catalog_manager.cc:5697] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.728752  8577 catalog_manager.cc:5697] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 10 to 16, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 16 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:34.723594  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:34.733125  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:34.734447  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:34.731133  8577 catalog_manager.cc:5697] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 15 to 22, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 22 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:34.738677 16796 raft_consensus.cc:493] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.739445 16796 raft_consensus.cc:515] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:34.741958 16796 leader_election.cc:290] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:34.756994  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:34.758535  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8acd9840694e4f62803bfe9810425202" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:34.760838  8577 catalog_manager.cc:5697] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:34.763449  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:34.763896  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 45166.8us
W20260709 13:05:34.764647  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:34.768201  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:34.770246  8627 auto_rebalancer.cc:629] Failed to schedule move for tablet 7fbae1ebcfaf4be58c5aecd1014cea01: Illegal state: Replica 0e9b0850145a4c66bfe983ffdfa93a57 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:34.767828  8577 catalog_manager.cc:5697] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 15 to 23, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 23 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.773782 16848 tablet_bootstrap.cc:492] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:05:34.780099  8577 catalog_manager.cc:5697] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 13 to 21, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 21 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:34.782091  8577 catalog_manager.cc:5697] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 15 to 20, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 20 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:34.783627  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 52127.6us
I20260709 13:05:34.784693  8645 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:34.784121  8577 catalog_manager.cc:5697] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.786921  8577 catalog_manager.cc:5697] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.767356 16812 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.791752  8647 leader_election.cc:304] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:34.793395  8645 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:34.795428 16814 raft_consensus.cc:493] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:34.795768 16771 raft_consensus.cc:2749] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:34.796087 16814 raft_consensus.cc:515] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:34.798288 16814 leader_election.cc:290] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:34.801853 16796 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:34.802680 16873 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:34.803618 16871 raft_consensus.cc:493] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:34.804090 16549 raft_consensus.cc:493] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.818145 16812 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:34.835497  8571 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43462 (request call id 271) took 42772 ms (client timeout 15000 ms). Trace:
W20260709 13:05:34.838667  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:34.819654  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:34.820744  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:34.818468  8577 catalog_manager.cc:5697] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.827081  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389e9e63581e4d1b941ba995f8cdbfc4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:34.823915 16871 raft_consensus.cc:515] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:34.848299  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:34.842273  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:34.853551 16796 raft_consensus.cc:493] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.854167 16873 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.854261 16796 raft_consensus.cc:515] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:34.854913 16873 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:34.857326 16873 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:34.850201 16871 leader_election.cc:290] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:34.860394  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:34.837767 16812 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:34.841261  8577 catalog_manager.cc:5697] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.822417 16877 ts_tablet_manager.cc:933] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:34.831589 16549 raft_consensus.cc:515] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:34.857856 16796 leader_election.cc:290] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:34.863389  8577 catalog_manager.cc:5697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.842840  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:34.870873  8825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:34.871145  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:34.864856  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7eadf5c488447886a90612fbf1a2bb" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:34.865367  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:34.872773  8645 leader_election.cc:304] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:34.873126  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:34.869479 16549 leader_election.cc:290] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:34.876422  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:34.877486  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:34.875222  8577 catalog_manager.cc:5697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.881563  8647 leader_election.cc:304] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:34.877892  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10f975b4911743768a7512f2ad3793c9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:34.884833 16796 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:34.889823  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88e9b3a77fe404e8e4caaefe74a04a2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:34.891989 16873 raft_consensus.cc:2749] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:34.838759  8571 rpcz_store.cc:269] 0709 13:04:52.068633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.253580 (+3184947us) service_pool.cc:224] Handling call
0709 13:05:34.835386 (+39581806us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:33.569525 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:33.570140 (+   615us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:33.570172 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:05:33.570189 (+    17us) tablet.cc:662] Decoding operations
0709 13:05:33.571234 (+  1045us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:33.571344 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:33.571374 (+    30us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:05:33.689647 (+118273us) tablet.cc:701] Row locks acquired
0709 13:05:33.692844 (+  3197us) write_op.cc:260] PREPARE: finished
0709 13:05:33.706059 (+ 13215us) write_op.cc:270] Start()
0709 13:05:33.714331 (+  8272us) write_op.cc:276] Timestamp: P: 1783602333703589 usec, L: 0
0709 13:05:33.714378 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 13:05:33.715325 (+   947us) log.cc:844] Serialized 3974 byte log entry
0709 13:05:34.083690 (+368365us) op_driver.cc:464] REPLICATION: finished
0709 13:05:34.180693 (+ 97003us) write_op.cc:301] APPLY: starting
0709 13:05:34.180790 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:34.244701 (+ 63911us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:34.244837 (+   136us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:34.251157 (+  6320us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:34.251462 (+   305us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 13:05:34.251514 (+    52us) write_op.cc:312] APPLY: finished
0709 13:05:34.251877 (+   363us) log.cc:844] Serialized 97 byte log entry
0709 13:05:34.252746 (+   869us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:34.252909 (+   163us) write_op.cc:454] Released schema lock
0709 13:05:34.252962 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:34.253434 (+   472us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:34.253634 (+   200us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":794735,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":96710,"apply.run_cpu_time_us":14730,"apply.run_wall_time_us":72711,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45887,"num_ops":10,"prepare.queue_time_us":102885,"prepare.run_cpu_time_us":94758,"prepare.run_wall_time_us":324975,"raft.queue_time_us":86617,"raft.run_cpu_time_us":27529,"raft.run_wall_time_us":104433,"replication_time_us":369134,"thread_start_us":370938,"threads_started":4,"wal-append.queue_time_us":196466}]]}
I20260709 13:05:34.896369  8645 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:34.901684  8645 leader_election.cc:304] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:34.903893  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:34.904982  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:34.888484 16549 raft_consensus.cc:2749] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:34.904126 16796 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:05:34.917833  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:34.905718 16873 raft_consensus.cc:2749] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:34.919552  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:34.924897  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:34.952131  8645 leader_election.cc:304] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:34.952472 16877 tablet_copy_client.cc:323] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.130:42321
I20260709 13:05:34.954514 16812 raft_consensus.cc:2749] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:34.953645 16873 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.955426 16873 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:34.956003  8766 tablet_copy_service.cc:140] P 0e9b0850145a4c66bfe983ffdfa93a57: Received BeginTabletCopySession request for tablet 304eb26fa48d4971bffbe6266846243b from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:35702)
I20260709 13:05:34.956722  8766 tablet_copy_service.cc:161] P 0e9b0850145a4c66bfe983ffdfa93a57: Beginning new tablet copy session on tablet 304eb26fa48d4971bffbe6266846243b from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:35702: session id = 7aff04f53d4845a6a4fa491fd2a06145-304eb26fa48d4971bffbe6266846243b
I20260709 13:05:34.961315  8756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:34.981037  8824 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:05:34.981477  8756 raft_consensus.cc:1275] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:05:34.983331  8756 pending_rounds.cc:85] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting all ops after (but not including) 15
I20260709 13:05:34.982944 16801 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:05:34.983582  8756 pending_rounds.cc:107] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:05:34.983877  8756 raft_consensus.cc:2889] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:34.993809 16801 consensus_queue.cc:1048] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:05:34.997911  8766 tablet_copy_source_session.cc:215] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:35.000653  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "364ad2097a2341cdb96752b9a370f9a9" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:35.001552 16873 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:35.002833  8647 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:35.004253 16873 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:35.021914  7526 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 8706)
I20260709 13:05:35.048394 16877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304eb26fa48d4971bffbe6266846243b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:35.105245 16877 tablet_copy_client.cc:806] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:35.105962 16877 tablet_copy_client.cc:670] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:35.106419 16873 raft_consensus.cc:493] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.107362 16871 raft_consensus.cc:493] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.108124 16871 raft_consensus.cc:515] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:35.112079 16871 leader_election.cc:290] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:35.116870  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:35.115161  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee1e688ccd74392befec90535c1efa4" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:35.120962  8647 leader_election.cc:304] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:35.107311 16873 raft_consensus.cc:515] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:35.127909  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:35.129752 16871 raft_consensus.cc:2749] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:35.152477 16877 tablet_copy_client.cc:538] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:35.154441 16873 leader_election.cc:290] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:35.155761  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:35.159653  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67bfc59e963f4ad0ac128fd11eecac17" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:35.165174 16877 tablet_bootstrap.cc:492] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:05:35.165007  8645 leader_election.cc:304] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:35.167333 16812 raft_consensus.cc:2749] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:35.173305 16848 tablet_bootstrap.cc:492] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:35.183848 16848 tablet_bootstrap.cc:492] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:05:35.184921 16848 ts_tablet_manager.cc:1403] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.412s	user 0.062s	sys 0.031s
I20260709 13:05:35.220446 16873 raft_consensus.cc:493] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.222239 16873 raft_consensus.cc:515] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:35.227037  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:35.227072  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3524354c9db64c0c9b532bef2a0ceb6f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:35.230772 16873 leader_election.cc:290] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:35.255432  8645 leader_election.cc:304] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:35.270129  8577 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40096 (request call id 146) took 14181 ms (client timeout 15000 ms). Trace:
I20260709 13:05:35.259232 16812 raft_consensus.cc:2749] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:35.301205  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:35.270598  8577 rpcz_store.cc:269] 0709 13:05:21.085633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.207113 (+7121480us) service_pool.cc:224] Handling call
0709 13:05:35.266503 (+7059390us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:34.954915 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:34.955731 (+   816us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:34.955768 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:05:34.955788 (+    20us) tablet.cc:662] Decoding operations
0709 13:05:34.987564 (+ 31776us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:34.989247 (+  1683us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:34.989614 (+   367us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:05:35.027893 (+ 38279us) tablet.cc:701] Row locks acquired
0709 13:05:35.027916 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:05:35.028091 (+   175us) write_op.cc:270] Start()
0709 13:05:35.028404 (+   313us) write_op.cc:276] Timestamp: P: 1783602335028034 usec, L: 0
0709 13:05:35.028443 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:05:35.029338 (+   895us) log.cc:844] Serialized 6990 byte log entry
0709 13:05:35.045773 (+ 16435us) op_driver.cc:464] REPLICATION: finished
0709 13:05:35.134812 (+ 89039us) write_op.cc:301] APPLY: starting
0709 13:05:35.134952 (+   140us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:35.200476 (+ 65524us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:35.200601 (+   125us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:35.213017 (+ 12416us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:35.213372 (+   355us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 13:05:35.213420 (+    48us) write_op.cc:312] APPLY: finished
0709 13:05:35.213847 (+   427us) log.cc:844] Serialized 164 byte log entry
0709 13:05:35.218376 (+  4529us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:35.221641 (+  3265us) write_op.cc:454] Released schema lock
0709 13:05:35.221738 (+    97us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:35.249788 (+ 28050us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:35.250702 (+   914us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":190269,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":86968,"apply.run_cpu_time_us":26120,"apply.run_wall_time_us":114980,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79295,"num_ops":18,"prepare.queue_time_us":58083,"prepare.run_cpu_time_us":6835,"prepare.run_wall_time_us":74788,"replication_time_us":17106,"thread_start_us":81548,"threads_started":2}]]}
I20260709 13:05:35.350016 16796 raft_consensus.cc:493] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
W20260709 13:05:35.357621  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:35.350991 16796 raft_consensus.cc:515] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:35.376802 16812 raft_consensus.cc:493] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.432039 16796 leader_election.cc:290] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:35.426254 16812 raft_consensus.cc:515] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:35.437196 16812 leader_election.cc:290] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:35.398123 16848 raft_consensus.cc:359] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:05:35.449122  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:35.447286 16848 raft_consensus.cc:740] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:05:35.457136  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3169f7f9f5a7440398ae5ebe01fc3c3f" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:35.450487 16848 consensus_queue.cc:260] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:05:35.462432 16848 ts_tablet_manager.cc:1434] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.277s	user 0.014s	sys 0.000s
W20260709 13:05:35.482076  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 61145.7us
I20260709 13:05:35.484124  8840 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-53cce7f107584cc28999319a785023ef received from {username='slave'} at 127.0.0.1:55168
I20260709 13:05:35.485497  8840 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-53cce7f107584cc28999319a785023ef on tablet 53cce7f107584cc28999319a785023ef with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:35.550125  8570 catalog_manager.cc:5697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 18 to 24, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 24 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.554323  8570 catalog_manager.cc:5697] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 20 to 23, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 23 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.558735  8570 catalog_manager.cc:5697] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 13 to 21, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 21 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:35.561273  8647 leader_election.cc:304] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:35.561647  8570 catalog_manager.cc:5697] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.564813  8570 catalog_manager.cc:5697] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.569881  8570 catalog_manager.cc:5697] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 9 to 20. New cstate: current_term: 20 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.573503  8570 catalog_manager.cc:5697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 15 to 17, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 17 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:35.578768  8570 catalog_manager.cc:5697] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 15 to 21, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 21 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:35.585450  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:35.601843  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cce7f107584cc28999319a785023ef" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:35.603655  8570 catalog_manager.cc:5697] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 14 to 18, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 18 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.615305  8570 catalog_manager.cc:5697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 8 to 20. New cstate: current_term: 20 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:35.617690  8570 catalog_manager.cc:5697] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 19 to 22, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 22 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:35.644556  8570 catalog_manager.cc:5697] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 21 to 26, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 26 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.653095 16549 raft_consensus.cc:2749] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:35.667079  8570 catalog_manager.cc:5697] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 17 to 18, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 18 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:35.690564 16892 raft_consensus.cc:493] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.691757 16892 raft_consensus.cc:515] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:35.712006  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:35.717505  7526 tablet_replica.cc:333] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
W20260709 13:05:35.736109  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 78652.3us
I20260709 13:05:35.759991 16871 raft_consensus.cc:493] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.760897 16871 raft_consensus.cc:515] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:35.765796 16796 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.766692 16796 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:35.773677 16871 leader_election.cc:290] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:35.769749 16796 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:35.802345  9388 raft_consensus.cc:1217] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.10->[16.11-16.11]   Dedup: 16.11->[]
I20260709 13:05:35.818727  7526 raft_consensus.cc:2243] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:35.820370  7526 raft_consensus.cc:2272] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:35.829231 16892 leader_election.cc:290] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:35.823382  8570 catalog_manager.cc:5697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.844981  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:35.855010 16892 raft_consensus.cc:493] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.855948 16892 raft_consensus.cc:515] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:35.862779  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 60881.7us
I20260709 13:05:35.864038  8645 leader_election.cc:304] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:35.885218 16812 raft_consensus.cc:2749] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:35.886118  7526 tablet_replica.cc:333] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:35.880131 16892 leader_election.cc:290] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:35.883816  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:35.893517  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 77477.4us
I20260709 13:05:35.892282  8570 catalog_manager.cc:5697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.911237  8570 catalog_manager.cc:5697] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 17 to 18, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 18 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:35.913470  8570 catalog_manager.cc:5697] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 17 to 19, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 19 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:35.920428 16892 raft_consensus.cc:493] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.924141  7526 raft_consensus.cc:2243] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:35.923946 16892 raft_consensus.cc:515] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:35.925384  7526 raft_consensus.cc:2272] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:35.927487 16892 leader_election.cc:290] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:35.929116  8570 catalog_manager.cc:5697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
W20260709 13:05:35.903110  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:35.940217  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:35.932761  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 68891.6us
I20260709 13:05:35.956575  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:35.960181  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:35.961681  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:35.968879  8570 catalog_manager.cc:5697] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 15 to 19, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 19 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:35.971482  8570 catalog_manager.cc:5697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:35.973219  7526 tablet_replica.cc:333] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:35.973726  8570 catalog_manager.cc:5697] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 11 to 22. New cstate: current_term: 22 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.977478  8570 catalog_manager.cc:5697] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 18 to 19, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 19 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.987604  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:36.006400  8753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb0366ec71a44feb8c7b426e851dcc6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:36.008497  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1148edfb8a5a41fdaced6489dfd22b63" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:36.010928  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:36.016367  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50ae323a197c4e5b994d66663473c7d2" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:36.022791 16877 log.cc:927] Time spent T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145: Append to log took a long time: real 0.062s	user 0.002s	sys 0.000s
I20260709 13:05:36.039448  7526 raft_consensus.cc:2243] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.040494  7526 pending_rounds.cc:70] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Trying to abort 1 pending ops.
I20260709 13:05:36.040877  7526 pending_rounds.cc:77] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 15 index: 9 } timestamp: 7305635126962356224 op_type: NO_OP noop_request { }
I20260709 13:05:36.041476  7526 raft_consensus.cc:2889] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:36.041970  7526 raft_consensus.cc:2272] T 53cce7f107584cc28999319a785023ef P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.049213  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "987153bedd0c4f86b726db1edfd1eec6" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:36.060035 16796 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.060884 16796 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:36.076490  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:36.089682 16892 raft_consensus.cc:493] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aff04f53d4845a6a4fa491fd2a06145)
I20260709 13:05:36.089891 16796 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:36.090642 16892 raft_consensus.cc:515] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:36.094667 16892 leader_election.cc:290] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:36.094952  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:36.103505  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:36.106084  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4de6a8f976f5458d807b777f3ac94a23" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:36.110737  8645 leader_election.cc:304] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:36.111666 16892 raft_consensus.cc:2749] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:36.112087  8645 leader_election.cc:304] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:36.112946 16892 raft_consensus.cc:2749] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:36.113400  8645 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
W20260709 13:05:36.114179  8645 raft_consensus.cc:2686] T 304eb26fa48d4971bffbe6266846243b P 0e8c59a1f0394eae8ce33b99f10efadc: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:05:36.128425  7526 tablet_replica.cc:333] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.135459  7526 raft_consensus.cc:2243] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:05:36.137220  7526 raft_consensus.cc:2272] T 3c997cc62c984d6e8962403382a451bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.137316  8645 leader_election.cc:304] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:36.139557 16892 raft_consensus.cc:2749] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:36.145004  8645 leader_election.cc:304] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:36.146023 16892 raft_consensus.cc:2749] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:36.166224  8645 leader_election.cc:304] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:36.169936 16812 raft_consensus.cc:2749] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:05:36.175845  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.196142  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc61bbf8aae94862a95ef23cde2e06e8" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:36.201069  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.203169 16877 tablet_bootstrap.cc:492] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=1 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:36.221511  8647 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:36.222963 16812 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:36.204794 16877 tablet_bootstrap.cc:492] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
W20260709 13:05:36.247138  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.250407 16877 ts_tablet_manager.cc:1403] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 1.070s	user 0.056s	sys 0.028s
I20260709 13:05:36.254724  7526 tablet_replica.cc:333] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.256011  7526 raft_consensus.cc:2243] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.257345  7526 raft_consensus.cc:2272] T 4de6a8f976f5458d807b777f3ac94a23 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.297111  7526 tablet_replica.cc:333] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
W20260709 13:05:36.312698  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.324541  7526 raft_consensus.cc:2243] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:36.326280  7526 raft_consensus.cc:2272] T abd86f4e9b544531a7cbfa0042b4fa6c P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.324622 16877 raft_consensus.cc:359] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:05:36.339903 16877 raft_consensus.cc:740] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:05:36.340826 16877 consensus_queue.cc:260] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:05:36.342008  7526 tablet_replica.cc:333] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
W20260709 13:05:36.343031  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.366935  7526 raft_consensus.cc:2243] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.374884 16829 raft_consensus.cc:1064] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:05:36.374894 16877 ts_tablet_manager.cc:1434] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.124s	user 0.010s	sys 0.001s
I20260709 13:05:36.378265  8766 tablet_copy_service.cc:342] P 0e9b0850145a4c66bfe983ffdfa93a57: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-304eb26fa48d4971bffbe6266846243b received from {username='slave'} at 127.0.0.1:35702
I20260709 13:05:36.379051  8766 tablet_copy_service.cc:434] P 0e9b0850145a4c66bfe983ffdfa93a57: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-304eb26fa48d4971bffbe6266846243b on tablet 304eb26fa48d4971bffbe6266846243b with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:36.383082  7526 raft_consensus.cc:2272] T 3a7eadf5c488447886a90612fbf1a2bb P 0e8c59a1f0394eae8ce33b99f10efadc [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:36.388645  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.382181 16829 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:36.444061  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.444734  7526 tablet_replica.cc:333] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.446095  7526 raft_consensus.cc:2243] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.451009 16812 raft_consensus.cc:493] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.451817 16812 raft_consensus.cc:515] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:36.454224 16812 leader_election.cc:290] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:36.458803  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:36.460587  7526 raft_consensus.cc:2272] T 35e078746b204711a1e15f221c16e31a P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.465409  7526 tablet_replica.cc:333] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.466955  7526 raft_consensus.cc:2243] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 29 LEADER]: Raft consensus shutting down.
W20260709 13:05:36.468199  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.468509  7526 raft_consensus.cc:2272] T 8e9173dea5bf4398a5e138cb273df07c P 0e8c59a1f0394eae8ce33b99f10efadc [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.472637  7526 tablet_replica.cc:333] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.474233  7526 raft_consensus.cc:2243] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:05:36.484895  9382 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:05:36.502562  8793 consensus_peers.cc:597] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:36.506031 16890 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:36.510887  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fcd6d0c3304abda83f416721e807c5" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:36.512583  8748 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:36.519654  9386 raft_consensus.cc:1217] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEARNER]: Deduplicated request from leader. Original: 23.14->[23.15-23.15]   Dedup: 23.15->[]
I20260709 13:05:36.528857  7526 raft_consensus.cc:2272] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:36.533464  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.538631 16812 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.539585 16829 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:36.540433  8647 leader_election.cc:304] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:36.541575  7526 tablet_replica.cc:333] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.541888 16796 raft_consensus.cc:2749] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:36.539592 16812 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:36.545478 16812 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:36.547997  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:36.551398  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:36.551693  7526 raft_consensus.cc:2243] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.552943  7526 pending_rounds.cc:70] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Trying to abort 2 pending ops.
I20260709 13:05:36.553289  7526 pending_rounds.cc:77] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 19 index: 14 } timestamp: 7305635082081288192 op_type: NO_OP noop_request { }
I20260709 13:05:36.553750  7526 raft_consensus.cc:2889] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:36.553867  8647 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:36.554137  7526 pending_rounds.cc:77] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 17 index: 13 } timestamp: 7305635027512606720 op_type: NO_OP noop_request { }
I20260709 13:05:36.558781  7526 raft_consensus.cc:2889] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:36.559334  7526 raft_consensus.cc:2272] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.559908  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:36.558800 16812 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:36.599043  7526 tablet_replica.cc:333] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.600566  7526 raft_consensus.cc:2243] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:36.601373 16812 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.602197 16812 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:36.605890  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:36.602223  7526 raft_consensus.cc:2272] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.610606  7526 tablet_replica.cc:333] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.612499 16812 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:36.613480  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.627650  7526 raft_consensus.cc:2243] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.630028  7526 raft_consensus.cc:2272] T 05559e602d4b4ea3986701fa621706a0 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.631904 16812 raft_consensus.cc:493] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.632572 16812 raft_consensus.cc:515] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:36.634697 16796 raft_consensus.cc:493] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.635557 16796 raft_consensus.cc:515] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:36.636937  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:36.638960  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f19e578ddc64100a13ab07a6b877a77" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:36.639094 16796 leader_election.cc:290] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:36.643137  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:36.639230  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4bab9153b047238bffb6abbeb83601" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:36.642467  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51388ea026cc4d098949a4a0327a35fc" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:36.646265 16812 leader_election.cc:290] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:36.648625  8645 leader_election.cc:304] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:36.649621  8647 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:36.650300  8645 leader_election.cc:304] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:36.638439  7526 tablet_replica.cc:333] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.650871 16892 raft_consensus.cc:2749] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:05:36.659204 16892 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:36.660008 16892 raft_consensus.cc:2749] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:36.671214  8719 consensus_peers.cc:597] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:36.677678  7526 raft_consensus.cc:2243] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.679015  7526 raft_consensus.cc:2272] T 1c963cd681ab4f45a5c339f9fd6e2687 P 0e8c59a1f0394eae8ce33b99f10efadc [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.687747  7526 tablet_replica.cc:333] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.688792  7526 raft_consensus.cc:2243] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:36.689946  7526 raft_consensus.cc:2272] T 614067c154284cfa9b7a01100f304cc9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:36.694386  8647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 44847.4us
W20260709 13:05:36.704866  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.714821  7526 tablet_replica.cc:333] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
W20260709 13:05:36.715794  8645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 65570.7us
I20260709 13:05:36.719403 16910 raft_consensus.cc:493] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.730053 16909 raft_consensus.cc:493] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e814ed2013244fa6be42623ea5c0c7b9)
I20260709 13:05:36.721686 16910 raft_consensus.cc:515] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:36.729023 16783 raft_consensus.cc:2955] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:36.737967  8750 raft_consensus.cc:2955] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:36.716230  7526 raft_consensus.cc:2243] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.760598 16910 leader_election.cc:290] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:36.730827 16909 raft_consensus.cc:515] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:36.764030  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:36.763175  7526 pending_rounds.cc:70] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Trying to abort 1 pending ops.
I20260709 13:05:36.765318  7526 pending_rounds.cc:77] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 22 index: 15 } timestamp: 7305635129426751488 op_type: NO_OP noop_request { }
I20260709 13:05:36.765818  7526 raft_consensus.cc:2889] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:36.766212  7526 raft_consensus.cc:2272] T 4f19e578ddc64100a13ab07a6b877a77 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.767563  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff71213103664c0bab63c42773d0880b" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:36.777268 16909 leader_election.cc:290] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:36.779760  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:36.786029  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:36.789091  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.790295  7526 tablet_replica.cc:333] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.791476  7526 raft_consensus.cc:2243] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.792584  7526 raft_consensus.cc:2272] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.811343  8645 leader_election.cc:304] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:36.813472 16910 raft_consensus.cc:2749] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:36.816437  7526 tablet_replica.cc:333] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
W20260709 13:05:36.824887  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:36.826853  8570 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43478 (request call id 260) took 13872 ms (client timeout 15000 ms). Trace:
I20260709 13:05:36.829263  7526 raft_consensus.cc:2243] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.830602  9382 raft_consensus.cc:2955] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:36.827417  8570 rpcz_store.cc:269] 0709 13:05:22.954558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.675827 (+10721269us) service_pool.cc:224] Handling call
0709 13:05:33.992909 (+317082us) spinlock_profiling.cc:243] Waited 39.3 ms on lock 0x7b54002d7dd4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2463090 00007f32e2501f3a 00007f32e82b81cc 00007f32e83b637e 00007f32e68772a4 00007f32e6877232 00007f32e6877161 00007f32e68770dc 00007f32e6876392 00007f32e083448c 00007f32e08338c6
0709 13:05:36.826760 (+2833851us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:36.070151 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:36.071005 (+   854us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:36.071064 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:05:36.071102 (+    38us) tablet.cc:662] Decoding operations
0709 13:05:36.074161 (+  3059us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:36.074264 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:36.074300 (+    36us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:05:36.142666 (+ 68366us) tablet.cc:701] Row locks acquired
0709 13:05:36.142736 (+    70us) write_op.cc:260] PREPARE: finished
0709 13:05:36.142953 (+   217us) write_op.cc:270] Start()
0709 13:05:36.143142 (+   189us) write_op.cc:276] Timestamp: P: 1783602336142875 usec, L: 0
0709 13:05:36.143185 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 13:05:36.144180 (+   995us) log.cc:844] Serialized 8599 byte log entry
0709 13:05:36.284485 (+140305us) op_driver.cc:464] REPLICATION: finished
0709 13:05:36.316998 (+ 32513us) write_op.cc:301] APPLY: starting
0709 13:05:36.317123 (+   125us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:36.374485 (+ 57362us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:36.374664 (+   179us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:36.547354 (+172690us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:36.564115 (+ 16761us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=0,mrs_lookups=0
0709 13:05:36.564173 (+    58us) write_op.cc:312] APPLY: finished
0709 13:05:36.566927 (+  2754us) log.cc:844] Serialized 196 byte log entry
0709 13:05:36.568041 (+  1114us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:36.568721 (+   680us) write_op.cc:454] Released schema lock
0709 13:05:36.568814 (+    93us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:36.624992 (+ 56178us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:36.625044 (+    52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":87637,"reported_tablets":50,"spinlock_wait_cycles":90496128,"child_traces":[["op",{"apply.queue_time_us":32185,"apply.run_cpu_time_us":34535,"apply.run_wall_time_us":307891,"cfile_cache_hit":90,"cfile_cache_hit_bytes":95999,"num_ops":22,"prepare.queue_time_us":80185,"prepare.run_cpu_time_us":7508,"prepare.run_wall_time_us":80568,"replication_time_us":141063,"thread_start_us":48486,"threads_started":2}]]}
I20260709 13:05:36.838675  7526 pending_rounds.cc:70] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Trying to abort 2 pending ops.
I20260709 13:05:36.839181  7526 pending_rounds.cc:77] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 22 index: 14 } timestamp: 7305635094408581120 op_type: NO_OP noop_request { }
I20260709 13:05:36.839649  7526 raft_consensus.cc:2889] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:36.839987  7526 pending_rounds.cc:77] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 20 index: 13 } timestamp: 7305635036274540544 op_type: NO_OP noop_request { }
I20260709 13:05:36.840329  7526 raft_consensus.cc:2889] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:36.840691  7526 raft_consensus.cc:2272] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.844480  7526 tablet_replica.cc:333] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.845522  7526 raft_consensus.cc:2243] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:36.856228  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.860868  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f9069613f4a83a64061e85b6f97da" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:36.863049  8645 leader_election.cc:304] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:36.865823 16910 raft_consensus.cc:2749] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:36.877840  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.893786  7526 raft_consensus.cc:2272] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:36.906060  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.906518  7526 tablet_replica.cc:333] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.915840  7526 raft_consensus.cc:2243] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:36.916114  8574 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40096 (request call id 145) took 36540 ms (client timeout 15000 ms). Trace:
I20260709 13:05:36.917243  7526 pending_rounds.cc:70] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Trying to abort 2 pending ops.
I20260709 13:05:36.917598  7526 pending_rounds.cc:77] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 23 index: 14 } timestamp: 7305635108441108480 op_type: NO_OP noop_request { }
I20260709 13:05:36.918040  7526 raft_consensus.cc:2889] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:36.918370  7526 pending_rounds.cc:77] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 21 index: 13 } timestamp: 7305635040817565696 op_type: NO_OP noop_request { }
I20260709 13:05:36.918767  7526 raft_consensus.cc:2889] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:36.919138  7526 raft_consensus.cc:2272] T 50ae323a197c4e5b994d66663473c7d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.914655  8571 catalog_manager.cc:5697] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 14 to 17, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 17 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.921852  8571 catalog_manager.cc:5697] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 21 to 22, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 22 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.923772  8571 catalog_manager.cc:5697] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 19 to 23, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 23 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.925577  8571 catalog_manager.cc:5697] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 20 to 21, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 21 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.927317  8571 catalog_manager.cc:5697] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
W20260709 13:05:36.916602  8574 rpcz_store.cc:269] 0709 13:05:00.376033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.257568 (+7881535us) service_pool.cc:224] Handling call
0709 13:05:36.915867 (+28658299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30532,"reported_tablets":28}
I20260709 13:05:36.931540  8571 catalog_manager.cc:5697] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.933322  8571 catalog_manager.cc:5697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 17 to 22, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 22 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:36.935118  8571 catalog_manager.cc:5697] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 20 to 21, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 21 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.938038  8571 catalog_manager.cc:5697] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 20 to 23, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 23 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:36.939740  8571 catalog_manager.cc:5697] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.941628  8571 catalog_manager.cc:5697] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 23 to 24, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 24 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.944479  8571 catalog_manager.cc:5697] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:36.946997  7526 tablet_replica.cc:333] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.960683  7526 raft_consensus.cc:2243] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.960351  8571 catalog_manager.cc:5697] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 22 to 23, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 23 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.962430  7526 pending_rounds.cc:70] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: Trying to abort 1 pending ops.
I20260709 13:05:36.962855  7526 pending_rounds.cc:77] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 20 index: 12 } timestamp: 7305635050899615744 op_type: NO_OP noop_request { }
I20260709 13:05:36.963338  7526 raft_consensus.cc:2889] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:36.962700  8571 catalog_manager.cc:5697] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 5 to 16. New cstate: current_term: 16 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.963742  7526 raft_consensus.cc:2272] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.967306  7526 tablet_replica.cc:333] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.968466  7526 raft_consensus.cc:2243] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:36.972118  7526 raft_consensus.cc:2272] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:36.964550  8571 catalog_manager.cc:5697] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 17 to 21, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 21 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.978587  8571 catalog_manager.cc:5697] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.980881 16910 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.981520 16910 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:36.983943 16910 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:36.985803  9382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:05:36.986346  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2217efcaa5fe4b729f1beef2f96e4189" candidate_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:36.989318  7526 tablet_replica.cc:333] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:36.980338  8571 catalog_manager.cc:5697] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.989595  8645 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e8c59a1f0394eae8ce33b99f10efadc; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:36.996008 16910 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:36.990326  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:36.997242  8571 catalog_manager.cc:5697] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 19 to 21, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 21 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.990598  7526 raft_consensus.cc:2243] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.004782  8571 catalog_manager.cc:5697] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 19 to 20, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 20 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.008183  7526 raft_consensus.cc:2272] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.008621  8571 catalog_manager.cc:5697] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 7 to 20, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 20 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:37.012463  7526 tablet_replica.cc:333] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.011927  8571 catalog_manager.cc:5697] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 19 to 20, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 20 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:37.013545  7526 raft_consensus.cc:2243] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.014333  7526 raft_consensus.cc:2272] T 51388ea026cc4d098949a4a0327a35fc P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.014536  8571 catalog_manager.cc:5697] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:37.025067  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:37.049226 16893 raft_consensus.cc:1064] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:05:37.068619  8571 catalog_manager.cc:5697] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 19 to 20, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 20 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:37.056728  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:37.071060  7526 tablet_replica.cc:333] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.072104  7526 raft_consensus.cc:2243] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.073479  7526 raft_consensus.cc:2272] T 844a3710898640c49df7cfb8d1c568cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.089564 16911 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.090471 16911 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:37.096365  7526 tablet_replica.cc:333] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.097525  7526 raft_consensus.cc:2243] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.098148  7526 raft_consensus.cc:2272] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.070350 16893 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 13, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:37.114730 16911 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:37.123502  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:37.101845  7526 tablet_replica.cc:333] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.125721  7526 raft_consensus.cc:2243] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.126847  7526 pending_rounds.cc:70] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Trying to abort 1 pending ops.
I20260709 13:05:37.127132  7526 pending_rounds.cc:77] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 17 index: 11 } timestamp: 7305635086277820416 op_type: NO_OP noop_request { }
I20260709 13:05:37.127595  7526 raft_consensus.cc:2889] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:37.127990  7526 raft_consensus.cc:2272] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:37.145325  8719 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:37.168617  7526 tablet_replica.cc:333] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.199906  7526 raft_consensus.cc:2243] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.200695  7526 raft_consensus.cc:2272] T 300460316b0849bf8014930d5e9bfada P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.201424  8823 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:05:37.198340  8719 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:37.163373  9382 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEARNER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:05:37.204581  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 59299.1us
I20260709 13:05:37.217442  7526 tablet_replica.cc:333] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.249203  7526 raft_consensus.cc:2243] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.250138  7526 raft_consensus.cc:2272] T 7d6f9069613f4a83a64061e85b6f97da P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.254065  7526 tablet_replica.cc:333] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.256115  7526 raft_consensus.cc:2243] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.257016  7526 raft_consensus.cc:2272] T ae08d8f5271e475baa4344843405b254 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.204396 16911 raft_consensus.cc:2749] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:05:37.261595  7526 tablet_replica.cc:333] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.262771  7526 raft_consensus.cc:2243] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.264228  7526 raft_consensus.cc:2272] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.268138  7526 tablet_replica.cc:333] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.269377  7526 raft_consensus.cc:2243] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.270089  7526 pending_rounds.cc:70] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Trying to abort 1 pending ops.
I20260709 13:05:37.270339  7526 pending_rounds.cc:77] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 16 index: 12 } timestamp: 7305635082575167488 op_type: NO_OP noop_request { }
I20260709 13:05:37.270833  7526 raft_consensus.cc:2889] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:37.271241  7526 raft_consensus.cc:2272] T 20fcd6d0c3304abda83f416721e807c5 P 0e8c59a1f0394eae8ce33b99f10efadc [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.275942 16911 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:05:37.281438  8719 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 353 similar messages]
W20260709 13:05:37.286954  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:37.287946  8719 consensus_peers.cc:597] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:37.288810  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:37.289682 16911 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:05:37.273659  8571 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43482 (request call id 234) took 7296 ms. Trace:
I20260709 13:05:37.317148  8571 rpcz_store.cc:276] 0709 13:05:29.977103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.898408 (+4921305us) service_pool.cc:224] Handling call
0709 13:05:37.273585 (+2375177us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:37.091585 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:37.092207 (+   622us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:37.092250 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:05:37.092270 (+    20us) tablet.cc:662] Decoding operations
0709 13:05:37.094113 (+  1843us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:37.094220 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:37.094246 (+    26us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:05:37.097321 (+  3075us) tablet.cc:701] Row locks acquired
0709 13:05:37.097341 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:05:37.097506 (+   165us) write_op.cc:270] Start()
0709 13:05:37.097635 (+   129us) write_op.cc:276] Timestamp: P: 1783602337097455 usec, L: 0
0709 13:05:37.097656 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:05:37.098648 (+   992us) log.cc:844] Serialized 9032 byte log entry
0709 13:05:37.110810 (+ 12162us) op_driver.cc:464] REPLICATION: finished
0709 13:05:37.111233 (+   423us) write_op.cc:301] APPLY: starting
0709 13:05:37.111336 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:37.141134 (+ 29798us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:37.141275 (+   141us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:37.228781 (+ 87506us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:37.229469 (+   688us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=0,mrs_lookups=0
0709 13:05:37.229534 (+    65us) write_op.cc:312] APPLY: finished
0709 13:05:37.230073 (+   539us) log.cc:844] Serialized 204 byte log entry
0709 13:05:37.230279 (+   206us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:37.230538 (+   259us) write_op.cc:454] Released schema lock
0709 13:05:37.233815 (+  3277us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:37.234342 (+   527us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:37.234381 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":50,"spinlock_wait_cycles":11648,"child_traces":[["op",{"apply.queue_time_us":126,"apply.run_cpu_time_us":32447,"apply.run_wall_time_us":123046,"cfile_cache_hit":92,"cfile_cache_hit_bytes":96048,"num_ops":23,"prepare.queue_time_us":11975,"prepare.run_cpu_time_us":6007,"prepare.run_wall_time_us":13537,"replication_time_us":12975,"thread_start_us":11768,"threads_started":1}]]}
I20260709 13:05:37.320278 16829 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.321027 16829 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:37.323797  7526 tablet_replica.cc:333] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.323928 16829 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:37.325093  7526 raft_consensus.cc:2243] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.325888  7526 pending_rounds.cc:70] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Trying to abort 1 pending ops.
I20260709 13:05:37.326159  7526 pending_rounds.cc:77] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 18 index: 11 } timestamp: 7305635048317657088 op_type: NO_OP noop_request { }
I20260709 13:05:37.326570  7526 raft_consensus.cc:2889] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:37.330919  7526 raft_consensus.cc:2272] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.338568  7526 tablet_replica.cc:333] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.339601  7526 raft_consensus.cc:2243] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.340256  7526 pending_rounds.cc:70] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Trying to abort 1 pending ops.
I20260709 13:05:37.340533  7526 pending_rounds.cc:77] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 18 index: 13 } timestamp: 7305635002423418880 op_type: NO_OP noop_request { }
I20260709 13:05:37.340929  7526 raft_consensus.cc:2889] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:37.341297  7526 raft_consensus.cc:2272] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.344909  7526 tablet_replica.cc:333] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.345819  7526 raft_consensus.cc:2243] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.346920  7526 raft_consensus.cc:2272] T c3796428796f4d6c8fd8e9ff2897d116 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.350307  7526 tablet_replica.cc:333] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.351331  7526 raft_consensus.cc:2243] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.352254  7526 raft_consensus.cc:2272] T 55c31a55ecab4a17acb00373b55d6f1d P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.367944  7526 tablet_replica.cc:333] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.368901  7526 raft_consensus.cc:2243] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.369899  7526 raft_consensus.cc:2272] T 142891c077a34ba289adf90a99aee612 P 0e8c59a1f0394eae8ce33b99f10efadc [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:37.371731  8719 consensus_peers.cc:597] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:37.373270  7526 tablet_replica.cc:333] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.374007  7526 raft_consensus.cc:2243] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.374703  7526 raft_consensus.cc:2272] T ac4bab9153b047238bffb6abbeb83601 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.377840  7526 tablet_replica.cc:333] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.378832  7526 raft_consensus.cc:2243] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.380271  7526 raft_consensus.cc:2272] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.384356  7526 tablet_replica.cc:333] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.385509  7526 raft_consensus.cc:2243] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.386248  7526 raft_consensus.cc:2272] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.388167  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:37.388895  8750 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 24.
I20260709 13:05:37.390568  7526 tablet_replica.cc:333] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.391673  7526 raft_consensus.cc:2243] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.392387  7526 raft_consensus.cc:2272] T 8acd9840694e4f62803bfe9810425202 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.396099  7526 tablet_replica.cc:333] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.397197  7526 raft_consensus.cc:2243] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.406890  7526 pending_rounds.cc:70] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Trying to abort 1 pending ops.
I20260709 13:05:37.407383  7526 pending_rounds.cc:77] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 20 index: 15 } timestamp: 7305634993031299072 op_type: NO_OP noop_request { }
I20260709 13:05:37.407779  7526 raft_consensus.cc:2889] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:37.408192  7526 raft_consensus.cc:2272] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e8c59a1f0394eae8ce33b99f10efadc [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.392534  8573 catalog_manager.cc:5697] T 149c2129693643678d9da7765768609c P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 12 to 18, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 18 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.410454  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:37.421057 16917 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:37.423125 16917 raft_consensus.cc:493] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.423782 16917 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:37.422363  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:37.439010  7526 tablet_replica.cc:333] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.440060  7526 raft_consensus.cc:2243] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.440728  7526 raft_consensus.cc:2272] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:37.455076  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:37.438197  8573 catalog_manager.cc:5697] T c01533e2a5204b1ebccac1133e495542 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 15 to 18, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 18 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:37.456578  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:37.463161  8573 catalog_manager.cc:5697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 15 to 19, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 19 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.464455  7526 tablet_replica.cc:333] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.495622  7526 raft_consensus.cc:2243] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.496626  7526 raft_consensus.cc:2272] T 95fdf24d372f40b8877c2792102f34cf P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.482194 16829 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.499054 16829 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:37.501701 16829 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:05:37.486788  8793 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:37.487458 16668 raft_consensus.cc:2955] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 LEADER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:37.503692  9386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:05:37.504810  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:37.489347 16902 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.510963 16902 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:37.512135  8824 raft_consensus.cc:2955] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:37.513569 16902 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:37.469993 16917 raft_consensus.cc:515] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:37.515930  7526 tablet_replica.cc:333] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.517406  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c997cc62c984d6e8962403382a451bb" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:37.518020  8750 raft_consensus.cc:3060] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:05:37.518983 16911 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.519531 16911 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:37.521876 16911 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:37.512212  9386 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 23.
I20260709 13:05:37.516239 16917 leader_election.cc:290] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 25 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:37.523682  7526 raft_consensus.cc:2243] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.525835  7526 pending_rounds.cc:70] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Trying to abort 1 pending ops.
I20260709 13:05:37.516206  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:37.526733  8796 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:37.527143  8823 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 23.
I20260709 13:05:37.528633 16829 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:37.529127 16829 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.529574 16829 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:37.529707  9355 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:37.530802 16865 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:37.531237 16865 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.531627 16865 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:05:37.532778  8719 leader_election.cc:336] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:37.533731  9352 leader_election.cc:336] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:37.526172  7526 pending_rounds.cc:77] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 16 index: 12 } timestamp: 7305635050190393344 op_type: NO_OP noop_request { }
I20260709 13:05:37.543251  7526 raft_consensus.cc:2889] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:37.543789  7526 raft_consensus.cc:2272] T 987153bedd0c4f86b726db1edfd1eec6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.547492  8573 catalog_manager.cc:5697] T 10f975b4911743768a7512f2ad3793c9 P 0e8c59a1f0394eae8ce33b99f10efadc reported cstate change: term changed from 8 to 16, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 16 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:37.554966  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:05:37.560424  8793 leader_election.cc:336] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:37.562822  8722 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:37.567288  7526 tablet_replica.cc:333] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.568711  7526 raft_consensus.cc:2243] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.570152  7526 pending_rounds.cc:70] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Trying to abort 1 pending ops.
I20260709 13:05:37.571058  8750 raft_consensus.cc:2468] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 25.
I20260709 13:05:37.572947  8794 leader_election.cc:304] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:37.573954 16917 raft_consensus.cc:2804] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:05:37.571655  7526 pending_rounds.cc:77] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc: Aborting op as it isn't in flight: id { term: 19 index: 15 } timestamp: 7305635112935186432 op_type: NO_OP noop_request { }
I20260709 13:05:37.574637 16917 raft_consensus.cc:697] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 25 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:37.574795  7526 raft_consensus.cc:2889] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:05:37.577142 16911 raft_consensus.cc:2749] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:05:37.575793 16917 consensus_queue.cc:237] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:37.578815  7526 raft_consensus.cc:2272] T 5ee1e688ccd74392befec90535c1efa4 P 0e8c59a1f0394eae8ce33b99f10efadc [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.571625 16865 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:37.591207  9352 leader_election.cc:336] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:37.592955  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:37.594305 16865 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:37.597411  7526 tablet_replica.cc:333] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.601679  9386 raft_consensus.cc:2955] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:37.603224  7526 raft_consensus.cc:2243] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.604205  7526 raft_consensus.cc:2272] T 9ca09750f1b44e018d62c21aeb12a929 P 0e8c59a1f0394eae8ce33b99f10efadc [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.606176 16829 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:37.607069 16893 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.607792  7526 tablet_replica.cc:333] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.637712 16893 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:37.638554  7526 raft_consensus.cc:2243] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.639356  7526 raft_consensus.cc:2272] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.647495 16893 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:37.649123  7526 tablet_replica.cc:333] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.651033  8824 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aff04f53d4845a6a4fa491fd2a06145 in current term 24: Already voted for candidate e814ed2013244fa6be42623ea5c0c7b9 in this term.
W20260709 13:05:37.652114  8719 leader_election.cc:336] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:37.653733  9355 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:37.655516 16865 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:05:37.659471 16829 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:05:37.662079  9388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:05:37.675947  7526 raft_consensus.cc:2243] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.679313  7526 raft_consensus.cc:2272] T ff71213103664c0bab63c42773d0880b P 0e8c59a1f0394eae8ce33b99f10efadc [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.679296  9388 raft_consensus.cc:2393] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 24: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
W20260709 13:05:37.626551  8793 leader_election.cc:336] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:37.683166  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 56663.5us
W20260709 13:05:37.683966  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:37.688071  7526 tablet_replica.cc:333] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.688961  7526 raft_consensus.cc:2243] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.699903 16829 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.703984 16829 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:37.705727 16783 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.707366 16783 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:37.708381  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:37.711102 16783 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:37.711522  8751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:37.716158  8750 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 29.
I20260709 13:05:37.706431 16829 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:37.722805  8794 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:37.724449 16890 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 13:05:37.724943 16890 raft_consensus.cc:493] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.725327 16890 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:05:37.743081 16890 raft_consensus.cc:515] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:37.756170  7526 raft_consensus.cc:2272] T fea1a7951ca9443d8f4d05bd1519d442 P 0e8c59a1f0394eae8ce33b99f10efadc [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.770234  7526 tablet_replica.cc:333] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc: stopping tablet replica
I20260709 13:05:37.771821  8574 catalog_manager.cc:5697] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 24 to 25, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 25 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:37.773732  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e9173dea5bf4398a5e138cb273df07c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:37.783624  8750 raft_consensus.cc:3060] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:05:37.770936 16890 leader_election.cc:290] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 30 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:37.787871  8574 catalog_manager.cc:5697] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 23 to 24, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 24 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:37.789791  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:37.792425  8574 catalog_manager.cc:5697] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:37.793790  7526 raft_consensus.cc:2243] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.794828  7526 raft_consensus.cc:2272] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e8c59a1f0394eae8ce33b99f10efadc [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.795307  8574 catalog_manager.cc:5697] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:37.797152  8574 catalog_manager.cc:5697] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 16 to 18, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 18 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:37.799116  8574 catalog_manager.cc:5697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:37.800433  8574 catalog_manager.cc:5697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 19 to 21, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 21 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:37.804107  8573 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43462 (request call id 272) took 17325 ms (client timeout 15000 ms). Trace:
I20260709 13:05:37.810125  8722 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:37.809136  8574 catalog_manager.cc:5697] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 18 to 21, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 21 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:37.815196  8750 raft_consensus.cc:2468] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 30.
I20260709 13:05:37.817648 16668 raft_consensus.cc:2749] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:05:37.816335  8574 catalog_manager.cc:5697] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 20 to 23, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 23 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:37.825147  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:37.804616  8573 rpcz_store.cc:269] 0709 13:05:20.478892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.212114 (+4733222us) service_pool.cc:224] Handling call
0709 13:05:37.603867 (+12391753us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:37.603900 (+    33us) write_op.cc:454] Released schema lock
0709 13:05:37.804002 (+200102us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:37.574345 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:37.575081 (+   736us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:37.575135 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:05:37.575172 (+    37us) tablet.cc:662] Decoding operations
0709 13:05:37.575921 (+   749us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:37.575992 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:37.576021 (+    29us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:05:37.576452 (+   431us) tablet.cc:701] Row locks acquired
0709 13:05:37.576476 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:05:37.576634 (+   158us) write_op.cc:270] Start()
0709 13:05:37.576757 (+   123us) write_op.cc:276] Timestamp: P: 1783602337576581 usec, L: 0
0709 13:05:37.576779 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:05:37.577565 (+   786us) log.cc:844] Serialized 1684 byte log entry
0709 13:05:37.581409 (+  3844us) op_driver.cc:464] REPLICATION: finished
0709 13:05:37.584192 (+  2783us) write_op.cc:301] APPLY: starting
0709 13:05:37.584288 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:37.591946 (+  7658us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:37.592044 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:37.594932 (+  2888us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:37.595334 (+   402us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:05:37.595413 (+    79us) write_op.cc:312] APPLY: finished
0709 13:05:37.595802 (+   389us) log.cc:844] Serialized 49 byte log entry
0709 13:05:37.596947 (+  1145us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:37.597110 (+   163us) write_op.cc:454] Released schema lock
0709 13:05:37.597190 (+    80us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":2518,"apply.run_cpu_time_us":7158,"apply.run_wall_time_us":13439,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16704,"num_ops":4,"prepare.queue_time_us":21757,"prepare.run_cpu_time_us":3602,"prepare.run_wall_time_us":17469,"replication_time_us":4467}]]}
I20260709 13:05:37.824497  8574 catalog_manager.cc:5697] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 22 to 25, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 25 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:37.865594  8574 catalog_manager.cc:5697] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 19 to 20, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 20 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:37.875068  8574 catalog_manager.cc:5697] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 26 to 29, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 29 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:37.879328  8794 leader_election.cc:304] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:37.880990 16783 raft_consensus.cc:2804] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 13:05:37.881875 16783 raft_consensus.cc:697] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 30 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:37.883261 16783 consensus_queue.cc:237] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.14, Last appended by leader: 14, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:37.876710  8574 catalog_manager.cc:5697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:37.917338  8574 catalog_manager.cc:5697] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 20 to 22, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 22 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:37.919031  8574 catalog_manager.cc:5697] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 16 to 19, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 19 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
W20260709 13:05:37.922639  8793 leader_election.cc:336] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:37.924237  8793 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:37.926774  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:37.926924 16917 raft_consensus.cc:2749] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:05:37.928820  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:37.931324  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:37.932130  8793 leader_election.cc:336] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:37.928987  8574 catalog_manager.cc:5697] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 14 to 20, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 20 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:37.934655  8574 catalog_manager.cc:5697] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 7 to 21, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 21 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:37.936411  8793 leader_election.cc:336] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:37.937534 16917 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.939119 16917 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:37.941918 16917 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:37.938203 16783 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.955618 16783 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:37.956657  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:37.957669  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:37.965220 16917 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:37.978927  8574 catalog_manager.cc:5697] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 18 to 20, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 20 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:37.981099  8793 leader_election.cc:336] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:37.984090  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:37.986016  8750 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 25.
I20260709 13:05:37.987743  8794 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:05:37.989015 16917 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:05:37.989533 16917 raft_consensus.cc:493] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:37.990005 16917 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:37.984845 16783 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:38.009028  8574 catalog_manager.cc:5697] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 17 to 20, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 20 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:38.030712  8574 catalog_manager.cc:5697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 15 to 18, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 18 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:38.047364  8574 catalog_manager.cc:5697] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:38.049788  8574 catalog_manager.cc:5697] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 18 to 24, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 24 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:38.068787 16915 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:38.069586 16915 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:38.073717 16915 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:38.082203 16917 raft_consensus.cc:515] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:38.086982  8574 catalog_manager.cc:5697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 20 to 23, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 23 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:38.090756 16917 leader_election.cc:290] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 26 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:38.086097  8719 leader_election.cc:336] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.093256  8750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e3bac87e89140ffa2dcde662c7fb621" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:38.094265  8750 raft_consensus.cc:3060] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:38.091989  8574 catalog_manager.cc:5697] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 10 to 21, leader changed from e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 21 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:38.114511  8574 catalog_manager.cc:5697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 15, term changed from 13 to 23, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 23 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.147342  8574 catalog_manager.cc:5697] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:38.160562  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:38.161419  8829 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 19.
W20260709 13:05:38.173959  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:38.176335  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:38.176476  8574 catalog_manager.cc:5697] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 12, VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 16 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:38.184548  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:05:38.192200  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 90660.1us
I20260709 13:05:38.193748 16915 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:38.195379 16915 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:38.195828 16915 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:38.197289  8751 raft_consensus.cc:1275] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:05:38.163762  8750 raft_consensus.cc:2468] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 26.
I20260709 13:05:38.214118 16890 consensus_queue.cc:1048] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:05:38.211510  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:38.215581  8793 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.216054  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:05:38.219225 16668 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:38.219887 16668 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:38.222626 16668 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:38.223809 16668 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:38.224433 16668 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:38.227854 16668 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:38.234185  8719 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:38.236889  8719 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.238018 16917 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:38.238476  8793 leader_election.cc:336] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:38.240751  8793 leader_election.cc:336] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:38.241400  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:38.254990  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:38.258798  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:38.259577  8824 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:38.258931 16915 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:38.261243  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:38.262573 16915 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:38.265556  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fbae1ebcfaf4be58c5aecd1014cea01" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:38.266304  8824 raft_consensus.cc:3060] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:05:38.271783  8719 leader_election.cc:336] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.278939  8722 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:38.280041 16915 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:38.280503 16915 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:38.280916 16915 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:38.280850  8722 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:38.282276 16893 raft_consensus.cc:2749] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:38.282998  8794 leader_election.cc:304] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:38.288615  8824 raft_consensus.cc:2468] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
I20260709 13:05:38.295390  8722 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:05:38.297590  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:38.303021 16893 raft_consensus.cc:2804] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:38.312080 16893 raft_consensus.cc:697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:38.313159 16893 consensus_queue.cc:237] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:38.314738 16924 raft_consensus.cc:2804] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:05:38.332856 16915 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:38.336205 16915 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:38.350692  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "614067c154284cfa9b7a01100f304cc9" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:38.351473  8824 raft_consensus.cc:3060] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:38.323438 16924 raft_consensus.cc:697] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 26 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:38.367434 16924 consensus_queue.cc:237] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:38.370910  8719 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.391047  8824 raft_consensus.cc:2468] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 22.
W20260709 13:05:38.417922  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:38.449031  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:38.461117  8722 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:38.462256 16929 raft_consensus.cc:2804] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:38.462885 16929 raft_consensus.cc:697] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:38.463841 16929 consensus_queue.cc:237] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:38.470767 16927 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:38.471490 16927 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:38.484233  8748 raft_consensus.cc:1275] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 30 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 29 index: 14. Preceding OpId from leader: term: 30 index: 15. (index mismatch)
I20260709 13:05:38.487612 16925 consensus_queue.cc:1048] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:05:38.490860  8793 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.504352 16927 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:38.510646 16829 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:38.511557 16829 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:38.517733  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.522482  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:38.523363  8748 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 21.
I20260709 13:05:38.532313  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
I20260709 13:05:38.520566 16829 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:05:38.533852  8793 leader_election.cc:336] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.536903 16893 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:38.548569 16893 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:38.551009  8749 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
I20260709 13:05:38.555779 16893 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:38.557818  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:38.558686  8829 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 20.
W20260709 13:05:38.560413  8719 leader_election.cc:336] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.561053  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:38.563144 16893 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:38.563623 16893 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:38.564047 16893 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:38.571713  8794 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:38.588035 16893 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:38.590950 16893 leader_election.cc:290] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:38.592736  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "149c2129693643678d9da7765768609c" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:38.593477  8829 raft_consensus.cc:3060] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:38.597136  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:38.604177  8719 leader_election.cc:336] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.612452  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:38.613504 16934 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:05:38.614027 16934 raft_consensus.cc:493] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:38.614463 16934 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:05:38.634917  8829 raft_consensus.cc:2468] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e9b0850145a4c66bfe983ffdfa93a57 in term 21.
I20260709 13:05:38.645599  8722 leader_election.cc:304] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:38.648331 16829 raft_consensus.cc:2764] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:05:38.639595 16934 raft_consensus.cc:515] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:38.688858  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:38.693219 16934 leader_election.cc:290] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:38.660302 16932 raft_consensus.cc:2804] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:38.694371 16932 raft_consensus.cc:697] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Becoming Leader. State: Replica: 0e9b0850145a4c66bfe983ffdfa93a57, State: Running, Role: LEADER
I20260709 13:05:38.695449 16932 consensus_queue.cc:237] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:38.659767  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:38.706383  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a3710898640c49df7cfb8d1c568cf" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:38.707093  8749 raft_consensus.cc:3060] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:05:38.712332  8793 leader_election.cc:336] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.716586  8749 raft_consensus.cc:2468] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 22.
I20260709 13:05:38.718636  8794 leader_election.cc:304] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:38.724182 16829 raft_consensus.cc:2804] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:05:38.725039 16829 raft_consensus.cc:697] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 22 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:38.726011 16829 consensus_queue.cc:237] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:38.794312 16932 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:38.795025 16932 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:38.797276 16932 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:38.800577  8829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:38.800968  8824 raft_consensus.cc:1275] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:05:38.805296 16893 consensus_queue.cc:1048] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:05:38.809952  8719 leader_election.cc:336] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.810561  8719 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:05:38.816390 16893 raft_consensus.cc:2749] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:38.825150  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:38.856849  8751 raft_consensus.cc:1275] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 26 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:05:38.886888 16917 consensus_queue.cc:1048] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:05:38.904369  8644 connection.cc:466] server connection from 127.0.0.1:53352 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 3463) could send its response
W20260709 13:05:38.905782  8644 connection.cc:466] server connection from 127.0.0.1:53352 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53352 (request call id 3464) could send its response
W20260709 13:05:38.906921  8644 connection.cc:466] server connection from 127.0.0.1:55676 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1956) could send its response
W20260709 13:05:38.908095  8644 connection.cc:466] server connection from 127.0.0.1:55676 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1957) could send its response
W20260709 13:05:38.909610  8644 connection.cc:466] server connection from 127.0.0.1:55676 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55676 (request call id 1958) could send its response
I20260709 13:05:38.937850  8574 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43478 (request call id 261) took 1948 ms. Trace:
W20260709 13:05:38.940295  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.949438  8574 rpcz_store.cc:276] 0709 13:05:36.988973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.989218 (+   245us) service_pool.cc:224] Handling call
0709 13:05:38.937734 (+1948516us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:38.409960 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:38.410710 (+   750us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:38.410747 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:05:38.410763 (+    16us) tablet.cc:662] Decoding operations
0709 13:05:38.414079 (+  3316us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:38.414227 (+   148us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:38.414275 (+    48us) tablet.cc:685] Acquiring locks for 27 operations
0709 13:05:38.452696 (+ 38421us) tablet.cc:701] Row locks acquired
0709 13:05:38.452727 (+    31us) write_op.cc:260] PREPARE: finished
0709 13:05:38.452976 (+   249us) write_op.cc:270] Start()
0709 13:05:38.453166 (+   190us) write_op.cc:276] Timestamp: P: 1783602338452902 usec, L: 0
0709 13:05:38.453220 (+    54us) op_driver.cc:348] REPLICATION: starting
0709 13:05:38.469497 (+ 16277us) log.cc:844] Serialized 10613 byte log entry
0709 13:05:38.486654 (+ 17157us) op_driver.cc:464] REPLICATION: finished
0709 13:05:38.564076 (+ 77422us) write_op.cc:301] APPLY: starting
0709 13:05:38.564172 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:38.724932 (+160760us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:38.725050 (+   118us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:38.775541 (+ 50491us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:38.775944 (+   403us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=0,mrs_lookups=0
0709 13:05:38.775990 (+    46us) write_op.cc:312] APPLY: finished
0709 13:05:38.776520 (+   530us) log.cc:844] Serialized 236 byte log entry
0709 13:05:38.776878 (+   358us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:38.777137 (+   259us) write_op.cc:454] Released schema lock
0709 13:05:38.777209 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:38.840077 (+ 62868us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:38.840113 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1261,"reported_tablets":50,"spinlock_wait_cycles":7424,"child_traces":[["op",{"apply.queue_time_us":77131,"apply.run_cpu_time_us":36668,"apply.run_wall_time_us":275968,"cfile_cache_hit":110,"cfile_cache_hit_bytes":116879,"num_ops":27,"prepare.queue_time_us":109083,"prepare.run_cpu_time_us":9513,"prepare.run_wall_time_us":59969,"replication_time_us":33206,"thread_start_us":159389,"threads_started":2}]]}
I20260709 13:05:39.044654 16927 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:39.045120  8829 raft_consensus.cc:1275] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:05:39.045317 16927 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:39.047750 16927 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:39.051462 16929 consensus_queue.cc:1048] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:39.053041  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" is_pre_election: true
W20260709 13:05:39.053435  8572 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43462 (request call id 270) took 66908 ms (client timeout 15000 ms). Trace:
I20260709 13:05:39.053875  8749 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 18.
W20260709 13:05:39.053828  8572 rpcz_store.cc:269] 0709 13:04:32.145354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.146516 (+  1162us) service_pool.cc:224] Handling call
0709 13:05:39.053175 (+66906659us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19241,"reported_tablets":51}
I20260709 13:05:39.057997  8794 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
W20260709 13:05:39.059430  8565 connection.cc:466] server connection from 127.0.0.1:43462 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43462 (request call id 270) could send its response
I20260709 13:05:39.067101 16917 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:39.067714 16917 raft_consensus.cc:493] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0e8c59a1f0394eae8ce33b99f10efadc)
I20260709 13:05:39.068156 16917 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:39.095450 16917 raft_consensus.cc:515] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:05:39.102288  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:39.108564 16917 leader_election.cc:290] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:39.111080  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fea1a7951ca9443d8f4d05bd1519d442" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0e9b0850145a4c66bfe983ffdfa93a57"
I20260709 13:05:39.112068  8749 raft_consensus.cc:3060] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:39.206905  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.208272  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.209102  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.215209  8575 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43482 (request call id 232) took 55052 ms (client timeout 15000 ms). Trace:
W20260709 13:05:39.215778  8575 rpcz_store.cc:269] 0709 13:04:44.163061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.163390 (+   329us) service_pool.cc:224] Handling call
0709 13:05:39.214933 (+55051543us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58613,"reported_tablets":50}
I20260709 13:05:39.225798  8824 raft_consensus.cc:1275] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:05:39.288352  8814 tablet_service.cc:1558] Processing DeleteTablet for tablet 364ad2097a2341cdb96752b9a370f9a9 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:37810
I20260709 13:05:39.301661  8749 raft_consensus.cc:2468] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 19.
I20260709 13:05:39.317325 16932 consensus_queue.cc:1048] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:05:39.339098  8749 raft_consensus.cc:1275] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 13:05:39.345139  8793 leader_election.cc:336] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:05:39.391206  8719 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:39.414523  8576 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43462 (request call id 269) took 94321 ms (client timeout 15000 ms). Trace:
W20260709 13:05:39.415304  8576 rpcz_store.cc:269] 0709 13:04:05.163380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.171353 (+  7973us) service_pool.cc:224] Handling call
0709 13:05:39.411419 (+94240066us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":287050,"reported_tablets":51}
W20260709 13:05:39.416498  8565 connection.cc:466] server connection from 127.0.0.1:43462 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43462 (request call id 269) could send its response
W20260709 13:05:39.428148  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 80080.6us
W20260709 13:05:39.441028  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 94513.4us
I20260709 13:05:39.431543 16958 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.486459  8794 leader_election.cc:304] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:39.471063 16958 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:39.493804 16958 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:39.496752 16934 raft_consensus.cc:2804] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:39.496796 16825 consensus_queue.cc:1048] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:39.501924  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:05:39.502810  8566 catalog_manager.cc:5028] TS e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): tablet 364ad2097a2341cdb96752b9a370f9a9 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:05:39.497467 16934 raft_consensus.cc:697] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:05:39.504967 16934 consensus_queue.cc:237] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:05:39.502838  8821 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 24.
I20260709 13:05:39.518479  8576 catalog_manager.cc:5697] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 19 to 20, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 20 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.526230  8576 catalog_manager.cc:5697] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 25 to 26, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 26 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
W20260709 13:05:39.543370  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 55276.5us
W20260709 13:05:39.551433  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:39.550588  8576 catalog_manager.cc:5697] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 21 to 22, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 22 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.552695  8576 catalog_manager.cc:5697] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 29 to 30, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 30 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:39.555917  8576 catalog_manager.cc:5697] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 20 to 21, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130). New cstate: current_term: 21 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:39.557826  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.563408  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.564030  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.564885  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.570935  8793 consensus_peers.cc:597] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:39.567121  8576 catalog_manager.cc:5697] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 24 to 25, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 25 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
W20260709 13:05:39.568390  8719 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:39.572582  8576 catalog_manager.cc:5697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 15 to 16, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:39.578059  8793 leader_election.cc:336] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:05:39.578472  9352 leader_election.cc:336] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:05:39.579329  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.588620  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.589254  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.590080  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.630946  8719 consensus_peers.cc:597] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:39.635854  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 132607us
W20260709 13:05:39.641297  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:39.658003  8814 tablet_service.cc:1558] Processing DeleteTablet for tablet 364ad2097a2341cdb96752b9a370f9a9 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:37810
I20260709 13:05:39.658893  9355 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
W20260709 13:05:39.664165  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.642481  8578 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43482 (request call id 231) took 84133 ms (client timeout 15000 ms). Trace:
W20260709 13:05:39.665575  8578 rpcz_store.cc:269] 0709 13:04:15.509090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.509443 (+   353us) service_pool.cc:224] Handling call
0709 13:05:39.642224 (+84132781us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8613,"reported_tablets":50}
I20260709 13:05:39.674350 16933 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:05:39.691758 16933 raft_consensus.cc:493] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.692337 16933 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:39.713972  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:39.744783  8566 catalog_manager.cc:5028] TS e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): tablet 364ad2097a2341cdb96752b9a370f9a9 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:05:39.758584 16933 raft_consensus.cc:515] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:39.769948 16933 leader_election.cc:290] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 election: Requested vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:39.772853  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd86f4e9b544531a7cbfa0042b4fa6c" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:05:39.773578  8821 raft_consensus.cc:3060] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:05:39.778198  9352 leader_election.cc:336] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:05:39.779947  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:39.791335  8821 raft_consensus.cc:2468] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 25.
I20260709 13:05:39.794348  9355 leader_election.cc:304] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc
I20260709 13:05:39.796198 16974 raft_consensus.cc:2804] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:05:39.797060 16974 raft_consensus.cc:697] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:05:39.798455 16974 consensus_queue.cc:237] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:39.858791  8719 consensus_peers.cc:597] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:39.863327  8750 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:39.874651  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:39.877980  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:39.888302  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:39.922561  9386 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:05:39.925617 16932 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:05:39.928294  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:39.949057  8821 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:05:39.964819 16973 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:39.947695  8749 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:40.002779  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:40.019446 16973 raft_consensus.cc:2955] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Committing config change with OpId 22.14: config changed from index 4 to 14, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:40.024885  7526 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 8702)
I20260709 13:05:40.079523  8821 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:05:40.082259  8821 raft_consensus.cc:2955] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 4 to 14, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:40.081629  8568 maintenance_manager.cc:419] P 06d3808b553142ef8f7a4e57845757cd: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033347
I20260709 13:05:40.083901 16929 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.003s
I20260709 13:05:40.122807  8564 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 12)
I20260709 13:05:40.125023  9386 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:05:40.129765  9386 raft_consensus.cc:2955] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 4 to 14, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:40.137478 16967 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:05:40.143613  8576 catalog_manager.cc:5697] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 21 to 22, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 22 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
W20260709 13:05:40.194902  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:40.201973  8751 raft_consensus.cc:1275] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:40.205871 16972 consensus_queue.cc:1048] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:40.207643  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:40.208345  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:40.221293  8821 raft_consensus.cc:1275] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
W20260709 13:05:40.236718  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:40.239184  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:40.242993 16933 consensus_queue.cc:1048] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.002s
I20260709 13:05:40.245198 16951 raft_consensus.cc:2955] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 LEADER]: Committing config change with OpId 23.17: config changed from index 16 to 17, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:40.250732  8821 raft_consensus.cc:2955] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Committing config change with OpId 23.17: config changed from index 16 to 17, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:40.277139  9386 raft_consensus.cc:2955] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Committing config change with OpId 23.17: config changed from index 16 to 17, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:40.292148  8577 catalog_manager.cc:5697] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 20 to 23, leader changed from 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) to 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129). New cstate: current_term: 23 leader_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:40.305083  8577 catalog_manager.cc:5697] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:40.397552  8579 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43482 (request call id 230) took 107295 ms (client timeout 15000 ms). Trace:
W20260709 13:05:40.398108  8579 rpcz_store.cc:269] 0709 13:03:53.102264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.102534 (+   270us) service_pool.cc:224] Handling call
0709 13:05:40.397258 (+107294724us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18789,"reported_tablets":50}
I20260709 13:05:40.367501  8814 tablet_service.cc:1558] Processing DeleteTablet for tablet 364ad2097a2341cdb96752b9a370f9a9 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:37810
W20260709 13:05:40.464937  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:40.479059  8576 catalog_manager.cc:5697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 4 to 14, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New cstate: current_term: 22 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.483487  8576 catalog_manager.cc:5697] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: term changed from 18 to 19, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 19 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } }
I20260709 13:05:40.506858  8564 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 304eb26fa48d4971bffbe6266846243b with cas_config_opid_index 16: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:40.540670  8577 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40096 (request call id 147) took 4995 ms. Trace:
I20260709 13:05:40.548727  8577 rpcz_store.cc:276] 0709 13:05:35.537777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.543213 (+  5436us) service_pool.cc:224] Handling call
0709 13:05:40.532305 (+4989092us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:40.394947 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:40.454940 (+ 59993us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:40.454982 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:05:40.455001 (+    19us) tablet.cc:662] Decoding operations
0709 13:05:40.455459 (+   458us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:40.455522 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:40.455552 (+    30us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:05:40.455945 (+   393us) tablet.cc:701] Row locks acquired
0709 13:05:40.455964 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:05:40.456119 (+   155us) write_op.cc:270] Start()
0709 13:05:40.456248 (+   129us) write_op.cc:276] Timestamp: P: 1783602340456060 usec, L: 0
0709 13:05:40.456273 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:05:40.456970 (+   697us) log.cc:844] Serialized 913 byte log entry
0709 13:05:40.459190 (+  2220us) op_driver.cc:464] REPLICATION: finished
0709 13:05:40.459684 (+   494us) write_op.cc:301] APPLY: starting
0709 13:05:40.459781 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:40.461303 (+  1522us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:40.461411 (+   108us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:40.462983 (+  1572us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:40.463199 (+   216us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:05:40.463242 (+    43us) write_op.cc:312] APPLY: finished
0709 13:05:40.463480 (+   238us) log.cc:844] Serialized 33 byte log entry
0709 13:05:40.463669 (+   189us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:40.463779 (+   110us) write_op.cc:454] Released schema lock
0709 13:05:40.463831 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:40.464290 (+   459us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:40.464318 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":88719,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":272,"apply.run_cpu_time_us":4553,"apply.run_wall_time_us":4553,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8352,"num_ops":2,"prepare.queue_time_us":206,"prepare.run_cpu_time_us":3606,"prepare.run_wall_time_us":96756,"replication_time_us":2743}]]}
W20260709 13:05:40.588128  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:40.603725  8566 catalog_manager.cc:5028] TS e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): tablet 364ad2097a2341cdb96752b9a370f9a9 (table test-workload [id=b1168ffe73d34cd9ab7768d0032f3ae6]) successfully deleted
I20260709 13:05:40.611850  8567 maintenance_manager.cc:643] P 06d3808b553142ef8f7a4e57845757cd: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.523s	user 0.039s	sys 0.010s Metrics: {"bytes_written":6643,"delete_count":0,"lbm_write_time_us":3501,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":5,"reinsert_count":0,"update_count":265}
I20260709 13:05:40.614538  8568 maintenance_manager.cc:419] P 06d3808b553142ef8f7a4e57845757cd: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.291206
I20260709 13:05:40.650765  8576 catalog_manager.cc:5697] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 16 to 17, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New cstate: current_term: 23 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.699406  8571 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43482 (request call id 235) took 2882 ms. Trace:
I20260709 13:05:40.702505  8571 rpcz_store.cc:276] 0709 13:05:37.817369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.818275 (+   906us) service_pool.cc:224] Handling call
0709 13:05:40.699289 (+2881014us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4428,"reported_tablets":40}
W20260709 13:05:40.716938  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:40.744133  8824 consensus_queue.cc:237] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:40.835412  9388 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:40.837141 17003 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:40.841995  8749 raft_consensus.cc:1275] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:40.844775 16927 consensus_queue.cc:1048] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:05:40.855798  8563 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 231735us
W20260709 13:05:40.890664  8563 catalog_manager.cc:4755] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:05:40.927580 17002 raft_consensus.cc:2955] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Committing config change with OpId 16.13: config changed from index 12 to 13, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:40.931697  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:40.932756  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:40.934046  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:40.934991  9388 raft_consensus.cc:2955] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:40.951838  8749 raft_consensus.cc:2955] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:40.956192  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:40.957193  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:40.982872  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 56993.2us
I20260709 13:05:41.008486  8577 catalog_manager.cc:5697] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index 12 to 13, VOTER 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) evicted. New cstate: current_term: 16 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:41.016331  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:41.016952  8576 catalog_manager.cc:5697] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 23 to 25, leader changed from 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 25 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:41.025746  7526 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:41.046470  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 62342.9us
I20260709 13:05:41.067847  8566 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 53cce7f107584cc28999319a785023ef with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:05:41.082355  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:41.085181  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:41.099928  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:41.236523  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 108119us
W20260709 13:05:41.298918  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:41.306725  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:41.318681  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:41.327550  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:41.355449  8567 maintenance_manager.cc:643] P 06d3808b553142ef8f7a4e57845757cd: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.740s	user 0.097s	sys 0.016s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":26897,"cfile_init":3,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"lbm_read_time_us":2395,"lbm_reads_lt_1ms":19,"lbm_write_time_us":2021,"lbm_writes_lt_1ms":11,"peak_mem_usage":293636,"reinsert_count":0,"thread_start_us":51664,"threads_started":2,"update_count":265}
W20260709 13:05:41.365296  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:41.416730  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:41.618599  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:41.680464  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:41.729914  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:41.835906  8719 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:41.844992  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:41.867342  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:41.880745  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:41.925155  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:41.932794  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:41.952702  8719 consensus_peers.cc:597] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:42.026321  7526 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:42.028546  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:42.040979  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:42.051412  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:42.064735  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:42.129174  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:42.150297  8719 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:42.157996  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:42.194876  8563 catalog_manager.cc:4755] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:05:42.250887  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:42.258177  8821 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
W20260709 13:05:42.285200  9352 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111) [suppressed 495 similar messages]
W20260709 13:05:42.312193  8796 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 10f975b4911743768a7512f2ad3793c9. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:42.332167  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:42.339395  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:42.341564  8751 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:05:42.344749 16959 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:05:42.402164  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:42.452626 16959 raft_consensus.cc:2955] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index -1 to 13, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:05:42.460613  8749 raft_consensus.cc:2955] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index -1 to 13, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
W20260709 13:05:42.462651  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:42.470089  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:42.487355  8750 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
W20260709 13:05:42.500797  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:42.508483  8571 catalog_manager.cc:5697] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 13, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 19 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
W20260709 13:05:42.526515  8722 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 300460316b0849bf8014930d5e9bfada. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.529230  8821 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:05:42.531764 16995 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:42.536511  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:42.582387  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:42.596984  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:42.598176 16995 raft_consensus.cc:2955] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:05:42.604417  8821 raft_consensus.cc:2955] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
W20260709 13:05:42.628883  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:42.691639  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:42.696724  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:42.702363  8821 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
W20260709 13:05:42.734862  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 46797.2us
I20260709 13:05:42.759469  8571 catalog_manager.cc:5697] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 reported cstate change: config changed from index -1 to 12, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 20 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:42.794236  8796 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7fb0366ec71a44feb8c7b426e851dcc6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.799470  8751 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 13:05:42.804360  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.805797 16959 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:42.822167 16959 raft_consensus.cc:2955] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 LEADER]: Committing config change with OpId 23.14: config changed from index -1 to 14, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:05:42.824645  8751 raft_consensus.cc:2955] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Committing config change with OpId 23.14: config changed from index -1 to 14, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
W20260709 13:05:42.836090  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:42.856803  8821 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:05:42.859371  8576 catalog_manager.cc:5697] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index -1 to 14, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 23 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.885057 17039 ts_tablet_manager.cc:933] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:42.927791  8796 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749): Couldn't send request to peer 7aff04f53d4845a6a4fa491fd2a06145. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3524354c9db64c0c9b532bef2a0ceb6f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.930512  8750 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:05:42.935704 17030 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:05:42.979133 17039 tablet_copy_client.cc:323] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:05:42.988790  8840 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet 10f975b4911743768a7512f2ad3793c9 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
W20260709 13:05:42.989523  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.989687  8840 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet 10f975b4911743768a7512f2ad3793c9 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-10f975b4911743768a7512f2ad3793c9
W20260709 13:05:42.990540  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:43.013053  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:43.016906 16927 raft_consensus.cc:2955] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Committing config change with OpId 21.16: config changed from index -1 to 16, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
W20260709 13:05:43.026576  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:43.027602  7526 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 8702)
I20260709 13:05:43.053958  8750 raft_consensus.cc:2955] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index -1 to 16, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } } }
I20260709 13:05:43.056978 17043 ts_tablet_manager.cc:933] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:05:43.079912 17043 tablet_copy_client.cc:323] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.130:42321
I20260709 13:05:43.081010  8840 tablet_copy_source_session.cc:215] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:43.084807  8766 tablet_copy_service.cc:140] P 0e9b0850145a4c66bfe983ffdfa93a57: Received BeginTabletCopySession request for tablet 300460316b0849bf8014930d5e9bfada from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:35702)
I20260709 13:05:43.088214  8766 tablet_copy_service.cc:161] P 0e9b0850145a4c66bfe983ffdfa93a57: Beginning new tablet copy session on tablet 300460316b0849bf8014930d5e9bfada from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:35702: session id = 7aff04f53d4845a6a4fa491fd2a06145-300460316b0849bf8014930d5e9bfada
I20260709 13:05:43.091955 17039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10f975b4911743768a7512f2ad3793c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:43.115150  8766 tablet_copy_source_session.cc:215] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:05:43.115936  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:43.150429 17039 tablet_copy_client.cc:806] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:43.149775  8576 catalog_manager.cc:5697] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index -1 to 16, NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) added. New cstate: current_term: 21 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:43.151284 17039 tablet_copy_client.cc:670] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:43.157296 17043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 300460316b0849bf8014930d5e9bfada. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:43.186231  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:43.198782 17039 tablet_copy_client.cc:538] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:05:43.206990  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:43.237363  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:43.251456  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:43.260368  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:43.313215  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 53678.9us
W20260709 13:05:43.362082  8563 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 90861.8us
I20260709 13:05:43.383144 17039 tablet_bootstrap.cc:492] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
W20260709 13:05:43.436950  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50326.1us
I20260709 13:05:43.437443 17043 tablet_copy_client.cc:806] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
W20260709 13:05:43.466879  8563 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 61780.2us
W20260709 13:05:43.470584  8563 catalog_manager.cc:4755] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:05:43.473304 17043 tablet_copy_client.cc:670] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:43.529800 17061 ts_tablet_manager.cc:933] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:05:43.540733 17062 ts_tablet_manager.cc:933] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145: Initiating tablet copy from peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:05:43.573827  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 55190.6us
I20260709 13:05:43.615193 17043 tablet_copy_client.cc:538] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:05:43.623351  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:43.634145 17043 tablet_bootstrap.cc:492] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
I20260709 13:05:43.639174 17061 tablet_copy_client.cc:323] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:05:43.656386 17062 tablet_copy_client.cc:323] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Beginning tablet copy session from remote peer at address 127.7.89.131:37857
I20260709 13:05:43.661485  8840 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet 7fb0366ec71a44feb8c7b426e851dcc6 from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
I20260709 13:05:43.662058  8839 tablet_copy_service.cc:140] P e814ed2013244fa6be42623ea5c0c7b9: Received BeginTabletCopySession request for tablet 3524354c9db64c0c9b532bef2a0ceb6f from peer 7aff04f53d4845a6a4fa491fd2a06145 ({username='slave'} at 127.0.0.1:55168)
I20260709 13:05:43.662179  8840 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet 7fb0366ec71a44feb8c7b426e851dcc6 from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-7fb0366ec71a44feb8c7b426e851dcc6
I20260709 13:05:43.666869  8839 tablet_copy_service.cc:161] P e814ed2013244fa6be42623ea5c0c7b9: Beginning new tablet copy session on tablet 3524354c9db64c0c9b532bef2a0ceb6f from peer 7aff04f53d4845a6a4fa491fd2a06145 at {username='slave'} at 127.0.0.1:55168: session id = 7aff04f53d4845a6a4fa491fd2a06145-3524354c9db64c0c9b532bef2a0ceb6f
W20260709 13:05:43.657436  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:43.690546  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:43.693122  8840 tablet_copy_source_session.cc:215] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:43.699698 17061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fb0366ec71a44feb8c7b426e851dcc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:43.731729 17061 tablet_copy_client.cc:806] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:43.732595 17061 tablet_copy_client.cc:670] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:43.738736 17061 tablet_copy_client.cc:538] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:43.739215  8839 tablet_copy_source_session.cc:215] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:43.749770 17062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3524354c9db64c0c9b532bef2a0ceb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:43.758850 17061 tablet_bootstrap.cc:492] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
W20260709 13:05:43.759990  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:43.792394 17062 tablet_copy_client.cc:806] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:43.793244 17062 tablet_copy_client.cc:670] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:43.800473 17062 tablet_copy_client.cc:538] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:43.858456 17062 tablet_bootstrap.cc:492] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap starting.
W20260709 13:05:43.921540  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:43.930847  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:44.018440  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:44.030282 17039 tablet_bootstrap.cc:492] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=1 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:05:44.035271  7526 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 8702)
I20260709 13:05:44.045469 17039 tablet_bootstrap.cc:492] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:05:44.072392 17039 ts_tablet_manager.cc:1403] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.704s	user 0.042s	sys 0.024s
W20260709 13:05:44.098554  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:44.127969  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:44.143082 17039 raft_consensus.cc:359] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:05:44.144315 17039 raft_consensus.cc:740] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:05:44.149094 17039 consensus_queue.cc:260] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:05:44.150485 17043 tablet_bootstrap.cc:492] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=1 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:44.162746 17043 tablet_bootstrap.cc:492] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:05:44.164176 17039 ts_tablet_manager.cc:1434] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.086s	user 0.010s	sys 0.000s
I20260709 13:05:44.182786 17043 ts_tablet_manager.cc:1403] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.549s	user 0.060s	sys 0.010s
I20260709 13:05:44.199584  8839 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-10f975b4911743768a7512f2ad3793c9 received from {username='slave'} at 127.0.0.1:55168
I20260709 13:05:44.200184  8839 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-10f975b4911743768a7512f2ad3793c9 on tablet 10f975b4911743768a7512f2ad3793c9 with peer 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:44.246358  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:44.267994 17061 tablet_bootstrap.cc:492] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:44.228626 17043 raft_consensus.cc:359] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:05:44.269181 17043 raft_consensus.cc:740] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:05:44.270153 17043 consensus_queue.cc:260] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:05:44.274166 17043 ts_tablet_manager.cc:1434] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.000s
I20260709 13:05:44.276782  8766 tablet_copy_service.cc:342] P 0e9b0850145a4c66bfe983ffdfa93a57: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-300460316b0849bf8014930d5e9bfada received from {username='slave'} at 127.0.0.1:35702
I20260709 13:05:44.277309  8766 tablet_copy_service.cc:434] P 0e9b0850145a4c66bfe983ffdfa93a57: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-300460316b0849bf8014930d5e9bfada on tablet 300460316b0849bf8014930d5e9bfada with peer 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:44.292095  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:44.311404 17061 tablet_bootstrap.cc:492] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:05:44.312469 17061 ts_tablet_manager.cc:1403] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.554s	user 0.064s	sys 0.017s
I20260709 13:05:44.327725 17062 tablet_bootstrap.cc:492] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=1 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:44.329167 17062 tablet_bootstrap.cc:492] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145: Bootstrap complete.
I20260709 13:05:44.330313 17062 ts_tablet_manager.cc:1403] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145: Time spent bootstrapping tablet: real 0.472s	user 0.054s	sys 0.016s
I20260709 13:05:44.328647 17061 raft_consensus.cc:359] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:05:44.332242 17061 raft_consensus.cc:740] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:05:44.334031 17061 consensus_queue.cc:260] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
I20260709 13:05:44.338310 17061 ts_tablet_manager.cc:1434] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.004s
I20260709 13:05:44.340819 17062 raft_consensus.cc:359] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
W20260709 13:05:44.365912  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:44.365954 17062 raft_consensus.cc:740] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Initialized, Role: LEARNER
I20260709 13:05:44.368712 17062 consensus_queue.cc:260] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: NON_VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: true } }
W20260709 13:05:44.371184  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:44.371366  8839 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-7fb0366ec71a44feb8c7b426e851dcc6 received from {username='slave'} at 127.0.0.1:55168
I20260709 13:05:44.372040  8839 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-7fb0366ec71a44feb8c7b426e851dcc6 on tablet 7fb0366ec71a44feb8c7b426e851dcc6 with peer 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:05:44.372958 17062 ts_tablet_manager.cc:1434] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.001s
I20260709 13:05:44.379243  8839 tablet_copy_service.cc:342] P e814ed2013244fa6be42623ea5c0c7b9: Request end of tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-3524354c9db64c0c9b532bef2a0ceb6f received from {username='slave'} at 127.0.0.1:55168
I20260709 13:05:44.379810  8839 tablet_copy_service.cc:434] P e814ed2013244fa6be42623ea5c0c7b9: ending tablet copy session 7aff04f53d4845a6a4fa491fd2a06145-3524354c9db64c0c9b532bef2a0ceb6f on tablet 3524354c9db64c0c9b532bef2a0ceb6f with peer 7aff04f53d4845a6a4fa491fd2a06145
W20260709 13:05:44.385879  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:44.396278  9388 raft_consensus.cc:1217] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEARNER]: Deduplicated request from leader. Original: 19.12->[19.13-19.13]   Dedup: 19.13->[]
I20260709 13:05:44.436937  9388 raft_consensus.cc:1217] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 LEARNER]: Deduplicated request from leader. Original: 21.15->[21.16-21.16]   Dedup: 21.16->[]
W20260709 13:05:44.438585  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:44.460388  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:44.489234  8719 consensus_peers.cc:597] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:44.518846  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:44.646910  8719 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:44.658906  9386 raft_consensus.cc:1217] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.11->[20.12-20.12]   Dedup: 20.12->[]
W20260709 13:05:44.693523  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:44.726227  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:44.776934  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:44.804806  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:44.805788  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:44.811295  9386 raft_consensus.cc:1217] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEARNER]: Deduplicated request from leader. Original: 23.13->[23.14-23.14]   Dedup: 23.14->[]
W20260709 13:05:44.894243  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:44.902211  8719 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:44.977646 17072 raft_consensus.cc:1064] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:05:44.980777 17072 consensus_queue.cc:237] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 14, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:44.986899  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:45.015837  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:45.015837 17056 raft_consensus.cc:1064] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:05:45.027379 17056 consensus_queue.cc:237] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:45.042389  7526 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 8702)
I20260709 13:05:45.042439  9386 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:05:45.043641  8750 raft_consensus.cc:1275] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:05:45.049544  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:45.051297 17053 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.010s
I20260709 13:05:45.061518 17059 consensus_queue.cc:1048] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:05:45.126063  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.146654  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:45.185125  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:45.194906  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:45.199663  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:45.204792  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:45.267966  8749 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:05:45.271570  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 67032.6us
I20260709 13:05:45.275749  9386 raft_consensus.cc:1275] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:05:45.292196 17082 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:05:45.298828  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.301386 17082 raft_consensus.cc:2955] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Committing config change with OpId 21.17: config changed from index 16 to 17, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:45.309808 17085 raft_consensus.cc:1064] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:05:45.312811  8748 raft_consensus.cc:2955] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Committing config change with OpId 21.17: config changed from index 16 to 17, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:45.316186  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:45.320214 17085 consensus_queue.cc:237] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 10, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:05:45.327396 17081 consensus_queue.cc:1048] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:05:45.341452 17086 raft_consensus.cc:1064] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: attempting to promote NON_VOTER 7aff04f53d4845a6a4fa491fd2a06145 to VOTER
I20260709 13:05:45.344626 17086 consensus_queue.cc:237] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 12, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:05:45.377950  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:45.419495  9386 raft_consensus.cc:2955] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Committing config change with OpId 21.17: config changed from index 16 to 17, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:45.433737  9382 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEARNER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:45.436076 17084 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:45.451645  8576 catalog_manager.cc:5697] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index 16 to 17, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:45.459048  8829 raft_consensus.cc:1275] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Refusing update from remote peer 0e9b0850145a4c66bfe983ffdfa93a57: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:05:45.461874 17085 consensus_queue.cc:1048] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:05:45.485704  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:45.487248 17059 raft_consensus.cc:2955] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Committing config change with OpId 19.14: config changed from index 13 to 14, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:45.489358  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.501823  9388 raft_consensus.cc:2955] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:45.517995  8829 raft_consensus.cc:2955] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:45.514581 17085 raft_consensus.cc:2955] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index 12 to 13, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:45.539574  9382 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEARNER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 13:05:45.539608  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:45.542514 17053 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:45.550973  8748 raft_consensus.cc:1275] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:05:45.564254 17081 consensus_queue.cc:1048] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:05:45.567469  8748 raft_consensus.cc:2955] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:45.599006  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.612103  9382 raft_consensus.cc:2955] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:45.617738  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:45.622740  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:45.658366 17072 raft_consensus.cc:2955] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 LEADER]: Committing config change with OpId 23.15: config changed from index 14 to 15, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:45.682819  8756 raft_consensus.cc:2955] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index 14 to 15, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:45.686774  8563 catalog_manager.cc:4755] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 53cce7f107584cc28999319a785023ef: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:05:45.695600  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:45.740866  8571 catalog_manager.cc:5697] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index 13 to 14, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:45.757313  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:45.757987  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:45.758491  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:45.758679  8571 catalog_manager.cc:5697] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: config changed from index 12 to 13, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
I20260709 13:05:45.781253  9382 raft_consensus.cc:2955] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index 14 to 15, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:45.829032  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:45.851547  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:46.039417  8577 catalog_manager.cc:5697] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: config changed from index 14 to 15, 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } attrs { replace: true } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } } }
W20260709 13:05:46.044247  7526 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:46.082465  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 47789.9us
W20260709 13:05:46.205775  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.264300  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.294728  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:46.320459  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:46.350169  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.362231  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.455348  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:46.666316  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.670874  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.749087  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.761327  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.770130  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.806340  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.874763  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.899078  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.943354  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:46.963425  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.044932  7526 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:47.118824  8719 consensus_peers.cc:597] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:47.289525  8719 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111) [suppressed 480 similar messages]
W20260709 13:05:47.291680  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.338239  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.340118  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.343693  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.396891  8719 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:47.444191  8719 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:47.475055  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.556160  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:47.609956  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:47.631870  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.683697  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.702001  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:47.703426  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.729462  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.773774  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.792145  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:47.830138  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:47.890457  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:47.964283  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:48.022347  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:48.045549  7526 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:48.061574  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:48.146466  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:48.158030  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:48.184353  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:48.347687  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:48.428040  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:48.442962  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:48.462668  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:48.481978  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:48.539518  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:48.566304  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:48.630100  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:48.705176  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:48.758205  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:48.790931  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:49.046882  7526 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:49.169210  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:49.197630  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:49.217736  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.216539  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.302404  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.347385  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.366528  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.378934  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.460265  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.517203  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.587224  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.602427  8719 consensus_peers.cc:597] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:49.621618  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.752619  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.790783  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.871768  8563 catalog_manager.cc:4755] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 53cce7f107584cc28999319a785023ef: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:05:49.962133  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:49.985591  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.016152  8719 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:50.039988  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.047529  7526 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:50.079035  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.090540  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:50.094108  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:50.115742  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.136361  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.146108  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.149161  8719 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:50.236846  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:50.280191  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.323925  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:50.345358  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.397811  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.440629  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 69285.3us
W20260709 13:05:50.456544  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:50.469492  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.480031  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:50.658027  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:50.670378  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.688086  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.728432  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.775033  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:50.775813  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:50.822479  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:50.946455  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:51.021257  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 134151us
W20260709 13:05:51.036511  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:51.048195  7526 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:51.161896  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:51.266144  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:51.281709  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:51.351356  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:51.515898  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:51.574572  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:51.605633  8719 consensus_peers.cc:597] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:51.670467  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:51.848819  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:51.875614  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:51.917394  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:51.919996  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:51.997114  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.048801  7526 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:52.047434  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.183977  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.223507  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 72445.1us
W20260709 13:05:52.240775  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.329782  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.341442  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 63791.7us
W20260709 13:05:52.356850  9352 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111) [suppressed 464 similar messages]
W20260709 13:05:52.382344  8719 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:52.416514  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.422020  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.461666  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:52.477761  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.539314  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.540766  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.544430  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.578209  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.589342  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:52.591621  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.656026  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.673156  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:52.751739  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:52.770177  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:52.775264  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:52.786140  8719 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:52.882851  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.049407  7526 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:53.099048  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.107084  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:53.120498  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:53.183466  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.207267  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.209041  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.213229  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.233960  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.278165  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:53.513463  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.626212  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:53.626833  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:53.692006  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.829752  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.946746  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:53.947189  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:53.959772  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:53.972391  8719 consensus_peers.cc:597] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:54.050864  7526 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:54.105224  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:54.142445  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:54.176062  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:54.389707  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:54.403276  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:54.509191  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:54.568783  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:54.615811  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:54.618958  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:54.646148  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:54.659178  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:54.667482  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:54.744262  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:54.880429  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:55.020742  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:55.051440  7526 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:55.064548  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:55.089485  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:55.128532  8719 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:55.215065  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:55.268188  8719 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:55.323817  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:55.417366  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:55.483034  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:55.498656  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:55.504254  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:55.544981  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:55.546257  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:55.546917  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:55.547441  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:55.547977  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:55.559088  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:55.582461  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:55.640035  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:55.730185  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:55.741770  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:56.097146  7526 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:56.162792  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:56.447008  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 47919.5us
W20260709 13:05:56.456374  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:56.502326  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:56.548111  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 120137us
W20260709 13:05:56.578668  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:56.833333  8719 consensus_peers.cc:597] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:56.847482  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:56.850152  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:56.905403  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:56.906550  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:56.907284  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:56.908263  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:56.929433  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:56.940953  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:56.957019  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:57.043251  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:57.053280  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:57.098441  7526 thread.cc:527] Waited for 18000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:57.129415  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:57.145222  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:57.255795  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:57.275473  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:57.297142  8851 debug-util.cc:398] Leaking SignalData structure 0x7b080036a1a0 after lost signal to thread 7529
W20260709 13:05:57.299110  8851 debug-util.cc:398] Leaking SignalData structure 0x7b08001e8780 after lost signal to thread 8613
W20260709 13:05:57.300927  8851 debug-util.cc:398] Leaking SignalData structure 0x7b08001dce80 after lost signal to thread 8780
W20260709 13:05:57.302412  8851 debug-util.cc:398] Leaking SignalData structure 0x7b08000cec60 after lost signal to thread 8854
W20260709 13:05:57.306488  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:57.307935  8851 debug-util.cc:398] Leaking SignalData structure 0x7b080016fe60 after lost signal to thread 9413
W20260709 13:05:57.366525  9352 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111) [suppressed 446 similar messages]
W20260709 13:05:57.413023  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:57.424726  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:57.468021  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:57.579854  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:57.650221  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:57.652387  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:57.758605  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:57.816871  8719 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:57.884553  8719 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:57.920944  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:57.946287  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:57.980019  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:58.017379  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:58.100436  7526 thread.cc:527] Waited for 19000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:58.192575  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:58.200752  8563 catalog_manager.cc:4755] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 304eb26fa48d4971bffbe6266846243b: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:05:58.211637  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:58.212383  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:58.219197  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:58.259626  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:58.284623  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:58.298509  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:58.445788  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:58.446530  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:58.459451  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:58.531049  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:58.542796  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:58.550211  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:58.711994  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:58.856861  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:58.960220  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:58.984180  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:58.986308  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:59.014149  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:59.099413  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:59.101245  7526 thread.cc:527] Waited for 20000ms trying to join with diag-logger (tid 8702)
W20260709 13:05:59.172765  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:59.186972  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:59.261229  8719 consensus_peers.cc:597] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:59.332151  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 47344us
W20260709 13:05:59.408349  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:59.409127  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:59.503305  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:59.547348  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:59.595827  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:59.597322  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:59.671248  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:59.683286  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:59.683975  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:59.693572  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 47677.5us
W20260709 13:05:59.848987  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 73244us
W20260709 13:05:59.868598  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:59.870182  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:59.869215  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:59.903514  8796 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 191553us
W20260709 13:05:59.904860  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:00.005769  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:00.022169  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:00.031046  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:00.102293  7526 thread.cc:527] Waited for 21000ms trying to join with diag-logger (tid 8702)
W20260709 13:06:00.123577  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:00.163419  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:00.175976  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:00.256084  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:00.280997  8719 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:00.303263  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:00.344347  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:00.364814  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:00.525797  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:00.541450  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:00.583976  8719 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:00.675788  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:00.747797  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:00.809809  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:00.837636  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:00.838361  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:00.886873  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:00.888018  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:00.893932  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:00.995688  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:01.080865  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:01.094322  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:01.106700  7526 thread.cc:527] Waited for 22000ms trying to join with diag-logger (tid 8702)
W20260709 13:06:01.205448  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:01.447491  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:01.492285  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:01.501905  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:01.531261  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:01.514436  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:01.559121  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 44748.5us
W20260709 13:06:01.577466  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:01.638950  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:01.814224  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:01.851393  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:01.852541  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:01.906900  8719 consensus_peers.cc:597] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:02.057271  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:02.107307  7526 thread.cc:527] Waited for 23000ms trying to join with diag-logger (tid 8702)
W20260709 13:06:02.110481  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:02.141664  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:02.151611  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:02.150487  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:02.186764  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:02.227669  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:02.300599  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 56356.2us
W20260709 13:06:02.325722  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:02.344231  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:02.369663  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:02.370674  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:02.364979  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:02.383396  8793 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111) [suppressed 487 similar messages]
W20260709 13:06:02.418840  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:02.462113  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:02.599506  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:02.667464  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:02.675125  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:02.736955  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:02.759795  8719 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:02.997357  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:03.012807  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:03.016219  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:03.018537  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:03.045322  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:03.099881  8719 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:03.100603  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:03.103832  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:03.107880  7526 thread.cc:527] Waited for 24000ms trying to join with diag-logger (tid 8702)
W20260709 13:06:03.222568  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:03.247449  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:03.275941  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:03.298875  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:03.361536  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:03.491127  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:03.491263  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:03.712713  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:03.741326  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:03.742769  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:03.769631  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:03.770533  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:03.978566  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:04.028996  8793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 50495.8us
W20260709 13:06:03.998981  8719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 104066us
W20260709 13:06:04.108621  7526 thread.cc:527] Waited for 25000ms trying to join with diag-logger (tid 8702)
W20260709 13:06:04.107791  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:04.293046  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:04.309042  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:04.369268  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:04.370461  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:04.371047  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:04.426705  8719 consensus_peers.cc:597] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:04.434021  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:04.554960  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:04.562211  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:04.587495  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:04.615880  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:04.646647  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:04.673557  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:04.729152  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:04.730353  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:04.764359  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:04.783839  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:04.842841  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:04.850699  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:04.866488  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:04.909729  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:05.109160  7526 thread.cc:527] Waited for 26000ms trying to join with diag-logger (tid 8702)
W20260709 13:06:05.160037  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:05.180100  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:05.183054  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:05.304800  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:05.382134  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:05.424543  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:05.440860  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:05.445564  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:05.500662  8719 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:05.560338  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:05.617303  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:05.695429  8719 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:05.757462  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:05.758183  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:05.840269  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:05.845541  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:05.862069  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:05.909987  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:06.003794  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:06.084641  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:06.109782  7526 thread.cc:527] Waited for 27000ms trying to join with diag-logger (tid 8702)
W20260709 13:06:06.136696  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:06.353926  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:06.419411  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:06.429423  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:06.433627  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:06.458256  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:06.485234  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:06.558959  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:06.750142  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:06.815372  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:06.945983  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:06.977139  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:06.989042  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:06.989845  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:07.011946  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:07.040237  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:07.086975  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:07.099818  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:06:07.107457  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:07.110571  7526 thread.cc:527] Waited for 28000ms trying to join with diag-logger (tid 8702)
W20260709 13:06:07.178721  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:07.188489  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:07.193881  8719 consensus_peers.cc:597] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:07.206357  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:07.302398  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:07.336714  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:07.386318  8793 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111) [suppressed 474 similar messages]
W20260709 13:06:07.403030  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:07.422161  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:07.545075  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:07.571712  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:07.599094  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:07.754498  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:07.814729  8719 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:07.848606  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:07.919513  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:07.971602  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:07.972895  9355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 79092us
W20260709 13:06:08.020707  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:08.044202  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:08.051483  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:08.111202  7526 thread.cc:527] Waited for 29000ms trying to join with diag-logger (tid 8702)
W20260709 13:06:08.165527  8719 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:08.241770  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:08.245472  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:08.243438  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:37857, user_credentials={real_user=slave}} blocked reactor thread for 72685.4us
W20260709 13:06:08.325201  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:08.398011  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:08.418929  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:08.449936  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:08.492612  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:08.675119  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:08.717907  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:08.877516  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:08.948150  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:08.975003  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:08.977373  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:08.980450  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:09.020359  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:09.111799  7526 thread.cc:527] Waited for 30000ms trying to join with diag-logger (tid 8702)
W20260709 13:06:09.134800  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:09.171031  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:09.321918  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:09.408118  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:09.483309  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:09.469533  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:09.547533  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:09.632678  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:06:09.650341  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:09.659045  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:09.672269  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:09.679117  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:09.679889  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:09.810540  8719 consensus_peers.cc:597] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:09.871750  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:09.917547  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:09.923180  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 48733.3us
W20260709 13:06:10.010952  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 79984us
W20260709 13:06:10.016224  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:10.055835  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:10.048408  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 131806us
W20260709 13:06:10.078065  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:10.087070  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:10.094309  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:10.112404  7526 thread.cc:527] Waited for 31000ms trying to join with diag-logger (tid 8702)
W20260709 13:06:10.116808  8722 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:41749, user_credentials={real_user=slave}} blocked reactor thread for 99406.6us
W20260709 13:06:10.137017  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:10.139099  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:10.209802  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:10.409166  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:10.481611  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:10.573784  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:10.588248  8719 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:10.637483  8719 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:10.723203  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:10.724170  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:10.799600  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:10.856841  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:10.867415  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:10.878196  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:10.879230  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:10.891192  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:10.954506  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:10.998407  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:10.999176  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:11.112969  7526 thread.cc:527] Waited for 32000ms trying to join with diag-logger (tid 8702)
W20260709 13:06:11.158574  8719 consensus_peers.cc:597] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:11.213181  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:11.229075  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:11.682595  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 59251.9us
W20260709 13:06:11.696300  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:11.698962  8719 consensus_peers.cc:597] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:11.762954  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:11.770347  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 54381.2us
W20260709 13:06:11.791251  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:11.792091  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:11.803071  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:11.812105  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:11.822698  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:11.825335  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:12.111701  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:12.145550  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:12.188383  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:12.193578  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:12.221482  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:12.246042  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:12.326270  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:12.358031  8719 consensus_peers.cc:597] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:12.381420  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:12.402544  8793 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111) [suppressed 462 similar messages]
W20260709 13:06:12.414324  8610 debug-util.cc:398] Leaking SignalData structure 0x7b080000b660 after lost signal to thread 7529
W20260709 13:06:12.415910  8610 debug-util.cc:398] Leaking SignalData structure 0x7b0800398900 after lost signal to thread 8613
W20260709 13:06:12.417722  8610 debug-util.cc:398] Leaking SignalData structure 0x7b080034c900 after lost signal to thread 8780
W20260709 13:06:12.419292  8610 debug-util.cc:398] Leaking SignalData structure 0x7b080036b320 after lost signal to thread 8854
W20260709 13:06:12.420840  8610 debug-util.cc:398] Leaking SignalData structure 0x7b08001ff780 after lost signal to thread 9413
W20260709 13:06:12.440725  8719 consensus_peers.cc:597] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:12.462962  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:12.470870  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:12.475392  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:12.479925  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:12.484440  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:12.490160  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:06:12.587522  8719 consensus_peers.cc:597] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:06:12.624599  7526 tablet_server.cc:196] TabletServer@127.7.89.129:0 shutdown complete.
W20260709 13:06:12.653872  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:12.751623  8719 consensus_peers.cc:597] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:12.784447  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:12.815407  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:13.047166  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:06:13.087841  7526 tablet_server.cc:179] TabletServer@127.7.89.130:0 shutting down...
W20260709 13:06:13.115993  8794 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.132439  8794 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.162767  8794 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.185863  8794 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.194901  9353 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.199133  9353 consensus_peers.cc:597] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.187055  8794 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.209053  8794 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.217785  8794 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.231467  8794 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.232311  8794 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.240016  8794 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.251330  8794 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.278115  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:13.281258  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:13.303603  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:13.309324  9353 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.311453  8719 consensus_peers.cc:597] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:13.312953  8794 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.320150  8794 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.326756  8794 consensus_peers.cc:597] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.349676  9353 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.351392  8719 consensus_peers.cc:597] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:06:13.345068  8794 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.366089  8794 consensus_peers.cc:597] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.366951  8794 consensus_peers.cc:597] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.377974  8794 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.378893  8794 consensus_peers.cc:597] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.379608  8794 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:13.387430  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:13.390991  7526 tablet_replica.cc:333] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
W20260709 13:06:13.397094  9353 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:13.401064  7526 raft_consensus.cc:2243] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:13.402035  9353 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.402035  8794 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:13.406637  7526 raft_consensus.cc:2272] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:13.410684  9353 consensus_peers.cc:597] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.409229  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:06:13.420257  7526 tablet_replica.cc:333] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.422739  7526 raft_consensus.cc:2243] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.427709  7526 raft_consensus.cc:2272] T 304eb26fa48d4971bffbe6266846243b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.438241  7526 tablet_replica.cc:333] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
W20260709 13:06:13.440225  9353 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:13.443130  7526 raft_consensus.cc:2243] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.443967  7526 raft_consensus.cc:2272] T 396dbfb60a3644b9b28c39c3dd36b322 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:13.452567  9353 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.453279  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:06:13.455391  7526 tablet_replica.cc:333] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.456555  7526 raft_consensus.cc:2243] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.457971  7526 raft_consensus.cc:2272] T 53cce7f107584cc28999319a785023ef P 0e9b0850145a4c66bfe983ffdfa93a57 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:13.475188  8794 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.476416  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:06:13.485056  7526 tablet_replica.cc:333] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.487398  7526 raft_consensus.cc:2243] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.488811  7526 raft_consensus.cc:2272] T 614067c154284cfa9b7a01100f304cc9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:13.496055  8794 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:13.509627  7526 tablet_replica.cc:333] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.510977  7526 raft_consensus.cc:2243] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.511863  7526 raft_consensus.cc:2272] T 35e078746b204711a1e15f221c16e31a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.515280  7526 tablet_replica.cc:333] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
W20260709 13:06:13.519183  8719 consensus_peers.cc:597] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:13.526958  9353 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:13.527884  7526 raft_consensus.cc:2243] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.528904  7526 raft_consensus.cc:2272] T 4de6a8f976f5458d807b777f3ac94a23 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:13.534453  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:06:13.538719  7526 tablet_replica.cc:333] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.541157  7526 raft_consensus.cc:2243] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.542531  7526 raft_consensus.cc:2272] T 7fbae1ebcfaf4be58c5aecd1014cea01 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.552275  7526 tablet_replica.cc:333] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
W20260709 13:06:13.553323  8794 consensus_peers.cc:597] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:13.558518  8794 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:13.553617  7526 raft_consensus.cc:2243] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.572700  7526 raft_consensus.cc:2272] T 1e6257b62c5c415aad31f5fc646ce4d4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.576035  7526 tablet_replica.cc:333] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.582024  7526 raft_consensus.cc:2243] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.583042  7526 raft_consensus.cc:2272] T 8e9173dea5bf4398a5e138cb273df07c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.595038  7526 tablet_replica.cc:333] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.596010  7526 raft_consensus.cc:2243] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.596797  7526 raft_consensus.cc:2272] T 7e3bac87e89140ffa2dcde662c7fb621 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.599817  7526 tablet_replica.cc:333] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.600657  7526 raft_consensus.cc:2243] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.614594  7526 raft_consensus.cc:2272] T 364ad2097a2341cdb96752b9a370f9a9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:13.615423  8794 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:13.617805  7526 tablet_replica.cc:333] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
W20260709 13:06:13.619441  8794 consensus_peers.cc:597] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:13.626240  7526 raft_consensus.cc:2243] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.627341  7526 raft_consensus.cc:2272] T 293a9dab5be3479a90d5803585682a00 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.633704  7526 tablet_replica.cc:333] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.634559  7526 raft_consensus.cc:2243] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.651199  7526 raft_consensus.cc:2272] T 4f19e578ddc64100a13ab07a6b877a77 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.658986  7526 tablet_replica.cc:333] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.663601  7526 raft_consensus.cc:2243] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.665043  7526 raft_consensus.cc:2272] T bc61bbf8aae94862a95ef23cde2e06e8 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.689327  7526 tablet_replica.cc:333] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.690323  7526 raft_consensus.cc:2243] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:13.697757  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:06:13.699494  7526 raft_consensus.cc:2272] T 389e9e63581e4d1b941ba995f8cdbfc4 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.704007  7526 tablet_replica.cc:333] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.704931  7526 raft_consensus.cc:2243] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.705711  7526 raft_consensus.cc:2272] T 50ae323a197c4e5b994d66663473c7d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:13.712061  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:06:13.731885  7526 tablet_replica.cc:333] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.733013  7526 raft_consensus.cc:2243] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.734489  7526 raft_consensus.cc:2272] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.737789  7526 tablet_replica.cc:333] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.754833  7526 raft_consensus.cc:2243] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.755645  7526 raft_consensus.cc:2272] T 41d15a8ebfe8480d9861c1285faaf7b5 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.764945  7526 tablet_replica.cc:333] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.775553  7526 raft_consensus.cc:2243] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.776424  7526 raft_consensus.cc:2272] T b88e9b3a77fe404e8e4caaefe74a04a2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.779769  7526 tablet_replica.cc:333] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.780670  7526 raft_consensus.cc:2243] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.781358  7526 raft_consensus.cc:2272] T 1148edfb8a5a41fdaced6489dfd22b63 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:13.784801  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:06:13.804039  7526 tablet_replica.cc:333] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.805029  7526 raft_consensus.cc:2243] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.805658  7526 raft_consensus.cc:2272] T 51388ea026cc4d098949a4a0327a35fc P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:13.818838  8719 consensus_peers.cc:597] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:06:13.822078  7526 tablet_replica.cc:333] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.823264  7526 raft_consensus.cc:2243] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.824673  7526 raft_consensus.cc:2272] T 10f975b4911743768a7512f2ad3793c9 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:13.828020  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:06:13.841359  7526 tablet_replica.cc:333] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.842468  7526 raft_consensus.cc:2243] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:13.869561  7526 raft_consensus.cc:2272] T 300460316b0849bf8014930d5e9bfada P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:13.870712  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:13.874965  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20260709 13:06:13.877549  7526 tablet_replica.cc:333] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.883615  7526 raft_consensus.cc:2243] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.884893  7526 raft_consensus.cc:2272] T 8611228be8dd4f08b2b609e202974a6a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.889778  7526 tablet_replica.cc:333] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.890849  7526 raft_consensus.cc:2243] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.891578  7526 raft_consensus.cc:2272] T 7d6f9069613f4a83a64061e85b6f97da P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.894721  7526 tablet_replica.cc:333] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.895634  7526 raft_consensus.cc:2243] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.896414  7526 raft_consensus.cc:2272] T 3c997cc62c984d6e8962403382a451bb P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.923446  7526 tablet_replica.cc:333] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.924393  7526 raft_consensus.cc:2243] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.925098  7526 raft_consensus.cc:2272] T 9c86a8d1b46a4dad9149c50458be45d2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.928387  7526 tablet_replica.cc:333] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.929270  7526 raft_consensus.cc:2243] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.930364  7526 raft_consensus.cc:2272] T 3524354c9db64c0c9b532bef2a0ceb6f P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.933418  7526 tablet_replica.cc:333] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.934279  7526 raft_consensus.cc:2243] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.978761  7526 raft_consensus.cc:2272] T c3796428796f4d6c8fd8e9ff2897d116 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.982735  7526 tablet_replica.cc:333] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
W20260709 13:06:13.983469  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:06:13.983737  7526 raft_consensus.cc:2243] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:13.984632  7526 raft_consensus.cc:2272] T ac4bab9153b047238bffb6abbeb83601 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:13.987553  7526 tablet_replica.cc:333] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:13.990674  7526 raft_consensus.cc:2243] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:06:13.992072  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20260709 13:06:13.992106  7526 raft_consensus.cc:2272] T 149c2129693643678d9da7765768609c P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.006412  7526 tablet_replica.cc:333] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.007805  7526 raft_consensus.cc:2243] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.009361  7526 raft_consensus.cc:2272] T ae08d8f5271e475baa4344843405b254 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.013015  7526 tablet_replica.cc:333] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.014204  7526 raft_consensus.cc:2243] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.015096  7526 raft_consensus.cc:2272] T c01533e2a5204b1ebccac1133e495542 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.018275  7526 tablet_replica.cc:333] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.035046  7526 raft_consensus.cc:2243] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.035846  7526 raft_consensus.cc:2272] T 55c31a55ecab4a17acb00373b55d6f1d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.041488  7526 tablet_replica.cc:333] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.042361  7526 raft_consensus.cc:2243] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.043365  7526 raft_consensus.cc:2272] T 142891c077a34ba289adf90a99aee612 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.047377  7526 tablet_replica.cc:333] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.048462  7526 raft_consensus.cc:2243] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.049943  7526 raft_consensus.cc:2272] T 1f9ddea0a887446f910d38f8302e8172 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.053018  7526 tablet_replica.cc:333] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.053797  7526 raft_consensus.cc:2243] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.060856  7526 raft_consensus.cc:2272] T 7fb0366ec71a44feb8c7b426e851dcc6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.064208  7526 tablet_replica.cc:333] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.065279  7526 raft_consensus.cc:2243] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.066220  7526 raft_consensus.cc:2272] T ff71213103664c0bab63c42773d0880b P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.069279  7526 tablet_replica.cc:333] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.070058  7526 raft_consensus.cc:2243] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.071046  7526 raft_consensus.cc:2272] T fea1a7951ca9443d8f4d05bd1519d442 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.073863  7526 tablet_replica.cc:333] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.074769  7526 raft_consensus.cc:2243] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.075829  7526 raft_consensus.cc:2272] T 09c390d8b5bb47c9a7d7a841a4fe135e P 0e9b0850145a4c66bfe983ffdfa93a57 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.079031  7526 tablet_replica.cc:333] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.079942  7526 raft_consensus.cc:2243] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.080760  7526 raft_consensus.cc:2272] T a4f0aa5ad2ef496d9a459b6742346f0a P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.084122  7526 tablet_replica.cc:333] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.085005  7526 raft_consensus.cc:2243] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.086072  7526 raft_consensus.cc:2272] T b50b38d6e5314ca190c9a14a5e0f655d P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.091861  7526 tablet_replica.cc:333] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
W20260709 13:06:14.106397  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:06:14.106034  7526 raft_consensus.cc:2243] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.107884  7526 raft_consensus.cc:2272] T 9f81d7aa5ed04c68b603d272f24760e2 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.115118  7526 tablet_replica.cc:333] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.116124  7526 raft_consensus.cc:2243] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.117233  7526 raft_consensus.cc:2272] T 295d39c61dd3489e98d9393a2da9c062 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.121155  7526 tablet_replica.cc:333] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.122078  7526 raft_consensus.cc:2243] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.123831  7526 raft_consensus.cc:2272] T 2217efcaa5fe4b729f1beef2f96e4189 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.127553  7526 tablet_replica.cc:333] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.129868  7526 raft_consensus.cc:2243] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.130723  7526 raft_consensus.cc:2272] T 844a3710898640c49df7cfb8d1c568cf P 0e9b0850145a4c66bfe983ffdfa93a57 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.134167  7526 tablet_replica.cc:333] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.135145  7526 raft_consensus.cc:2243] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.135917  7526 raft_consensus.cc:2272] T 67bfc59e963f4ad0ac128fd11eecac17 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.139461  7526 tablet_replica.cc:333] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.140307  7526 raft_consensus.cc:2243] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:14.141170  7526 raft_consensus.cc:2272] T 987153bedd0c4f86b726db1edfd1eec6 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:14.144111  7526 tablet_replica.cc:333] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57: stopping tablet replica
I20260709 13:06:14.144797  7526 raft_consensus.cc:2243] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:14.146234  7526 raft_consensus.cc:2272] T 9ca09750f1b44e018d62c21aeb12a929 P 0e9b0850145a4c66bfe983ffdfa93a57 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:14.335409  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:14.345840  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:14.357898  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:14.358788  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:14.368690  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:14.589138  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:14.608600  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:06:14.646652 17215 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:06:14.655112  8563 catalog_manager.cc:4755] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 53cce7f107584cc28999319a785023ef: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:06:14.652262  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20260709 13:06:14.655161 17215 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:06:14.669164 17215 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:06:14.671312  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:14.671654  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:14.673728  8794 leader_election.cc:336] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
W20260709 13:06:14.679579  8793 leader_election.cc:336] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:06:14.680064  8793 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:14.685626 17211 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:14.701519  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:06:14.736202 17215 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:14.736933 17215 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:14.741849  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:06:14.744853  8794 leader_election.cc:336] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:14.746893  8796 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:06:14.748205 17201 raft_consensus.cc:2749] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:14.751483  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:06:14.755333 17215 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:06:14.784430 17223 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:14.785161 17223 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:14.788287 17223 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:06:14.799768 17201 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:14.800755 17201 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:06:14.821376  9353 leader_election.cc:336] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:14.823658 17201 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:06:14.830147  8794 leader_election.cc:336] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:14.845108 17209 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:14.845881 17209 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:06:14.847571  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:06:14.850345  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:06:14.851199  8824 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 23.
I20260709 13:06:14.852824  9355 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:06:14.854115 17225 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:14.854588 17225 raft_consensus.cc:493] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:14.854986 17225 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:06:14.863782  9352 leader_election.cc:336] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:06:14.864426  9352 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:14.880486 17218 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:14.887831  9353 leader_election.cc:336] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
W20260709 13:06:14.894114  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:06:14.898140  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20260709 13:06:14.899820  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
W20260709 13:06:14.905054  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20260709 13:06:14.890530 17209 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:06:14.908684 17225 raft_consensus.cc:515] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:14.912050 17201 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:14.913872 17215 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:14.931100 17215 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:14.930327 17201 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:06:14.951220  9389 raft_consensus.cc:2393] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e814ed2013244fa6be42623ea5c0c7b9 in current term 24: Already voted for candidate 7aff04f53d4845a6a4fa491fd2a06145 in this term.
I20260709 13:06:14.956696 17215 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:06:14.959030 17225 leader_election.cc:290] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:06:14.963776 17201 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:06:14.970449  9383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:06:14.985831  8796 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:06:14.988173 17225 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:14.990991 17225 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:14.995294 17225 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:06:14.997556  8794 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
W20260709 13:06:14.998296  8794 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:15.000658  8796 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, 7aff04f53d4845a6a4fa491fd2a06145
I20260709 13:06:15.000792 17232 raft_consensus.cc:3060] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:15.002203 17231 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:15.010660  9353 leader_election.cc:336] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
W20260709 13:06:15.022969  9353 leader_election.cc:336] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:15.027058  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "304eb26fa48d4971bffbe6266846243b" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:06:15.031437  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:06:15.031997 17235 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:15.032387  8822 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 21.
I20260709 13:06:15.032686 17235 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:06:15.034109 17225 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:15.034790 17225 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:15.036069  9355 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:15.037010 17225 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:06:15.037643 17225 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:15.037976 17225 raft_consensus.cc:493] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:15.038334 17225 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:06:15.047461  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
I20260709 13:06:15.054023  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
W20260709 13:06:15.056669  8794 leader_election.cc:336] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:15.061425 17218 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:15.062088 17218 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:15.066004 17235 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:06:15.067588 17218 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:06:15.070752 17225 raft_consensus.cc:515] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:15.073213 17225 leader_election.cc:290] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
I20260709 13:06:15.054322 17232 raft_consensus.cc:2749] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:06:15.074656  9353 leader_election.cc:336] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:15.075202  9353 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e9b0850145a4c66bfe983ffdfa93a57, e814ed2013244fa6be42623ea5c0c7b9
I20260709 13:06:15.078980 17225 raft_consensus.cc:2749] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:15.087491  8824 raft_consensus.cc:2468] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 24.
W20260709 13:06:15.090076  9352 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:06:15.095041  9353 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:15.097939  9353 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:15.106861  9355 leader_election.cc:304] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:15.107816 17232 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:15.118647 17232 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:15.121308 17232 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:06:15.122330 17225 raft_consensus.cc:2804] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:06:15.123317 17225 raft_consensus.cc:697] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:06:15.124055  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:06:15.124801  9389 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 22.
I20260709 13:06:15.124452 17225 consensus_queue.cc:237] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:15.126294  8796 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 
I20260709 13:06:15.128727 17218 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:15.130717  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9" is_pre_election: true
I20260709 13:06:15.131222 17232 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:15.131701  8824 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 20.
I20260709 13:06:15.133600  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6257b62c5c415aad31f5fc646ce4d4" candidate_uuid: "7aff04f53d4845a6a4fa491fd2a06145" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e814ed2013244fa6be42623ea5c0c7b9"
I20260709 13:06:15.134239  8822 raft_consensus.cc:3060] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:15.131752 17232 raft_consensus.cc:493] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:15.135300 17232 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:06:15.142894  8794 leader_election.cc:336] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:15.147612  8822 raft_consensus.cc:2468] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aff04f53d4845a6a4fa491fd2a06145 in term 22.
W20260709 13:06:15.116832  8793 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:06:15.150939  8793 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:06:15.152961 17234 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:06:15.148399  9353 leader_election.cc:336] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:15.153772 17234 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:06:15.154919  9355 leader_election.cc:304] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:15.156347 17234 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:06:15.159022 17209 raft_consensus.cc:2804] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:15.159618 17209 raft_consensus.cc:697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 LEADER]: Becoming Leader. State: Replica: 7aff04f53d4845a6a4fa491fd2a06145, State: Running, Role: LEADER
I20260709 13:06:15.160540 17209 consensus_queue.cc:237] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:15.160929 17232 raft_consensus.cc:515] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:15.163257 17232 leader_election.cc:290] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 election: Requested vote from peers 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
W20260709 13:06:15.163009  8794 leader_election.cc:336] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:15.164906  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147fd3c99a94f0c8e9c1a914bdb79c3" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145"
I20260709 13:06:15.165576  9389 raft_consensus.cc:3060] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:15.174912  8793 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:15.181921 17234 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:15.180630  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:15.183777  8794 leader_election.cc:336] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:15.189370  9389 raft_consensus.cc:2468] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 23.
I20260709 13:06:15.193568  8796 leader_election.cc:304] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:15.194700 17232 raft_consensus.cc:2804] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:15.194798 17235 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:15.195322 17232 raft_consensus.cc:697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 LEADER]: Becoming Leader. State: Replica: e814ed2013244fa6be42623ea5c0c7b9, State: Running, Role: LEADER
I20260709 13:06:15.195389 17235 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:06:15.196514 17232 consensus_queue.cc:237] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:15.197763 17235 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:06:15.209901  8794 leader_election.cc:336] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
W20260709 13:06:15.210217  8793 leader_election.cc:336] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:06:15.211012  8793 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:15.214816 17231 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:06:15.216099  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:06:15.219365  8576 catalog_manager.cc:5697] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 23 to 24, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 24 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:15.242328 17235 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:15.243211 17235 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:06:15.244468  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20260709 13:06:15.246479 17235 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:06:15.307739  8793 leader_election.cc:336] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:06:15.308324  8793 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:15.318463 17234 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:15.320833  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:15.327948  8794 leader_election.cc:336] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
W20260709 13:06:15.332185  8793 leader_election.cc:336] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:06:15.332819  8793 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:15.332973 17209 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:15.333601 17209 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
W20260709 13:06:15.334065  8793 leader_election.cc:336] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:06:15.334347 17232 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:15.340036  8571 catalog_manager.cc:5697] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 reported cstate change: term changed from 22 to 23, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131). New cstate: current_term: 23 leader_uuid: "e814ed2013244fa6be42623ea5c0c7b9" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:15.344796 17209 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:06:15.354005  9352 leader_election.cc:336] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:06:15.357924  9353 leader_election.cc:336] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:15.358448  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:15.362036 17225 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:15.334559  8793 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:15.375006 17232 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:15.436774  8576 catalog_manager.cc:5697] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 reported cstate change: term changed from 21 to 22, leader changed from 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130) to 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132). New cstate: current_term: 22 leader_uuid: "7aff04f53d4845a6a4fa491fd2a06145" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:15.499784  7526 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 8777)
I20260709 13:06:15.509915 17234 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:15.510774 17234 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:06:15.513692 17234 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:06:15.538404  9353 consensus_peers.cc:597] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:15.548076  8793 leader_election.cc:336] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:06:15.555825  8794 leader_election.cc:336] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:15.560030  8794 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:15.562716 17232 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:15.569479  8822 raft_consensus.cc:1275] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:06:15.571484 17209 consensus_queue.cc:1048] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:06:15.605633  8824 raft_consensus.cc:1275] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Refusing update from remote peer 7aff04f53d4845a6a4fa491fd2a06145: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:06:15.607568 17209 consensus_queue.cc:1048] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:06:15.679618  9353 consensus_peers.cc:597] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:15.729100  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:15.729839  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:15.749037  9353 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:15.781561  9389 raft_consensus.cc:1275] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Refusing update from remote peer e814ed2013244fa6be42623ea5c0c7b9: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:06:15.788232 17235 consensus_queue.cc:1048] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:06:15.840256  9353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 68393.6us
W20260709 13:06:15.837868  9352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:43221, user_credentials={real_user=slave}} blocked reactor thread for 68323.9us
W20260709 13:06:15.846812  8794 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:15.852969  8794 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:15.860092  8794 consensus_peers.cc:597] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:15.859321  9353 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:15.916525  9353 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:15.961202  8793 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:15.978089  9353 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:15.971848  8793 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:06:15.983734  8794 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:15.987287  8794 consensus_peers.cc:597] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:16.008397  8794 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.032590  8794 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.036082  9353 consensus_peers.cc:597] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.038995  9353 consensus_peers.cc:597] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.036859  8793 consensus_peers.cc:597] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:16.054186  8794 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.081678  9353 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.082379  8794 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.088366  8794 consensus_peers.cc:597] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.099704  8794 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.110320  8794 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.111763  8794 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.112391  8794 consensus_peers.cc:597] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.111145  9353 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.107348  8793 consensus_peers.cc:597] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:16.114161  8793 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:16.120076  8794 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.130442  9353 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.146775  9353 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.148897  8793 consensus_peers.cc:597] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:06:16.154349  8793 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:16.191970  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:16.193156  8794 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.202919  8793 consensus_peers.cc:597] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:16.208957  8794 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.218545  8794 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.223927  8794 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.278764  8794 consensus_peers.cc:597] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.279436  8794 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.279932  8794 consensus_peers.cc:597] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.282382  8794 consensus_peers.cc:597] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.282898  8794 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.286345  8794 consensus_peers.cc:597] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.291702  8794 consensus_peers.cc:597] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.429126  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 86190.5us
W20260709 13:06:16.430079  8794 consensus_peers.cc:597] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.441988  8793 consensus_peers.cc:597] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:16.442560  8793 consensus_peers.cc:597] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:16.568524  9352 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:16.622920  8794 consensus_peers.cc:597] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:16.639423  8793 consensus_peers.cc:597] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:16.654062  8793 consensus_peers.cc:597] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20260709 13:06:16.730088 17260 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:16.731745 17260 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:16.735373 17234 raft_consensus.cc:493] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:16.736255 17234 raft_consensus.cc:515] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:06:16.742751  9352 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:16.749691  8793 leader_election.cc:336] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:06:16.742445 17234 leader_election.cc:290] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:06:16.755920  9389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300460316b0849bf8014930d5e9bfada" candidate_uuid: "e814ed2013244fa6be42623ea5c0c7b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "7aff04f53d4845a6a4fa491fd2a06145" is_pre_election: true
I20260709 13:06:16.757195  9389 raft_consensus.cc:2468] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e814ed2013244fa6be42623ea5c0c7b9 in term 20.
W20260709 13:06:16.772404  8793 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:06:16.775085  8794 leader_election.cc:336] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:16.775647  8794 leader_election.cc:304] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:16.777338 17260 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), 7aff04f53d4845a6a4fa491fd2a06145 (127.7.89.132:41749)
I20260709 13:06:16.783047 17258 raft_consensus.cc:2749] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:06:16.787217  8793 consensus_peers.cc:597] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:06:16.788216  8794 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:16.788671  8794 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145, e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:16.789422 17258 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:16.817859 17253 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:16.818979 17253 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:16.824388 17234 raft_consensus.cc:493] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:16.825270 17234 raft_consensus.cc:515] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:06:16.827239  9353 leader_election.cc:336] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:16.827786 17234 leader_election.cc:290] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:06:16.828192 17253 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:06:16.835958  9352 leader_election.cc:336] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:06:16.847435  9352 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:16.848327 17253 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:16.852452  8793 leader_election.cc:336] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:06:16.855131  8794 leader_election.cc:336] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:16.855675  8794 leader_election.cc:304] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:16.857380 17260 raft_consensus.cc:2749] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:16.905145  8793 consensus_peers.cc:597] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:06:16.908725 17234 raft_consensus.cc:493] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:16.909641 17234 raft_consensus.cc:515] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:06:16.912379 17234 leader_election.cc:290] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:06:16.940564  8793 leader_election.cc:336] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:06:16.958137  8794 leader_election.cc:336] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:16.962471  8794 leader_election.cc:304] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:16.963513 17260 raft_consensus.cc:2749] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:16.996156  9352 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:17.006728  8793 consensus_peers.cc:597] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:17.050149  8793 consensus_peers.cc:597] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:17.077185  9352 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:17.094559  8793 consensus_peers.cc:597] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:17.134003  8793 consensus_peers.cc:597] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:17.178018  8794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:42321, user_credentials={real_user=slave}} blocked reactor thread for 50890.4us
W20260709 13:06:17.183347  9352 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
I20260709 13:06:17.202663 17234 raft_consensus.cc:493] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:17.203522 17234 raft_consensus.cc:515] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:06:17.205813 17234 leader_election.cc:290] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:06:17.211613  8793 leader_election.cc:336] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:06:17.218456  8793 consensus_peers.cc:597] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20260709 13:06:17.240674  7526 tablet_server.cc:196] TabletServer@127.7.89.130:0 shutdown complete.
W20260709 13:06:17.259963  8794 leader_election.cc:336] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:17.260713  8794 leader_election.cc:304] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:17.271510 17260 raft_consensus.cc:2749] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:17.274147 17234 raft_consensus.cc:493] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
W20260709 13:06:17.291179  9352 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
I20260709 13:06:17.276415 17234 raft_consensus.cc:515] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
W20260709 13:06:17.307948  8793 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:06:17.318440 17234 leader_election.cc:290] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:06:17.327419  8794 leader_election.cc:336] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:17.328078  8794 leader_election.cc:304] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:17.329535 17234 raft_consensus.cc:2749] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:17.378804  8793 consensus_peers.cc:597] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:17.398095  9352 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111) [suppressed 771 similar messages]
W20260709 13:06:17.413453  8793 consensus_peers.cc:597] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:17.471284  9352 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
I20260709 13:06:17.474781 17260 raft_consensus.cc:493] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:17.475859 17253 raft_consensus.cc:493] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:17.475659 17260 raft_consensus.cc:515] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:06:17.476630 17253 raft_consensus.cc:515] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:17.478376 17260 leader_election.cc:290] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
I20260709 13:06:17.485816 17253 leader_election.cc:290] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:06:17.489283  8794 leader_election.cc:336] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
W20260709 13:06:17.490736  8793 leader_election.cc:336] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:06:17.491853  8793 leader_election.cc:304] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:17.492638 17260 raft_consensus.cc:2749] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:17.971287  7526 tablet_server.cc:179] TabletServer@127.7.89.131:0 shutting down...
W20260709 13:06:17.974738  9352 leader_election.cc:336] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:06:18.022421  9353 leader_election.cc:336] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:18.022987  9353 leader_election.cc:304] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:18.025206 17209 raft_consensus.cc:2749] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:18.043666  9352 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:18.050314  9353 consensus_peers.cc:597] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:18.055123 17266 raft_consensus.cc:493] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:18.055939 17266 raft_consensus.cc:515] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } }
I20260709 13:06:18.058351 17266 leader_election.cc:290] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:06:18.075492  9355 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Couldn't send request to peer e814ed2013244fa6be42623ea5c0c7b9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.101080  9355 consensus_peers.cc:597] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Couldn't send request to peer e814ed2013244fa6be42623ea5c0c7b9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.104423  9355 consensus_peers.cc:597] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Couldn't send request to peer e814ed2013244fa6be42623ea5c0c7b9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.106026  9355 consensus_peers.cc:597] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Couldn't send request to peer e814ed2013244fa6be42623ea5c0c7b9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.106699  9355 consensus_peers.cc:597] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Couldn't send request to peer e814ed2013244fa6be42623ea5c0c7b9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.107239  9355 consensus_peers.cc:597] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Couldn't send request to peer e814ed2013244fa6be42623ea5c0c7b9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.107844  9355 consensus_peers.cc:597] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Couldn't send request to peer e814ed2013244fa6be42623ea5c0c7b9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.108439  9355 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Couldn't send request to peer e814ed2013244fa6be42623ea5c0c7b9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.109046  9352 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:06:18.114006  9352 consensus_peers.cc:597] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
W20260709 13:06:18.117430  9355 consensus_peers.cc:597] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Couldn't send request to peer e814ed2013244fa6be42623ea5c0c7b9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.163697  9355 consensus_peers.cc:597] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Couldn't send request to peer e814ed2013244fa6be42623ea5c0c7b9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.179738  8794 leader_election.cc:336] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
W20260709 13:06:18.190596  8793 consensus_peers.cc:597] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:18.191516  8793 consensus_peers.cc:597] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:06:18.192250  8793 leader_election.cc:336] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:06:18.192646  8793 leader_election.cc:304] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e814ed2013244fa6be42623ea5c0c7b9; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:18.193426 17260 raft_consensus.cc:2749] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:18.194505  8793 consensus_peers.cc:597] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:06:18.203629  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:18.204861  7526 tablet_replica.cc:333] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.205964  7526 raft_consensus.cc:2243] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.207801  7526 raft_consensus.cc:2272] T 13d1c43d45594a8d9dc9c717fd6e4ca7 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.211684  7526 tablet_replica.cc:333] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.212704  7526 raft_consensus.cc:2243] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.213433  7526 raft_consensus.cc:2272] T 304eb26fa48d4971bffbe6266846243b P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.216665  7526 tablet_replica.cc:333] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.217628  7526 raft_consensus.cc:2243] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.219084  7526 raft_consensus.cc:2272] T 396dbfb60a3644b9b28c39c3dd36b322 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.222669  7526 tablet_replica.cc:333] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.223594  7526 raft_consensus.cc:2243] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.225096  7526 raft_consensus.cc:2272] T 53cce7f107584cc28999319a785023ef P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.228250  7526 tablet_replica.cc:333] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.229166  7526 raft_consensus.cc:2243] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.230151  7526 raft_consensus.cc:2272] T 3a7eadf5c488447886a90612fbf1a2bb P e814ed2013244fa6be42623ea5c0c7b9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.233306  7526 tablet_replica.cc:333] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.234200  7526 raft_consensus.cc:2243] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.235885  7526 raft_consensus.cc:2272] T 35e078746b204711a1e15f221c16e31a P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.239538  7526 tablet_replica.cc:333] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.240631  7526 raft_consensus.cc:2243] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.241586  7526 raft_consensus.cc:2272] T abd86f4e9b544531a7cbfa0042b4fa6c P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.244797  7526 tablet_replica.cc:333] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.245790  7526 raft_consensus.cc:2243] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.246678  7526 raft_consensus.cc:2272] T 7fbae1ebcfaf4be58c5aecd1014cea01 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.249889  7526 tablet_replica.cc:333] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.250866  7526 raft_consensus.cc:2243] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.251761  7526 raft_consensus.cc:2272] T 1e6257b62c5c415aad31f5fc646ce4d4 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.255295  7526 tablet_replica.cc:333] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.256313  7526 raft_consensus.cc:2243] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.257441  7526 raft_consensus.cc:2272] T bc61bbf8aae94862a95ef23cde2e06e8 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.260738  7526 tablet_replica.cc:333] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.261680  7526 raft_consensus.cc:2243] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.263154  7526 raft_consensus.cc:2272] T 50ae323a197c4e5b994d66663473c7d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.266080  7526 tablet_replica.cc:333] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.266989  7526 raft_consensus.cc:2243] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.268784  7526 raft_consensus.cc:2272] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.271893  7526 tablet_replica.cc:333] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.272795  7526 raft_consensus.cc:2243] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.273794  7526 raft_consensus.cc:2272] T 3169f7f9f5a7440398ae5ebe01fc3c3f P e814ed2013244fa6be42623ea5c0c7b9 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.276850  7526 tablet_replica.cc:333] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.277666  7526 raft_consensus.cc:2243] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.279439  7526 raft_consensus.cc:2272] T b88e9b3a77fe404e8e4caaefe74a04a2 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.282806  7526 tablet_replica.cc:333] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.283829  7526 raft_consensus.cc:2243] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.285403  7526 raft_consensus.cc:2272] T 1148edfb8a5a41fdaced6489dfd22b63 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.289049  7526 tablet_replica.cc:333] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.290110  7526 raft_consensus.cc:2243] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.291740  7526 raft_consensus.cc:2272] T 05559e602d4b4ea3986701fa621706a0 P e814ed2013244fa6be42623ea5c0c7b9 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.295010  7526 tablet_replica.cc:333] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.295701  7526 raft_consensus.cc:2243] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.296346  7526 raft_consensus.cc:2272] T 1c963cd681ab4f45a5c339f9fd6e2687 P e814ed2013244fa6be42623ea5c0c7b9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.299008  7526 tablet_replica.cc:333] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.299751  7526 raft_consensus.cc:2243] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.300981  7526 raft_consensus.cc:2272] T ff71213103664c0bab63c42773d0880b P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.303421  7526 tablet_replica.cc:333] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.304171  7526 raft_consensus.cc:2243] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.304862  7526 raft_consensus.cc:2272] T 614067c154284cfa9b7a01100f304cc9 P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.307173  7526 tablet_replica.cc:333] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.307842  7526 raft_consensus.cc:2243] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.308600  7526 raft_consensus.cc:2272] T 4f19e578ddc64100a13ab07a6b877a77 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.311144  7526 tablet_replica.cc:333] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.311789  7526 raft_consensus.cc:2243] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.312958  7526 raft_consensus.cc:2272] T 293a9dab5be3479a90d5803585682a00 P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.315299  7526 tablet_replica.cc:333] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.315981  7526 raft_consensus.cc:2243] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.317116  7526 raft_consensus.cc:2272] T 41d15a8ebfe8480d9861c1285faaf7b5 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.319653  7526 tablet_replica.cc:333] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.320366  7526 raft_consensus.cc:2243] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.321707  7526 raft_consensus.cc:2272] T 844a3710898640c49df7cfb8d1c568cf P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.324156  7526 tablet_replica.cc:333] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.324960  7526 raft_consensus.cc:2243] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.326308  7526 raft_consensus.cc:2272] T 67bfc59e963f4ad0ac128fd11eecac17 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.328640  7526 tablet_replica.cc:333] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.329276  7526 raft_consensus.cc:2243] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.330988  7526 raft_consensus.cc:2272] T 10f975b4911743768a7512f2ad3793c9 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.333384  7526 tablet_replica.cc:333] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.333978  7526 raft_consensus.cc:2243] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.335106  7526 raft_consensus.cc:2272] T 300460316b0849bf8014930d5e9bfada P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.337512  7526 tablet_replica.cc:333] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.338116  7526 raft_consensus.cc:2243] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.338694  7526 raft_consensus.cc:2272] T 8611228be8dd4f08b2b609e202974a6a P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.340926  7526 tablet_replica.cc:333] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.341629  7526 raft_consensus.cc:2243] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.342785  7526 raft_consensus.cc:2272] T 55c31a55ecab4a17acb00373b55d6f1d P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.345135  7526 tablet_replica.cc:333] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.346014  7526 raft_consensus.cc:2243] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.347517  7526 raft_consensus.cc:2272] T 3c997cc62c984d6e8962403382a451bb P e814ed2013244fa6be42623ea5c0c7b9 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.350230  7526 tablet_replica.cc:333] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.352945  7526 raft_consensus.cc:2243] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.354427  7526 raft_consensus.cc:2272] T 9c86a8d1b46a4dad9149c50458be45d2 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.358022  7526 tablet_replica.cc:333] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.359076  7526 raft_consensus.cc:2243] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.361469  7526 raft_consensus.cc:2272] T 3524354c9db64c0c9b532bef2a0ceb6f P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.365182  7526 tablet_replica.cc:333] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.366132  7526 raft_consensus.cc:2243] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.367894  7526 raft_consensus.cc:2272] T c3796428796f4d6c8fd8e9ff2897d116 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.371270  7526 tablet_replica.cc:333] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.372218  7526 raft_consensus.cc:2243] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 26 LEADER]: Raft consensus shutting down.
W20260709 13:06:18.372385  9353 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:18.374293  7526 raft_consensus.cc:2272] T 7e3bac87e89140ffa2dcde662c7fb621 P e814ed2013244fa6be42623ea5c0c7b9 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.377825  7526 tablet_replica.cc:333] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.378762  7526 raft_consensus.cc:2243] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 30 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.380280  7526 raft_consensus.cc:2272] T 8e9173dea5bf4398a5e138cb273df07c P e814ed2013244fa6be42623ea5c0c7b9 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.383585  7526 tablet_replica.cc:333] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.384402  7526 raft_consensus.cc:2243] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.385099  7526 raft_consensus.cc:2272] T 1f9ddea0a887446f910d38f8302e8172 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.388552  7526 tablet_replica.cc:333] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.389382  7526 raft_consensus.cc:2243] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.390090  7526 raft_consensus.cc:2272] T 149c2129693643678d9da7765768609c P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.393394  7526 tablet_replica.cc:333] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.394241  7526 raft_consensus.cc:2243] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.396024  7526 raft_consensus.cc:2272] T 20fcd6d0c3304abda83f416721e807c5 P e814ed2013244fa6be42623ea5c0c7b9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.399194  7526 tablet_replica.cc:333] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.400074  7526 raft_consensus.cc:2243] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.401422  7526 raft_consensus.cc:2272] T 7d6f9069613f4a83a64061e85b6f97da P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.404511  7526 tablet_replica.cc:333] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.405376  7526 raft_consensus.cc:2243] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.406437  7526 raft_consensus.cc:2272] T ae08d8f5271e475baa4344843405b254 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.409686  7526 tablet_replica.cc:333] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.410709  7526 raft_consensus.cc:2243] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.412879  7526 raft_consensus.cc:2272] T 7fb0366ec71a44feb8c7b426e851dcc6 P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.416226  7526 tablet_replica.cc:333] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.417055  7526 raft_consensus.cc:2243] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.418706  7526 raft_consensus.cc:2272] T fea1a7951ca9443d8f4d05bd1519d442 P e814ed2013244fa6be42623ea5c0c7b9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.422461  7526 tablet_replica.cc:333] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.423417  7526 raft_consensus.cc:2243] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.424811  7526 raft_consensus.cc:2272] T 09c390d8b5bb47c9a7d7a841a4fe135e P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.428256  7526 tablet_replica.cc:333] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.429092  7526 raft_consensus.cc:2243] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.429912  7526 raft_consensus.cc:2272] T 95fdf24d372f40b8877c2792102f34cf P e814ed2013244fa6be42623ea5c0c7b9 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.432968  7526 tablet_replica.cc:333] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.433856  7526 raft_consensus.cc:2243] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.436535  7526 raft_consensus.cc:2272] T 295d39c61dd3489e98d9393a2da9c062 P e814ed2013244fa6be42623ea5c0c7b9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.440161  7526 tablet_replica.cc:333] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.441016  7526 raft_consensus.cc:2243] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.442976  7526 raft_consensus.cc:2272] T a4f0aa5ad2ef496d9a459b6742346f0a P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.446115  7526 tablet_replica.cc:333] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.446939  7526 raft_consensus.cc:2243] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.449643  7526 raft_consensus.cc:2272] T 8acd9840694e4f62803bfe9810425202 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.452718  7526 tablet_replica.cc:333] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.453337  7526 raft_consensus.cc:2243] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:18.454439  7526 raft_consensus.cc:2272] T b50b38d6e5314ca190c9a14a5e0f655d P e814ed2013244fa6be42623ea5c0c7b9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.456914  7526 tablet_replica.cc:333] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.457455  7526 raft_consensus.cc:2243] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.458181  7526 raft_consensus.cc:2272] T 5ee1e688ccd74392befec90535c1efa4 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.460566  7526 tablet_replica.cc:333] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9: stopping tablet replica
I20260709 13:06:18.461067  7526 raft_consensus.cc:2243] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.461669  7526 raft_consensus.cc:2272] T 9ca09750f1b44e018d62c21aeb12a929 P e814ed2013244fa6be42623ea5c0c7b9 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:18.510882  9353 consensus_peers.cc:597] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:18.521756  9353 consensus_peers.cc:597] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:18.531714  9352 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
W20260709 13:06:18.543628  9353 consensus_peers.cc:597] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:18.551882  9352 consensus_peers.cc:597] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:18.563184  9353 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:18.584100  9352 consensus_peers.cc:597] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20260709 13:06:18.630075  9353 consensus_peers.cc:597] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:18.641968 17209 raft_consensus.cc:493] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:18.642726 17209 raft_consensus.cc:515] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:18.645015 17209 leader_election.cc:290] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321)
W20260709 13:06:18.650053  9352 leader_election.cc:336] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:06:18.651767  9353 leader_election.cc:336] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
I20260709 13:06:18.652256  9353 leader_election.cc:304] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:18.653180 17263 raft_consensus.cc:2749] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:18.656512  9353 consensus_peers.cc:597] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:18.659085  9353 consensus_peers.cc:597] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:18.897195  9352 consensus_peers.cc:597] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Couldn't send request to peer 0e8c59a1f0394eae8ce33b99f10efadc. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
W20260709 13:06:18.925712  9353 consensus_peers.cc:597] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:18.928164  9353 consensus_peers.cc:597] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:18.981251  9353 consensus_peers.cc:597] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 -> Peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Couldn't send request to peer 0e9b0850145a4c66bfe983ffdfa93a57. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:19.044585 17209 raft_consensus.cc:493] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0e9b0850145a4c66bfe983ffdfa93a57)
I20260709 13:06:19.045230 17209 raft_consensus.cc:515] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0e8c59a1f0394eae8ce33b99f10efadc" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 43221 } } peers { permanent_uuid: "0e9b0850145a4c66bfe983ffdfa93a57" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42321 } } peers { permanent_uuid: "e814ed2013244fa6be42623ea5c0c7b9" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 37857 } attrs { replace: true } } peers { permanent_uuid: "7aff04f53d4845a6a4fa491fd2a06145" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 41749 } attrs { promote: false } }
I20260709 13:06:19.048074 17209 leader_election.cc:290] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221), 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321), e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857)
W20260709 13:06:19.052752  9410 debug-util.cc:398] Leaking SignalData structure 0x7b080003efa0 after lost signal to thread 7529
W20260709 13:06:19.053094  9353 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e9b0850145a4c66bfe983ffdfa93a57 (127.7.89.130:42321): Network error: Client connection negotiation failed: client connection to 127.7.89.130:42321: connect: Connection refused (error 111)
W20260709 13:06:19.054215  9410 debug-util.cc:398] Leaking SignalData structure 0x7b0800329780 after lost signal to thread 8613
W20260709 13:06:19.055545  9352 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
W20260709 13:06:19.055886  9410 debug-util.cc:398] Leaking SignalData structure 0x7b0800315220 after lost signal to thread 8854
I20260709 13:06:19.056723  9352 leader_election.cc:304] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aff04f53d4845a6a4fa491fd2a06145; no voters: 0e8c59a1f0394eae8ce33b99f10efadc, 0e9b0850145a4c66bfe983ffdfa93a57
I20260709 13:06:19.057943 17209 raft_consensus.cc:2749] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:19.058353  9410 debug-util.cc:398] Leaking SignalData structure 0x7b08002e9e60 after lost signal to thread 9413
W20260709 13:06:19.061496  9355 leader_election.cc:336] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e814ed2013244fa6be42623ea5c0c7b9 (127.7.89.131:37857): Network error: Client connection negotiation failed: client connection to 127.7.89.131:37857: connect: Connection refused (error 111)
I20260709 13:06:19.072185  7526 tablet_server.cc:196] TabletServer@127.7.89.131:0 shutdown complete.
I20260709 13:06:19.231560  7526 tablet_server.cc:179] TabletServer@127.7.89.132:0 shutting down...
I20260709 13:06:19.298846  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:19.300004  7526 tablet_replica.cc:333] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.301178  7526 raft_consensus.cc:2243] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.302145  7526 raft_consensus.cc:2272] T 7fb0366ec71a44feb8c7b426e851dcc6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.305617  7526 tablet_replica.cc:333] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.306695  7526 raft_consensus.cc:2243] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.307758  7526 raft_consensus.cc:2272] T 10f975b4911743768a7512f2ad3793c9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.311514  7526 tablet_replica.cc:333] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.312635  7526 raft_consensus.cc:2243] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.314323  7526 raft_consensus.cc:2272] T 304eb26fa48d4971bffbe6266846243b P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.317721  7526 tablet_replica.cc:333] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.318578  7526 raft_consensus.cc:2243] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.319525  7526 raft_consensus.cc:2272] T 3524354c9db64c0c9b532bef2a0ceb6f P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.323716  7526 tablet_replica.cc:333] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.324565  7526 raft_consensus.cc:2243] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.325476  7526 raft_consensus.cc:2272] T 53cce7f107584cc28999319a785023ef P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.329077  7526 tablet_replica.cc:333] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.330001  7526 raft_consensus.cc:2243] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.330880  7526 raft_consensus.cc:2272] T 364ad2097a2341cdb96752b9a370f9a9 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.334313  7526 tablet_replica.cc:333] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.335206  7526 raft_consensus.cc:2243] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.336536  7526 raft_consensus.cc:2272] T abd86f4e9b544531a7cbfa0042b4fa6c P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.339432  7526 tablet_replica.cc:333] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.340169  7526 raft_consensus.cc:2243] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.341282  7526 raft_consensus.cc:2272] T 51388ea026cc4d098949a4a0327a35fc P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.343900  7526 tablet_replica.cc:333] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.344669  7526 raft_consensus.cc:2243] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.345440  7526 raft_consensus.cc:2272] T 2217efcaa5fe4b729f1beef2f96e4189 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.348151  7526 tablet_replica.cc:333] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.348975  7526 raft_consensus.cc:2243] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.350068  7526 raft_consensus.cc:2272] T 95fdf24d372f40b8877c2792102f34cf P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.352833  7526 tablet_replica.cc:333] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.353765  7526 raft_consensus.cc:2243] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.354496  7526 raft_consensus.cc:2272] T 05559e602d4b4ea3986701fa621706a0 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.356951  7526 tablet_replica.cc:333] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.357784  7526 raft_consensus.cc:2243] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.358489  7526 raft_consensus.cc:2272] T 293a9dab5be3479a90d5803585682a00 P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.361797  7526 tablet_replica.cc:333] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.362805  7526 raft_consensus.cc:2243] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.364189  7526 raft_consensus.cc:2272] T 3a7eadf5c488447886a90612fbf1a2bb P 7aff04f53d4845a6a4fa491fd2a06145 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.367373  7526 tablet_replica.cc:333] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.368252  7526 raft_consensus.cc:2243] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.368881  7526 raft_consensus.cc:2272] T 41d15a8ebfe8480d9861c1285faaf7b5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.372000  7526 tablet_replica.cc:333] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.372681  7526 raft_consensus.cc:2243] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.373786  7526 raft_consensus.cc:2272] T 3169f7f9f5a7440398ae5ebe01fc3c3f P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.376540  7526 tablet_replica.cc:333] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.377269  7526 raft_consensus.cc:2243] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.378372  7526 raft_consensus.cc:2272] T 142891c077a34ba289adf90a99aee612 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.381034  7526 tablet_replica.cc:333] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.381686  7526 raft_consensus.cc:2243] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.382727  7526 raft_consensus.cc:2272] T 1f9ddea0a887446f910d38f8302e8172 P 7aff04f53d4845a6a4fa491fd2a06145 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.385156  7526 tablet_replica.cc:333] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.385794  7526 raft_consensus.cc:2243] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.386891  7526 raft_consensus.cc:2272] T 5ee1e688ccd74392befec90535c1efa4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.389258  7526 tablet_replica.cc:333] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.389909  7526 raft_consensus.cc:2243] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.390987  7526 raft_consensus.cc:2272] T c01533e2a5204b1ebccac1133e495542 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.393575  7526 tablet_replica.cc:333] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.394264  7526 raft_consensus.cc:2243] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.395638  7526 raft_consensus.cc:2272] T 987153bedd0c4f86b726db1edfd1eec6 P 7aff04f53d4845a6a4fa491fd2a06145 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.398213  7526 tablet_replica.cc:333] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.399050  7526 raft_consensus.cc:2243] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.400187  7526 raft_consensus.cc:2272] T 4de6a8f976f5458d807b777f3ac94a23 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.402832  7526 tablet_replica.cc:333] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.403694  7526 raft_consensus.cc:2243] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.404448  7526 raft_consensus.cc:2272] T 300460316b0849bf8014930d5e9bfada P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.407702  7526 tablet_replica.cc:333] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.408387  7526 raft_consensus.cc:2243] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.409441  7526 raft_consensus.cc:2272] T 8611228be8dd4f08b2b609e202974a6a P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.411746  7526 tablet_replica.cc:333] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.412392  7526 raft_consensus.cc:2243] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.412964  7526 raft_consensus.cc:2272] T b50b38d6e5314ca190c9a14a5e0f655d P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.415441  7526 tablet_replica.cc:333] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.416092  7526 raft_consensus.cc:2243] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.416612  7526 raft_consensus.cc:2272] T 8acd9840694e4f62803bfe9810425202 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.419312  7526 tablet_replica.cc:333] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.419982  7526 raft_consensus.cc:2243] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.420660  7526 raft_consensus.cc:2272] T 295d39c61dd3489e98d9393a2da9c062 P 7aff04f53d4845a6a4fa491fd2a06145 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.423430  7526 tablet_replica.cc:333] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.424072  7526 raft_consensus.cc:2243] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.424669  7526 raft_consensus.cc:2272] T 396dbfb60a3644b9b28c39c3dd36b322 P 7aff04f53d4845a6a4fa491fd2a06145 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.427059  7526 tablet_replica.cc:333] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.427783  7526 raft_consensus.cc:2243] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.428944  7526 raft_consensus.cc:2272] T 1c963cd681ab4f45a5c339f9fd6e2687 P 7aff04f53d4845a6a4fa491fd2a06145 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.431310  7526 tablet_replica.cc:333] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.431942  7526 raft_consensus.cc:2243] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.432585  7526 raft_consensus.cc:2272] T 20fcd6d0c3304abda83f416721e807c5 P 7aff04f53d4845a6a4fa491fd2a06145 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.435017  7526 tablet_replica.cc:333] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.435665  7526 raft_consensus.cc:2243] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.436815  7526 raft_consensus.cc:2272] T 9f81d7aa5ed04c68b603d272f24760e2 P 7aff04f53d4845a6a4fa491fd2a06145 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.439688  7526 tablet_replica.cc:333] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.440376  7526 raft_consensus.cc:2243] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.441536  7526 raft_consensus.cc:2272] T ac4bab9153b047238bffb6abbeb83601 P 7aff04f53d4845a6a4fa491fd2a06145 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.444248  7526 tablet_replica.cc:333] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.444847  7526 raft_consensus.cc:2243] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:19.445657  7526 raft_consensus.cc:2272] T 2147fd3c99a94f0c8e9c1a914bdb79c3 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.448377  7526 tablet_replica.cc:333] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.448978  7526 raft_consensus.cc:2243] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.450093  7526 raft_consensus.cc:2272] T 389e9e63581e4d1b941ba995f8cdbfc4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:19.452895  7526 tablet_replica.cc:333] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145: stopping tablet replica
I20260709 13:06:19.453576  7526 raft_consensus.cc:2243] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:19.454963  7526 raft_consensus.cc:2272] T 1e6257b62c5c415aad31f5fc646ce4d4 P 7aff04f53d4845a6a4fa491fd2a06145 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:20.529223  7526 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 9410)
I20260709 13:06:21.114547  7526 tablet_server.cc:196] TabletServer@127.7.89.132:0 shutdown complete.
I20260709 13:06:21.187844  7526 master.cc:562] Master@127.7.89.190:45099 shutting down...
W20260709 13:06:21.225656  8627 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:06:21.226070  8627 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:06:21.226326  8627 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:06:36.838204  8610 debug-util.cc:398] Leaking SignalData structure 0x7b0800401ac0 after lost signal to thread 7529
W20260709 13:06:36.839407  8610 debug-util.cc:398] Leaking SignalData structure 0x7b0800356960 after lost signal to thread 8613
W20260709 13:06:47.431856  8563 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111) [suppressed 188 similar messages]
W20260709 13:06:47.436288  8563 catalog_manager.cc:4755] TS 0e8c59a1f0394eae8ce33b99f10efadc (127.7.89.129:43221): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2147fd3c99a94f0c8e9c1a914bdb79c3: Network error: Client connection negotiation failed: client connection to 127.7.89.129:43221: connect: Connection refused (error 111)
I20260709 13:06:47.801064  7526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:47.803573  7526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:47.804162  7526 tablet_replica.cc:333] T 00000000000000000000000000000000 P 06d3808b553142ef8f7a4e57845757cd: stopping tablet replica
I20260709 13:06:48.030470  7526 master.cc:584] Master@127.7.89.190:45099 shutdown complete.
I20260709 13:06:48.097019  7526 test_util.cc:182] -----------------------------------------------
I20260709 13:06:48.097389  7526 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602061986899-7526-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (319591 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:06:48.102284  7526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.89.190:45519
I20260709 13:06:48.103767  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:48.110459 17283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:48.110592 17284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:48.113322 17286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:48.113354  7526 server_base.cc:1061] running on GCE node
I20260709 13:06:48.118367  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:48.118714  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:48.118904  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602408118888 us; error 0 us; skew 500 ppm
I20260709 13:06:48.119546  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:48.122537  7526 webserver.cc:533] Webserver started at http://127.7.89.190:33489/ using document root <none> and password file <none>
I20260709 13:06:48.123215  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:48.123420  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:48.123744  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:48.125265  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/master-0-root/instance:
uuid: "dead242a90f4462c89f826224a05a3de"
format_stamp: "Formatted at 2026-07-09 13:06:48 on dist-test-slave-h0rz"
I20260709 13:06:48.130901  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:06:48.135011 17291 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:48.135951  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:06:48.136298  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/master-0-root
uuid: "dead242a90f4462c89f826224a05a3de"
format_stamp: "Formatted at 2026-07-09 13:06:48 on dist-test-slave-h0rz"
I20260709 13:06:48.136673  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:48.162979  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:48.164619  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:48.213616  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.190:45519
I20260709 13:06:48.213747 17342 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.190:45519 every 8 connection(s)
I20260709 13:06:48.218600 17343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.237797 17343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de: Bootstrap starting.
I20260709 13:06:48.244527 17343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.249943 17343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de: No bootstrap required, opened a new log
I20260709 13:06:48.252739 17343 raft_consensus.cc:359] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dead242a90f4462c89f826224a05a3de" member_type: VOTER }
I20260709 13:06:48.253302 17343 raft_consensus.cc:385] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.253497 17343 raft_consensus.cc:740] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dead242a90f4462c89f826224a05a3de, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.254379 17343 consensus_queue.cc:260] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dead242a90f4462c89f826224a05a3de" member_type: VOTER }
I20260709 13:06:48.255055 17343 raft_consensus.cc:399] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:48.255295 17343 raft_consensus.cc:493] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:48.255540 17343 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:48.261502 17343 raft_consensus.cc:515] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dead242a90f4462c89f826224a05a3de" member_type: VOTER }
I20260709 13:06:48.262144 17343 leader_election.cc:304] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dead242a90f4462c89f826224a05a3de; no voters: 
I20260709 13:06:48.263649 17343 leader_election.cc:290] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:48.264183 17346 raft_consensus.cc:2804] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:48.265877 17346 raft_consensus.cc:697] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [term 1 LEADER]: Becoming Leader. State: Replica: dead242a90f4462c89f826224a05a3de, State: Running, Role: LEADER
I20260709 13:06:48.266742 17346 consensus_queue.cc:237] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dead242a90f4462c89f826224a05a3de" member_type: VOTER }
I20260709 13:06:48.267261 17343 sys_catalog.cc:565] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:48.278038 17347 sys_catalog.cc:455] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dead242a90f4462c89f826224a05a3de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dead242a90f4462c89f826224a05a3de" member_type: VOTER } }
I20260709 13:06:48.278029 17348 sys_catalog.cc:455] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [sys.catalog]: SysCatalogTable state changed. Reason: New leader dead242a90f4462c89f826224a05a3de. Latest consensus state: current_term: 1 leader_uuid: "dead242a90f4462c89f826224a05a3de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dead242a90f4462c89f826224a05a3de" member_type: VOTER } }
I20260709 13:06:48.279132 17347 sys_catalog.cc:458] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:48.279156 17348 sys_catalog.cc:458] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:48.281633 17356 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:48.290580 17356 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:48.292083  7526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:48.303040 17356 catalog_manager.cc:1383] Generated new cluster ID: 31fb5ef729494f93847bc27c10c4ab51
I20260709 13:06:48.303375 17356 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:48.336560 17356 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:48.338131 17356 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:48.359206 17356 catalog_manager.cc:6070] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de: Generated new TSK 0
I20260709 13:06:48.359964 17356 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:48.424521  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:48.432744 17364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:48.434365 17365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:48.436744 17367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:48.438014  7526 server_base.cc:1061] running on GCE node
I20260709 13:06:48.442137  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:48.442497  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:48.442793  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602408442768 us; error 0 us; skew 500 ppm
I20260709 13:06:48.443427  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:48.446382  7526 webserver.cc:533] Webserver started at http://127.7.89.129:40525/ using document root <none> and password file <none>
I20260709 13:06:48.447086  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:48.447314  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:48.447633  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:48.449061  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-0-root/instance:
uuid: "337bbc857b9b4e329c817b81bfb0971c"
format_stamp: "Formatted at 2026-07-09 13:06:48 on dist-test-slave-h0rz"
I20260709 13:06:48.454717  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:06:48.458856 17372 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:48.459744  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:06:48.460115  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-0-root
uuid: "337bbc857b9b4e329c817b81bfb0971c"
format_stamp: "Formatted at 2026-07-09 13:06:48 on dist-test-slave-h0rz"
I20260709 13:06:48.460486  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:48.473832  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:48.475510  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:48.477308  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:48.480435  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:48.480705  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:48.481099  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:48.481349  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:48.534235  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.129:34943
I20260709 13:06:48.534382 17434 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.129:34943 every 8 connection(s)
I20260709 13:06:48.540747  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:48.550083 17439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:48.554510 17442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:48.555433  7526 server_base.cc:1061] running on GCE node
W20260709 13:06:48.557013 17440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:48.557508 17435 heartbeater.cc:344] Connected to a master server at 127.7.89.190:45519
I20260709 13:06:48.558197 17435 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:48.559252 17435 heartbeater.cc:507] Master 127.7.89.190:45519 requested a full tablet report, sending...
I20260709 13:06:48.560156  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:48.560501  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:48.560738  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602408560715 us; error 0 us; skew 500 ppm
I20260709 13:06:48.561623  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:48.562168 17308 ts_manager.cc:194] Registered new tserver with Master: 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:48.564829 17308 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40050
I20260709 13:06:48.565193  7526 webserver.cc:533] Webserver started at http://127.7.89.130:33683/ using document root <none> and password file <none>
I20260709 13:06:48.565961  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:48.566237  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:48.566597  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:48.568783  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-1-root/instance:
uuid: "4df2ca35be5341afa5a621233a4da94e"
format_stamp: "Formatted at 2026-07-09 13:06:48 on dist-test-slave-h0rz"
I20260709 13:06:48.574286  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 13:06:48.578742 17447 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:48.579825  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:48.580233  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-1-root
uuid: "4df2ca35be5341afa5a621233a4da94e"
format_stamp: "Formatted at 2026-07-09 13:06:48 on dist-test-slave-h0rz"
I20260709 13:06:48.580542  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:48.600634  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:48.602172  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:48.604025  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:48.606921  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:48.607177  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:48.607646  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:48.607846  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:48.659330  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.130:36599
I20260709 13:06:48.659497 17509 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.130:36599 every 8 connection(s)
I20260709 13:06:48.666195  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:48.675307 17513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:48.676746 17514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:48.681341  7526 server_base.cc:1061] running on GCE node
W20260709 13:06:48.682005 17516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:48.682991 17510 heartbeater.cc:344] Connected to a master server at 127.7.89.190:45519
I20260709 13:06:48.683405 17510 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:48.683470  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:48.683833  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:48.684036  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602408684017 us; error 0 us; skew 500 ppm
I20260709 13:06:48.684371 17510 heartbeater.cc:507] Master 127.7.89.190:45519 requested a full tablet report, sending...
I20260709 13:06:48.684738  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:48.687031 17308 ts_manager.cc:194] Registered new tserver with Master: 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:48.688220  7526 webserver.cc:533] Webserver started at http://127.7.89.131:34467/ using document root <none> and password file <none>
I20260709 13:06:48.688969  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:48.689231  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:48.689234 17308 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40054
I20260709 13:06:48.689718  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:48.691461  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-2-root/instance:
uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
format_stamp: "Formatted at 2026-07-09 13:06:48 on dist-test-slave-h0rz"
I20260709 13:06:48.696776  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:06:48.700802 17521 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:48.701609  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:06:48.701952  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-2-root
uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
format_stamp: "Formatted at 2026-07-09 13:06:48 on dist-test-slave-h0rz"
I20260709 13:06:48.702278  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:48.719771  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:48.721271  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:48.723062  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:48.725586  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:48.725808  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:48.726083  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:48.726248  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:48.775377  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.131:40851
I20260709 13:06:48.775488 17583 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.131:40851 every 8 connection(s)
I20260709 13:06:48.781459  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:48.791131 17587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:48.793735 17588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:48.797623 17590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:48.798578 17584 heartbeater.cc:344] Connected to a master server at 127.7.89.190:45519
I20260709 13:06:48.798976  7526 server_base.cc:1061] running on GCE node
I20260709 13:06:48.799168 17584 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:48.800077  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:48.800350  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:48.800226 17584 heartbeater.cc:507] Master 127.7.89.190:45519 requested a full tablet report, sending...
I20260709 13:06:48.800683  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602408800665 us; error 0 us; skew 500 ppm
I20260709 13:06:48.801498  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:48.803073 17308 ts_manager.cc:194] Registered new tserver with Master: 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:48.805203  7526 webserver.cc:533] Webserver started at http://127.7.89.132:39931/ using document root <none> and password file <none>
I20260709 13:06:48.805290 17308 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40062
I20260709 13:06:48.806135  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:48.806432  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:48.806839  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:48.808204  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-3-root/instance:
uuid: "a9a65d4c22be403682be9bc817650ada"
format_stamp: "Formatted at 2026-07-09 13:06:48 on dist-test-slave-h0rz"
I20260709 13:06:48.813901  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:06:48.818290 17595 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:48.819267  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:48.819644  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-3-root
uuid: "a9a65d4c22be403682be9bc817650ada"
format_stamp: "Formatted at 2026-07-09 13:06:48 on dist-test-slave-h0rz"
I20260709 13:06:48.820009  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:48.857278  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:48.858925  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:48.860417  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:48.862960  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:48.863225  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:48.863497  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:48.863687  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:48.916400  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.132:40637
I20260709 13:06:48.916500 17657 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.132:40637 every 8 connection(s)
I20260709 13:06:48.934731 17658 heartbeater.cc:344] Connected to a master server at 127.7.89.190:45519
I20260709 13:06:48.935247 17658 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:48.936198 17658 heartbeater.cc:507] Master 127.7.89.190:45519 requested a full tablet report, sending...
I20260709 13:06:48.938781 17308 ts_manager.cc:194] Registered new tserver with Master: a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:48.939134  7526 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.01822809s
I20260709 13:06:48.940488 17308 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40072
I20260709 13:06:48.973256 17308 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40080:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:06:49.249316 17474 tablet_service.cc:1511] Processing CreateTablet for tablet b5fb377eea3d41e68cb4f46212e6e857 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:49.251237 17474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5fb377eea3d41e68cb4f46212e6e857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.256135 17473 tablet_service.cc:1511] Processing CreateTablet for tablet 5a9a47a9cc56480fada4e8084a44682a (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:49.258169 17473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a9a47a9cc56480fada4e8084a44682a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.260172 17475 tablet_service.cc:1511] Processing CreateTablet for tablet 800f4d4505654dc58f390fcc9f0d90db (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:49.264067 17472 tablet_service.cc:1511] Processing CreateTablet for tablet 0f7e0518788149c19cf493a65bc598c2 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:49.267623 17622 tablet_service.cc:1511] Processing CreateTablet for tablet 8f8bf7f6c34a4edfb28a4914ad4ce49e (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:49.268117 17620 tablet_service.cc:1511] Processing CreateTablet for tablet 2ccac61a2bab40708cfe17c17ae1de65 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:49.268715 17621 tablet_service.cc:1511] Processing CreateTablet for tablet 5a9a47a9cc56480fada4e8084a44682a (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:49.269596 17622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f8bf7f6c34a4edfb28a4914ad4ce49e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.269683 17623 tablet_service.cc:1511] Processing CreateTablet for tablet b5fb377eea3d41e68cb4f46212e6e857 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:49.270639 17621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a9a47a9cc56480fada4e8084a44682a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.271473 17623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5fb377eea3d41e68cb4f46212e6e857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.273432 17620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ccac61a2bab40708cfe17c17ae1de65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.273274 17471 tablet_service.cc:1511] Processing CreateTablet for tablet 2ccac61a2bab40708cfe17c17ae1de65 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:49.273838 17549 tablet_service.cc:1511] Processing CreateTablet for tablet 800f4d4505654dc58f390fcc9f0d90db (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:49.274257 17548 tablet_service.cc:1511] Processing CreateTablet for tablet 8f8bf7f6c34a4edfb28a4914ad4ce49e (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:49.275207 17472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7e0518788149c19cf493a65bc598c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.275483 17549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 800f4d4505654dc58f390fcc9f0d90db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.276141 17471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ccac61a2bab40708cfe17c17ae1de65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.278724 17475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 800f4d4505654dc58f390fcc9f0d90db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.280376 17548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f8bf7f6c34a4edfb28a4914ad4ce49e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:49.281401 17355 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:49.285631 17547 tablet_service.cc:1511] Processing CreateTablet for tablet 5a9a47a9cc56480fada4e8084a44682a (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:49.288015 17547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a9a47a9cc56480fada4e8084a44682a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.297027 17546 tablet_service.cc:1511] Processing CreateTablet for tablet 0f7e0518788149c19cf493a65bc598c2 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:49.298933 17546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7e0518788149c19cf493a65bc598c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.315510 17545 tablet_service.cc:1511] Processing CreateTablet for tablet a5414b9916b148bba0932c91f3770f57 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:49.317504 17545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5414b9916b148bba0932c91f3770f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.320943 17619 tablet_service.cc:1511] Processing CreateTablet for tablet a5414b9916b148bba0932c91f3770f57 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:49.323300 17619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5414b9916b148bba0932c91f3770f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.335726 17396 tablet_service.cc:1511] Processing CreateTablet for tablet 2ccac61a2bab40708cfe17c17ae1de65 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:49.337355 17398 tablet_service.cc:1511] Processing CreateTablet for tablet 8f8bf7f6c34a4edfb28a4914ad4ce49e (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:49.335745 17397 tablet_service.cc:1511] Processing CreateTablet for tablet 0f7e0518788149c19cf493a65bc598c2 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:49.340094 17400 tablet_service.cc:1511] Processing CreateTablet for tablet 800f4d4505654dc58f390fcc9f0d90db (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:49.343219 17399 tablet_service.cc:1511] Processing CreateTablet for tablet b5fb377eea3d41e68cb4f46212e6e857 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:49.337805 17396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ccac61a2bab40708cfe17c17ae1de65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.347173 17397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7e0518788149c19cf493a65bc598c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.348451 17678 tablet_bootstrap.cc:492] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:49.351132 17400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 800f4d4505654dc58f390fcc9f0d90db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.355113 17398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f8bf7f6c34a4edfb28a4914ad4ce49e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.357820 17677 tablet_bootstrap.cc:492] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:49.359115 17399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5fb377eea3d41e68cb4f46212e6e857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.368777 17677 tablet_bootstrap.cc:654] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.375056 17678 tablet_bootstrap.cc:654] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.378365 17395 tablet_service.cc:1511] Processing CreateTablet for tablet a5414b9916b148bba0932c91f3770f57 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:49.382941 17395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5414b9916b148bba0932c91f3770f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.384327 17471 tablet_service.cc:1511] Processing CreateTablet for tablet eef6e41b27ca4b229606bc1df6b023fb (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:49.386124 17471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eef6e41b27ca4b229606bc1df6b023fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.417719 17619 tablet_service.cc:1511] Processing CreateTablet for tablet eef6e41b27ca4b229606bc1df6b023fb (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:49.418669 17394 tablet_service.cc:1511] Processing CreateTablet for tablet 37b734b40edd440fad746ea61f858c1b (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:49.425848 17394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37b734b40edd440fad746ea61f858c1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.431375 17475 tablet_service.cc:1511] Processing CreateTablet for tablet 37b734b40edd440fad746ea61f858c1b (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:49.433053 17475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37b734b40edd440fad746ea61f858c1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.438030 17678 tablet_bootstrap.cc:492] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:49.438830 17678 ts_tablet_manager.cc:1403] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.091s	user 0.008s	sys 0.012s
I20260709 13:06:49.443411 17549 tablet_service.cc:1511] Processing CreateTablet for tablet eef6e41b27ca4b229606bc1df6b023fb (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:49.445180 17548 tablet_service.cc:1511] Processing CreateTablet for tablet 37b734b40edd440fad746ea61f858c1b (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:49.445261 17623 tablet_service.cc:1511] Processing CreateTablet for tablet 01c85566c0a845438507d71c6936d507 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:06:49.438868 17619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eef6e41b27ca4b229606bc1df6b023fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.454212 17393 tablet_service.cc:1511] Processing CreateTablet for tablet 01c85566c0a845438507d71c6936d507 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:06:49.460924 17548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37b734b40edd440fad746ea61f858c1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.479759 17393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c85566c0a845438507d71c6936d507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.491303 17623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c85566c0a845438507d71c6936d507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.468760 17678 raft_consensus.cc:359] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:49.502800 17678 raft_consensus.cc:385] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.503320 17678 raft_consensus.cc:740] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.504812 17678 consensus_queue.cc:260] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:49.514933 17549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eef6e41b27ca4b229606bc1df6b023fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.524338 17681 tablet_bootstrap.cc:492] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:49.525678 17473 tablet_service.cc:1511] Processing CreateTablet for tablet 01c85566c0a845438507d71c6936d507 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:06:49.527427 17473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c85566c0a845438507d71c6936d507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.530500 17677 tablet_bootstrap.cc:492] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:49.531316 17677 ts_tablet_manager.cc:1403] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.174s	user 0.038s	sys 0.041s
I20260709 13:06:49.533521 17682 tablet_bootstrap.cc:492] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:49.549721 17677 raft_consensus.cc:359] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:49.550662 17677 raft_consensus.cc:385] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.552191 17677 raft_consensus.cc:740] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.554318 17547 tablet_service.cc:1511] Processing CreateTablet for tablet 8259e46a7776417a80d1be42d4419a13 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:06:49.553332 17677 consensus_queue.cc:260] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:49.556329 17682 tablet_bootstrap.cc:654] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.556561 17620 tablet_service.cc:1511] Processing CreateTablet for tablet 8259e46a7776417a80d1be42d4419a13 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:06:49.558074 17620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8259e46a7776417a80d1be42d4419a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.561299 17399 tablet_service.cc:1511] Processing CreateTablet for tablet 8259e46a7776417a80d1be42d4419a13 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:06:49.563007 17547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8259e46a7776417a80d1be42d4419a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.563258 17399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8259e46a7776417a80d1be42d4419a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.567724 17681 tablet_bootstrap.cc:654] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.568965 17435 heartbeater.cc:499] Master 127.7.89.190:45519 was elected leader, sending a full tablet report...
I20260709 13:06:49.575573 17546 tablet_service.cc:1511] Processing CreateTablet for tablet 5d19239b38b1485ca53f239a71a11444 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:49.576542 17678 ts_tablet_manager.cc:1434] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.137s	user 0.007s	sys 0.012s
I20260709 13:06:49.577407 17546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d19239b38b1485ca53f239a71a11444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.577838 17658 heartbeater.cc:499] Master 127.7.89.190:45519 was elected leader, sending a full tablet report...
I20260709 13:06:49.588837 17471 tablet_service.cc:1511] Processing CreateTablet for tablet 5d19239b38b1485ca53f239a71a11444 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:49.599189 17678 tablet_bootstrap.cc:492] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:49.622236 17471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d19239b38b1485ca53f239a71a11444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.648420 17510 heartbeater.cc:499] Master 127.7.89.190:45519 was elected leader, sending a full tablet report...
I20260709 13:06:49.648880 17677 ts_tablet_manager.cc:1434] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.117s	user 0.016s	sys 0.023s
I20260709 13:06:49.665483 17677 tablet_bootstrap.cc:492] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
W20260709 13:06:49.665839 17511 tablet.cc:2406] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:06:49.667375 17681 tablet_bootstrap.cc:492] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:49.668026 17681 ts_tablet_manager.cc:1403] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.144s	user 0.018s	sys 0.000s
I20260709 13:06:49.675158 17677 tablet_bootstrap.cc:654] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.681521 17682 tablet_bootstrap.cc:492] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:49.681986 17678 tablet_bootstrap.cc:654] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.682184 17682 ts_tablet_manager.cc:1403] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.149s	user 0.038s	sys 0.046s
I20260709 13:06:49.692813 17677 tablet_bootstrap.cc:492] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:49.693562 17677 ts_tablet_manager.cc:1403] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.002s
I20260709 13:06:49.698361 17620 tablet_service.cc:1511] Processing CreateTablet for tablet 5d19239b38b1485ca53f239a71a11444 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:49.699693 17620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d19239b38b1485ca53f239a71a11444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.700409 17396 tablet_service.cc:1511] Processing CreateTablet for tablet bd407ea24c014a8690ac37978d4b0b50 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:49.701984 17396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd407ea24c014a8690ac37978d4b0b50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.702481 17682 raft_consensus.cc:359] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:49.703418 17682 raft_consensus.cc:385] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.707881 17683 raft_consensus.cc:493] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.708331 17475 tablet_service.cc:1511] Processing CreateTablet for tablet bd407ea24c014a8690ac37978d4b0b50 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:49.708518 17683 raft_consensus.cc:515] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:49.694964 17681 raft_consensus.cc:359] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:49.709893 17475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd407ea24c014a8690ac37978d4b0b50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.710036 17681 raft_consensus.cc:385] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.710443 17681 raft_consensus.cc:740] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.711776 17683 leader_election.cc:290] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:49.711393 17681 consensus_queue.cc:260] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:49.706846 17677 raft_consensus.cc:359] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:49.717254 17547 tablet_service.cc:1511] Processing CreateTablet for tablet bd407ea24c014a8690ac37978d4b0b50 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:49.718815 17547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd407ea24c014a8690ac37978d4b0b50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.720197 17678 tablet_bootstrap.cc:492] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:49.722846 17678 ts_tablet_manager.cc:1403] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.124s	user 0.010s	sys 0.008s
I20260709 13:06:49.716741 17677 raft_consensus.cc:385] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.714476 17682 raft_consensus.cc:740] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.736390 17623 tablet_service.cc:1511] Processing CreateTablet for tablet 6633559a9d2643a29622caab848e5442 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:06:49.738035 17623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6633559a9d2643a29622caab848e5442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.727660 17677 raft_consensus.cc:740] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.739513 17677 consensus_queue.cc:260] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:49.742043 17677 ts_tablet_manager.cc:1434] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.005s
I20260709 13:06:49.755298 17678 raft_consensus.cc:359] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:49.756644 17678 raft_consensus.cc:385] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.757104 17678 raft_consensus.cc:740] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.759773 17677 tablet_bootstrap.cc:492] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:49.767764 17682 consensus_queue.cc:260] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:49.771015 17549 tablet_service.cc:1511] Processing CreateTablet for tablet 61235f863ca44ed3b6c4715dfd68e223 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:49.771870 17584 heartbeater.cc:499] Master 127.7.89.190:45519 was elected leader, sending a full tablet report...
I20260709 13:06:49.758292 17678 consensus_queue.cc:260] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:49.772710 17549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61235f863ca44ed3b6c4715dfd68e223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.774197 17678 ts_tablet_manager.cc:1434] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.001s
I20260709 13:06:49.778083 17678 tablet_bootstrap.cc:492] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:49.787047 17399 tablet_service.cc:1511] Processing CreateTablet for tablet 61235f863ca44ed3b6c4715dfd68e223 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:49.788352 17681 ts_tablet_manager.cc:1434] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.120s	user 0.022s	sys 0.005s
I20260709 13:06:49.788954 17399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61235f863ca44ed3b6c4715dfd68e223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.789812 17681 tablet_bootstrap.cc:492] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:49.792093 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9a47a9cc56480fada4e8084a44682a" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:06:49.794137 17596 leader_election.cc:343] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.795678 17475 tablet_service.cc:1511] Processing CreateTablet for tablet 6633559a9d2643a29622caab848e5442 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:06:49.796770 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9a47a9cc56480fada4e8084a44682a" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:49.797286 17475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6633559a9d2643a29622caab848e5442. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:49.798873 17599 leader_election.cc:343] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.799476 17599 leader_election.cc:304] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:49.787047 17396 tablet_service.cc:1511] Processing CreateTablet for tablet 6633559a9d2643a29622caab848e5442 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:06:49.800889 17683 raft_consensus.cc:2749] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.805490 17471 tablet_service.cc:1511] Processing CreateTablet for tablet e559727ff02540749beb944b741d471f (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:49.822813 17623 tablet_service.cc:1511] Processing CreateTablet for tablet e559727ff02540749beb944b741d471f (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:49.824270 17471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e559727ff02540749beb944b741d471f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.824275 17623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e559727ff02540749beb944b741d471f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.824970 17396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6633559a9d2643a29622caab848e5442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.807888 17682 ts_tablet_manager.cc:1434] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.120s	user 0.014s	sys 0.001s
I20260709 13:06:49.809336 17681 tablet_bootstrap.cc:654] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.812114 17548 tablet_service.cc:1511] Processing CreateTablet for tablet e559727ff02540749beb944b741d471f (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:49.831137 17548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e559727ff02540749beb944b741d471f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.834822 17620 tablet_service.cc:1511] Processing CreateTablet for tablet 61235f863ca44ed3b6c4715dfd68e223 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:49.836191 17677 tablet_bootstrap.cc:654] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.836577 17620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61235f863ca44ed3b6c4715dfd68e223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.839105 17682 tablet_bootstrap.cc:492] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:49.849121 17393 tablet_service.cc:1511] Processing CreateTablet for tablet 2d047496a466464ba1777cf63c5c84b0 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:49.852777 17393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d047496a466464ba1777cf63c5c84b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.855242 17685 raft_consensus.cc:493] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.855664 17475 tablet_service.cc:1511] Processing CreateTablet for tablet 2d047496a466464ba1777cf63c5c84b0 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:49.855978 17685 raft_consensus.cc:515] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:49.857188 17475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d047496a466464ba1777cf63c5c84b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.858982 17691 raft_consensus.cc:493] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.860070 17682 tablet_bootstrap.cc:654] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.859644 17691 raft_consensus.cc:515] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:49.864512 17691 leader_election.cc:290] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:49.868387 17678 tablet_bootstrap.cc:654] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.867420 17685 leader_election.cc:290] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:49.893991 17681 tablet_bootstrap.cc:492] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:49.894796 17681 ts_tablet_manager.cc:1403] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.105s	user 0.016s	sys 0.010s
I20260709 13:06:49.886746 17685 raft_consensus.cc:493] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.902351 17623 tablet_service.cc:1511] Processing CreateTablet for tablet a7f14faf54714af3abeff5693ace0e8c (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:49.905131 17623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7f14faf54714af3abeff5693ace0e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.921516 17473 tablet_service.cc:1511] Processing CreateTablet for tablet a7f14faf54714af3abeff5693ace0e8c (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:49.910759 17685 raft_consensus.cc:515] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:49.932483 17549 tablet_service.cc:1511] Processing CreateTablet for tablet 2d047496a466464ba1777cf63c5c84b0 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:49.935138 17549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d047496a466464ba1777cf63c5c84b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.943672 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800f4d4505654dc58f390fcc9f0d90db" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:49.936489 17473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7f14faf54714af3abeff5693ace0e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.959833 17685 leader_election.cc:290] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:49.977269 17547 tablet_service.cc:1511] Processing CreateTablet for tablet 944f8419f8c94888bb53565c46d6f69b (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:49.981724 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800f4d4505654dc58f390fcc9f0d90db" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:49.983975 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ccac61a2bab40708cfe17c17ae1de65" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
W20260709 13:06:49.988034 17450 leader_election.cc:343] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.991320 17547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 944f8419f8c94888bb53565c46d6f69b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:49.991238 17450 leader_election.cc:343] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.998937 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7e0518788149c19cf493a65bc598c2" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:49.998648 17399 tablet_service.cc:1511] Processing CreateTablet for tablet a7f14faf54714af3abeff5693ace0e8c (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:50.000349 17399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7f14faf54714af3abeff5693ace0e8c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:50.001401 17524 leader_election.cc:343] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.018504 17619 tablet_service.cc:1511] Processing CreateTablet for tablet 944f8419f8c94888bb53565c46d6f69b (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:50.020372 17619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 944f8419f8c94888bb53565c46d6f69b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.020840 17472 tablet_service.cc:1511] Processing CreateTablet for tablet 09bc57cc906f4179a8f6ff86c42b0247 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:50.023146 17472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09bc57cc906f4179a8f6ff86c42b0247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.917045 17681 raft_consensus.cc:359] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:50.023260 17682 tablet_bootstrap.cc:492] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:50.027421 17683 raft_consensus.cc:493] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.030417 17683 raft_consensus.cc:515] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:50.033368 17683 leader_election.cc:290] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:50.036368 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fb377eea3d41e68cb4f46212e6e857" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:06:50.039167 17596 leader_election.cc:343] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.039740 17396 tablet_service.cc:1511] Processing CreateTablet for tablet 944f8419f8c94888bb53565c46d6f69b (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:50.041356 17396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 944f8419f8c94888bb53565c46d6f69b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.027204 17681 raft_consensus.cc:385] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.048699 17678 tablet_bootstrap.cc:492] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:50.049448 17678 ts_tablet_manager.cc:1403] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.272s	user 0.022s	sys 0.015s
I20260709 13:06:50.045701 17682 ts_tablet_manager.cc:1403] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.207s	user 0.025s	sys 0.028s
I20260709 13:06:50.052253 17622 tablet_service.cc:1511] Processing CreateTablet for tablet 09bc57cc906f4179a8f6ff86c42b0247 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:50.053771 17678 raft_consensus.cc:359] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:50.058383 17622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09bc57cc906f4179a8f6ff86c42b0247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.058178 17681 raft_consensus.cc:740] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.032255 17548 tablet_service.cc:1511] Processing CreateTablet for tablet 09bc57cc906f4179a8f6ff86c42b0247 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:50.066424 17681 consensus_queue.cc:260] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:50.063160 17678 raft_consensus.cc:385] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.067881 17394 tablet_service.cc:1511] Processing CreateTablet for tablet b1f46bc9df3a44dd927d40f3778a5ec3 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:50.073277 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fb377eea3d41e68cb4f46212e6e857" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:50.083552 17409 raft_consensus.cc:2468] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 0.
I20260709 13:06:50.086787 17598 leader_election.cc:304] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:50.088505 17683 raft_consensus.cc:2804] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.088967 17683 raft_consensus.cc:493] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.089296 17683 raft_consensus.cc:3060] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.086495 17394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1f46bc9df3a44dd927d40f3778a5ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.096539 17471 tablet_service.cc:1511] Processing CreateTablet for tablet b1f46bc9df3a44dd927d40f3778a5ec3 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:50.098336 17471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1f46bc9df3a44dd927d40f3778a5ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.076745 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7e0518788149c19cf493a65bc598c2" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:50.079035 17559 raft_consensus.cc:2468] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:50.103703 17682 raft_consensus.cc:359] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:50.104856 17682 raft_consensus.cc:385] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.105257 17682 raft_consensus.cc:740] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.106375 17682 consensus_queue.cc:260] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:50.109754 17677 tablet_bootstrap.cc:492] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:50.110664 17451 leader_election.cc:304] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:50.111856 17621 tablet_service.cc:1511] Processing CreateTablet for tablet 36168ca382614c3f9061a116e5d12df5 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:50.112568 17699 raft_consensus.cc:2804] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.113116 17699 raft_consensus.cc:493] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.113413 17621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36168ca382614c3f9061a116e5d12df5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.115046 17682 ts_tablet_manager.cc:1434] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.062s	user 0.005s	sys 0.004s
I20260709 13:06:50.116371 17682 tablet_bootstrap.cc:492] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:50.116614 17546 tablet_service.cc:1511] Processing CreateTablet for tablet b1f46bc9df3a44dd927d40f3778a5ec3 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:50.116709 17393 tablet_service.cc:1511] Processing CreateTablet for tablet 36168ca382614c3f9061a116e5d12df5 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:50.118299 17393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36168ca382614c3f9061a116e5d12df5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.102108 17678 raft_consensus.cc:740] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.119817 17678 consensus_queue.cc:260] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:50.110471 17677 ts_tablet_manager.cc:1403] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.351s	user 0.024s	sys 0.019s
W20260709 13:06:50.123179 17522 leader_election.cc:343] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.123828 17522 leader_election.cc:304] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:50.125263 17691 raft_consensus.cc:2749] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.125701 17677 raft_consensus.cc:359] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:50.128796 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ccac61a2bab40708cfe17c17ae1de65" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:50.129073 17475 tablet_service.cc:1511] Processing CreateTablet for tablet 36168ca382614c3f9061a116e5d12df5 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:50.130695 17475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36168ca382614c3f9061a116e5d12df5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:50.130666 17450 leader_election.cc:343] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.131345 17450 leader_election.cc:304] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:50.132725 17685 raft_consensus.cc:2749] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.113518 17699 raft_consensus.cc:3060] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.076884 17548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09bc57cc906f4179a8f6ff86c42b0247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.093351 17681 ts_tablet_manager.cc:1434] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.198s	user 0.003s	sys 0.007s
I20260709 13:06:50.144269 17546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1f46bc9df3a44dd927d40f3778a5ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.145300 17681 tablet_bootstrap.cc:492] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:50.147781 17547 tablet_service.cc:1511] Processing CreateTablet for tablet 445a4c9a8b0441f0ad2725785483a5d4 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:50.150868 17618 tablet_service.cc:1511] Processing CreateTablet for tablet 445a4c9a8b0441f0ad2725785483a5d4 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:50.152587 17618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 445a4c9a8b0441f0ad2725785483a5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.153163 17395 tablet_service.cc:1511] Processing CreateTablet for tablet 445a4c9a8b0441f0ad2725785483a5d4 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:50.154692 17395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 445a4c9a8b0441f0ad2725785483a5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.149333 17547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 445a4c9a8b0441f0ad2725785483a5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.126573 17677 raft_consensus.cc:385] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.165095 17677 raft_consensus.cc:740] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.169477 17677 consensus_queue.cc:260] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:50.161422 17699 raft_consensus.cc:515] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:50.184204 17545 tablet_service.cc:1511] Processing CreateTablet for tablet deeb668bf4814b1bb7e01791d6f96914 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:50.185966 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800f4d4505654dc58f390fcc9f0d90db" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:50.187064 17699 leader_election.cc:290] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:50.188135 17473 tablet_service.cc:1511] Processing CreateTablet for tablet deeb668bf4814b1bb7e01791d6f96914 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:50.188774 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800f4d4505654dc58f390fcc9f0d90db" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:50.189569 17559 raft_consensus.cc:3060] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.189785 17473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deeb668bf4814b1bb7e01791d6f96914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.186038 17545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deeb668bf4814b1bb7e01791d6f96914. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:50.197835 17450 leader_election.cc:343] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.203801 17619 tablet_service.cc:1511] Processing CreateTablet for tablet deeb668bf4814b1bb7e01791d6f96914 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:50.205433 17619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deeb668bf4814b1bb7e01791d6f96914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.217003 17678 ts_tablet_manager.cc:1434] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.167s	user 0.011s	sys 0.007s
I20260709 13:06:50.217150 17692 raft_consensus.cc:493] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.218541 17474 tablet_service.cc:1511] Processing CreateTablet for tablet a74eee34cc514b79b981a6ef6c45d7e2 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:50.219461 17692 raft_consensus.cc:515] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:50.220489 17474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a74eee34cc514b79b981a6ef6c45d7e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.224257 17559 raft_consensus.cc:2468] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:50.236419 17399 tablet_service.cc:1511] Processing CreateTablet for tablet a74eee34cc514b79b981a6ef6c45d7e2 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:50.239732 17692 leader_election.cc:290] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:50.245740 17682 tablet_bootstrap.cc:654] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.218520 17678 tablet_bootstrap.cc:492] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:50.250973 17677 ts_tablet_manager.cc:1434] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.129s	user 0.006s	sys 0.005s
I20260709 13:06:50.258071 17704 raft_consensus.cc:493] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.269992 17704 raft_consensus.cc:515] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:50.272785 17704 leader_election.cc:290] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:50.277824 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5414b9916b148bba0932c91f3770f57" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:06:50.285351 17355 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:50.288664 17678 tablet_bootstrap.cc:654] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.311515 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5414b9916b148bba0932c91f3770f57" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:50.315377 17451 leader_election.cc:304] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:50.316500 17699 raft_consensus.cc:2804] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:06:50.317167 17599 leader_election.cc:343] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.317250 17699 raft_consensus.cc:697] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:50.319114 17699 consensus_queue.cc:237] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
W20260709 13:06:50.321272 17598 leader_election.cc:343] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.321386 17546 tablet_service.cc:1511] Processing CreateTablet for tablet a74eee34cc514b79b981a6ef6c45d7e2 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:50.323173 17685 raft_consensus.cc:493] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.323968 17685 raft_consensus.cc:515] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:50.322222 17598 leader_election.cc:304] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:50.323182 17546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a74eee34cc514b79b981a6ef6c45d7e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.328568 17704 raft_consensus.cc:2749] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.331815 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fb377eea3d41e68cb4f46212e6e857" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:50.334017 17685 leader_election.cc:290] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:50.346204 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7e0518788149c19cf493a65bc598c2" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:50.347164 17558 raft_consensus.cc:2468] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:50.295275 17683 raft_consensus.cc:515] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:50.349069 17633 raft_consensus.cc:2393] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 1: Already voted for candidate a9a65d4c22be403682be9bc817650ada in this term.
I20260709 13:06:50.337361 17677 tablet_bootstrap.cc:492] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:50.352181 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fb377eea3d41e68cb4f46212e6e857" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:50.352766 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fb377eea3d41e68cb4f46212e6e857" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:50.353487 17410 raft_consensus.cc:3060] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.352931 17485 raft_consensus.cc:3060] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.349069 17376 leader_election.cc:304] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:06:50.357434 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fb377eea3d41e68cb4f46212e6e857" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:50.359719 17692 raft_consensus.cc:2804] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.350415 17683 leader_election.cc:290] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:50.360256 17692 raft_consensus.cc:493] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.360728 17692 raft_consensus.cc:3060] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.361572 17621 tablet_service.cc:1511] Processing CreateTablet for tablet 62f330683b8d4b0696fad129a8f2ce23 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:50.363386 17621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62f330683b8d4b0696fad129a8f2ce23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.363901 17450 leader_election.cc:304] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:50.366442 17394 tablet_service.cc:1511] Processing CreateTablet for tablet 62f330683b8d4b0696fad129a8f2ce23 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:50.370641 17410 raft_consensus.cc:2468] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 1.
I20260709 13:06:50.373926 17598 leader_election.cc:304] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:50.374506 17470 tablet_service.cc:1511] Processing CreateTablet for tablet 62f330683b8d4b0696fad129a8f2ce23 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:50.375288 17683 raft_consensus.cc:2804] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.376196 17470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62f330683b8d4b0696fad129a8f2ce23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.376133 17683 raft_consensus.cc:697] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [term 1 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:06:50.377336 17683 consensus_queue.cc:237] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:50.377871 17681 tablet_bootstrap.cc:654] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.392514 17677 tablet_bootstrap.cc:654] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.393935 17544 tablet_service.cc:1511] Processing CreateTablet for tablet 5b1eabbf0fc942fabea0c59e6997ad23 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:50.395859 17544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b1eabbf0fc942fabea0c59e6997ad23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.398029 17399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a74eee34cc514b79b981a6ef6c45d7e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.409054 17394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62f330683b8d4b0696fad129a8f2ce23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.411937 17622 tablet_service.cc:1511] Processing CreateTablet for tablet 5b1eabbf0fc942fabea0c59e6997ad23 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:50.413535 17682 tablet_bootstrap.cc:492] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:50.415563 17682 ts_tablet_manager.cc:1403] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.299s	user 0.029s	sys 0.031s
I20260709 13:06:50.416083 17622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b1eabbf0fc942fabea0c59e6997ad23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.421285 17308 catalog_manager.cc:5697] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.414666 17678 tablet_bootstrap.cc:492] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:50.433244 17678 ts_tablet_manager.cc:1403] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.215s	user 0.025s	sys 0.004s
I20260709 13:06:50.432029 17682 raft_consensus.cc:359] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:50.435864 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7e0518788149c19cf493a65bc598c2" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:50.434883 17692 raft_consensus.cc:515] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
W20260709 13:06:50.454396 17373 leader_election.cc:343] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.455999 17692 leader_election.cc:290] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:50.444659 17393 tablet_service.cc:1511] Processing CreateTablet for tablet 5b1eabbf0fc942fabea0c59e6997ad23 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:50.445350 17474 tablet_service.cc:1511] Processing CreateTablet for tablet 626dcd42c0f74fccb9c2a42ee7d443ee (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:50.444665 17543 tablet_service.cc:1511] Processing CreateTablet for tablet 626dcd42c0f74fccb9c2a42ee7d443ee (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:50.451898 17618 tablet_service.cc:1511] Processing CreateTablet for tablet 626dcd42c0f74fccb9c2a42ee7d443ee (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:50.421407 17485 raft_consensus.cc:2468] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 1.
I20260709 13:06:50.460114 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7e0518788149c19cf493a65bc598c2" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:50.456960 17682 raft_consensus.cc:385] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.461167 17395 tablet_service.cc:1511] Processing CreateTablet for tablet 14ead71306714d3a9784b47ddaa7588c (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:06:50.462107 17393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b1eabbf0fc942fabea0c59e6997ad23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.463073 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7e0518788149c19cf493a65bc598c2" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:50.463073 17678 raft_consensus.cc:359] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:50.465633 17472 tablet_service.cc:1511] Processing CreateTablet for tablet 14ead71306714d3a9784b47ddaa7588c (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:06:50.467039 17549 tablet_service.cc:1511] Processing CreateTablet for tablet 14ead71306714d3a9784b47ddaa7588c (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:06:50.471909 17620 tablet_service.cc:1511] Processing CreateTablet for tablet 2226d75e22544facaa146d281f852f27 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:50.496677 17543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626dcd42c0f74fccb9c2a42ee7d443ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.488170 17685 raft_consensus.cc:2749] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.490473 17618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626dcd42c0f74fccb9c2a42ee7d443ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.504537 17474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626dcd42c0f74fccb9c2a42ee7d443ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.506300 17396 tablet_service.cc:1511] Processing CreateTablet for tablet 2226d75e22544facaa146d281f852f27 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:50.497699 17678 raft_consensus.cc:385] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.494144 17558 raft_consensus.cc:3060] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.491746 17682 raft_consensus.cc:740] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.520155 17472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14ead71306714d3a9784b47ddaa7588c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.522886 17549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14ead71306714d3a9784b47ddaa7588c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.523690 17620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2226d75e22544facaa146d281f852f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.531692 17396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2226d75e22544facaa146d281f852f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.532401 17682 consensus_queue.cc:260] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:50.518548 17678 raft_consensus.cc:740] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.539531 17395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14ead71306714d3a9784b47ddaa7588c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:50.539927 17373 leader_election.cc:343] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.540818 17682 ts_tablet_manager.cc:1434] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.125s	user 0.008s	sys 0.003s
I20260709 13:06:50.542189 17682 tablet_bootstrap.cc:492] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:50.539929 17678 consensus_queue.cc:260] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:50.544297 17681 tablet_bootstrap.cc:492] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:50.544970 17681 ts_tablet_manager.cc:1403] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.400s	user 0.019s	sys 0.003s
I20260709 13:06:50.547173 17678 ts_tablet_manager.cc:1434] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.113s	user 0.008s	sys 0.003s
I20260709 13:06:50.567428 17623 tablet_service.cc:1511] Processing CreateTablet for tablet e84a923caa9c4755bcb78a35143dfbd9 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:50.569159 17623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e84a923caa9c4755bcb78a35143dfbd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.572324 17678 tablet_bootstrap.cc:492] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:50.576143 17470 tablet_service.cc:1511] Processing CreateTablet for tablet 2226d75e22544facaa146d281f852f27 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:50.577829 17470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2226d75e22544facaa146d281f852f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.577764 17548 tablet_service.cc:1511] Processing CreateTablet for tablet e84a923caa9c4755bcb78a35143dfbd9 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:50.583824 17681 raft_consensus.cc:359] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:50.584816 17681 raft_consensus.cc:385] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.585135 17681 raft_consensus.cc:740] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.585551 17548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e84a923caa9c4755bcb78a35143dfbd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.586084 17681 consensus_queue.cc:260] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:50.584091 17677 tablet_bootstrap.cc:492] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:50.584128 17558 raft_consensus.cc:2468] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:50.591861 17376 leader_election.cc:304] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:50.594332 17692 raft_consensus.cc:2804] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.601478 17473 tablet_service.cc:1511] Processing CreateTablet for tablet 2e4b5ba400ad420f89c16a97b71e3b70 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:50.603281 17473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4b5ba400ad420f89c16a97b71e3b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.605938 17393 tablet_service.cc:1511] Processing CreateTablet for tablet e84a923caa9c4755bcb78a35143dfbd9 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:50.590615 17677 ts_tablet_manager.cc:1403] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.254s	user 0.021s	sys 0.023s
I20260709 13:06:50.608901 17681 ts_tablet_manager.cc:1434] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.004s
I20260709 13:06:50.610213 17681 tablet_bootstrap.cc:492] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:50.610673 17393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e84a923caa9c4755bcb78a35143dfbd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.610365 17547 tablet_service.cc:1511] Processing CreateTablet for tablet 2e4b5ba400ad420f89c16a97b71e3b70 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:50.611979 17547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4b5ba400ad420f89c16a97b71e3b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.627161 17307 catalog_manager.cc:5697] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 0 to 1, leader changed from <none> to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 1 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.633572 17682 tablet_bootstrap.cc:654] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.635005 17399 tablet_service.cc:1511] Processing CreateTablet for tablet 6170cc45e2ea48598650c74f9c06b644 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:50.636715 17399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6170cc45e2ea48598650c74f9c06b644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.640395 17621 tablet_service.cc:1511] Processing CreateTablet for tablet 2e4b5ba400ad420f89c16a97b71e3b70 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:50.645748 17621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4b5ba400ad420f89c16a97b71e3b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.651608 17677 raft_consensus.cc:359] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:50.660730 17678 tablet_bootstrap.cc:654] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.655982 17677 raft_consensus.cc:385] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.691280 17677 raft_consensus.cc:740] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.692091 17692 raft_consensus.cc:697] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:06:50.693238 17692 consensus_queue.cc:237] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:50.695317 17677 consensus_queue.cc:260] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:50.700008 17681 tablet_bootstrap.cc:654] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.703588 17678 tablet_bootstrap.cc:492] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:50.704340 17678 ts_tablet_manager.cc:1403] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.132s	user 0.012s	sys 0.005s
I20260709 13:06:50.705655 17548 tablet_service.cc:1511] Processing CreateTablet for tablet 6170cc45e2ea48598650c74f9c06b644 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:50.707963 17548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6170cc45e2ea48598650c74f9c06b644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.709254 17678 raft_consensus.cc:359] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:50.710511 17678 raft_consensus.cc:385] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.711325 17678 raft_consensus.cc:740] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.712651 17678 consensus_queue.cc:260] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:50.745126 17677 ts_tablet_manager.cc:1434] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.136s	user 0.009s	sys 0.000s
I20260709 13:06:50.746690 17677 tablet_bootstrap.cc:492] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
W20260709 13:06:50.747143 17450 consensus_peers.cc:597] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:50.758244 17399 tablet_service.cc:1511] Processing CreateTablet for tablet 1e00eadc811946b48dfdf947398a84c1 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:50.760111 17399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e00eadc811946b48dfdf947398a84c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.751400 17618 tablet_service.cc:1511] Processing CreateTablet for tablet 1e00eadc811946b48dfdf947398a84c1 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:50.761868 17618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e00eadc811946b48dfdf947398a84c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.762632 17473 tablet_service.cc:1511] Processing CreateTablet for tablet 1e00eadc811946b48dfdf947398a84c1 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:50.764705 17473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e00eadc811946b48dfdf947398a84c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.767831 17678 ts_tablet_manager.cc:1434] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.005s
I20260709 13:06:50.769688 17678 tablet_bootstrap.cc:492] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:50.751400 17474 tablet_service.cc:1511] Processing CreateTablet for tablet 6170cc45e2ea48598650c74f9c06b644 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:50.778550 17547 tablet_service.cc:1511] Processing CreateTablet for tablet 032dc48c2ae44160841c456abd340e40 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:06:50.779834 17621 tablet_service.cc:1511] Processing CreateTablet for tablet 032dc48c2ae44160841c456abd340e40 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:06:50.780951 17715 raft_consensus.cc:493] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.781898 17621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 032dc48c2ae44160841c456abd340e40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.781590 17715 raft_consensus.cc:515] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:50.780293 17547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 032dc48c2ae44160841c456abd340e40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.785058 17682 tablet_bootstrap.cc:492] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:50.785734 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8bf7f6c34a4edfb28a4914ad4ce49e" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:50.786574 17558 raft_consensus.cc:2468] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:50.788089 17376 leader_election.cc:304] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:06:50.785743 17682 ts_tablet_manager.cc:1403] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.244s	user 0.028s	sys 0.012s
I20260709 13:06:50.786126 17393 tablet_service.cc:1511] Processing CreateTablet for tablet 032dc48c2ae44160841c456abd340e40 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:06:50.803140 17623 tablet_service.cc:1511] Processing CreateTablet for tablet b85fb92208674e2ba20983a453290a8b (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:06:50.804451 17393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 032dc48c2ae44160841c456abd340e40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.806838 17681 tablet_bootstrap.cc:492] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:50.807555 17681 ts_tablet_manager.cc:1403] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.198s	user 0.012s	sys 0.021s
I20260709 13:06:50.809365 17395 tablet_service.cc:1511] Processing CreateTablet for tablet fa3738962c3242b5aa69f93b778bea53 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:50.812454 17470 tablet_service.cc:1511] Processing CreateTablet for tablet b85fb92208674e2ba20983a453290a8b (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:06:50.816560 17549 tablet_service.cc:1511] Processing CreateTablet for tablet b85fb92208674e2ba20983a453290a8b (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:06:50.804785 17623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85fb92208674e2ba20983a453290a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.819864 17677 tablet_bootstrap.cc:654] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.793591 17474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6170cc45e2ea48598650c74f9c06b644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.830039 17678 tablet_bootstrap.cc:654] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.828409 17715 leader_election.cc:290] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:50.832499 17470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85fb92208674e2ba20983a453290a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.834960 17549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85fb92208674e2ba20983a453290a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.838672 17395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa3738962c3242b5aa69f93b778bea53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.846002 17681 raft_consensus.cc:359] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:50.846979 17681 raft_consensus.cc:385] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.847301 17681 raft_consensus.cc:740] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.847664 17472 tablet_service.cc:1511] Processing CreateTablet for tablet fa3738962c3242b5aa69f93b778bea53 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:50.850128 17472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa3738962c3242b5aa69f93b778bea53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.840453 17692 raft_consensus.cc:2804] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.857462 17692 raft_consensus.cc:493] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:50.841552 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 53607.5us
I20260709 13:06:50.858544 17692 raft_consensus.cc:3060] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.860630 17548 tablet_service.cc:1511] Processing CreateTablet for tablet fa3738962c3242b5aa69f93b778bea53 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:50.862249 17548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa3738962c3242b5aa69f93b778bea53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.868389 17682 raft_consensus.cc:359] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:50.869776 17682 raft_consensus.cc:385] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.870162 17682 raft_consensus.cc:740] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.869197 17304 catalog_manager.cc:5697] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.871963 17620 tablet_service.cc:1511] Processing CreateTablet for tablet 8b8583086b304aa484d68b0a8175ceb4 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:50.879770 17620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b8583086b304aa484d68b0a8175ceb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.879356 17682 consensus_queue.cc:260] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:50.882094 17682 ts_tablet_manager.cc:1434] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.001s
I20260709 13:06:50.874993 17681 consensus_queue.cc:260] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:50.894274 17681 ts_tablet_manager.cc:1434] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.086s	user 0.007s	sys 0.002s
I20260709 13:06:50.895200 17396 tablet_service.cc:1511] Processing CreateTablet for tablet 8b8583086b304aa484d68b0a8175ceb4 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:50.895774 17681 tablet_bootstrap.cc:492] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:50.896885 17396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b8583086b304aa484d68b0a8175ceb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.899389 17547 tablet_service.cc:1511] Processing CreateTablet for tablet 0b795b9f53f64ad387b207e8da82ff6f (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:50.901497 17547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b795b9f53f64ad387b207e8da82ff6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.920977 17715 raft_consensus.cc:493] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.921725 17715 raft_consensus.cc:515] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:50.907740 17475 tablet_service.cc:1511] Processing CreateTablet for tablet 8b8583086b304aa484d68b0a8175ceb4 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:50.923628 17394 tablet_service.cc:1511] Processing CreateTablet for tablet 0b795b9f53f64ad387b207e8da82ff6f (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:50.924234 17475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b8583086b304aa484d68b0a8175ceb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.925264 17394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b795b9f53f64ad387b207e8da82ff6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.926482 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800f4d4505654dc58f390fcc9f0d90db" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:50.927466 17558 raft_consensus.cc:2393] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
I20260709 13:06:50.929309 17715 leader_election.cc:290] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:50.930401 17692 raft_consensus.cc:515] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:50.933097 17691 raft_consensus.cc:493] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.933730 17691 raft_consensus.cc:515] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:50.938064 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800f4d4505654dc58f390fcc9f0d90db" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:50.939837 17683 consensus_queue.cc:1048] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.941790 17618 tablet_service.cc:1511] Processing CreateTablet for tablet 0b795b9f53f64ad387b207e8da82ff6f (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:50.945468 17691 leader_election.cc:290] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:50.947119 17677 tablet_bootstrap.cc:492] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:50.934841 17682 tablet_bootstrap.cc:492] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:50.959594 17677 ts_tablet_manager.cc:1403] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.213s	user 0.028s	sys 0.003s
I20260709 13:06:50.959733 17618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b795b9f53f64ad387b207e8da82ff6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.947237 17685 raft_consensus.cc:493] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.962944 17685 raft_consensus.cc:515] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:50.965755 17685 leader_election.cc:290] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:50.948276 17373 leader_election.cc:304] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:50.955650 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9a47a9cc56480fada4e8084a44682a" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:50.969588 17400 tablet_service.cc:1511] Processing CreateTablet for tablet 050a17ead1c04486b472a25b0bb179ca (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:50.975824 17678 tablet_bootstrap.cc:492] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:50.976634 17678 ts_tablet_manager.cc:1403] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.207s	user 0.019s	sys 0.011s
I20260709 13:06:50.981334 17678 raft_consensus.cc:359] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:50.982570 17678 raft_consensus.cc:385] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.983075 17678 raft_consensus.cc:740] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.984216 17678 consensus_queue.cc:260] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:50.997872 17400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 050a17ead1c04486b472a25b0bb179ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:50.971719 17473 tablet_service.cc:1511] Processing CreateTablet for tablet a81d2393d5664464837df0b9d2a0d25b (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:51.002068 17546 tablet_service.cc:1511] Processing CreateTablet for tablet a81d2393d5664464837df0b9d2a0d25b (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:51.003120 17473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a81d2393d5664464837df0b9d2a0d25b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.003695 17546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a81d2393d5664464837df0b9d2a0d25b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.004119 17682 tablet_bootstrap.cc:654] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.989017 17715 raft_consensus.cc:3060] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.996344 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7e0518788149c19cf493a65bc598c2" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
W20260709 13:06:51.000458 17522 leader_election.cc:343] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.023607 17683 raft_consensus.cc:493] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.032337 17397 tablet_service.cc:1511] Processing CreateTablet for tablet 22993c01d4634a3a92c5a2903aa07ffe (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:51.032672 17683 raft_consensus.cc:515] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:51.034016 17397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22993c01d4634a3a92c5a2903aa07ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.035873 17398 tablet_service.cc:1511] Processing CreateTablet for tablet 34919e11221f465cbcef1151a1c5d4b0 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:51.037461 17398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34919e11221f465cbcef1151a1c5d4b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.043347 17677 raft_consensus.cc:359] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:51.044312 17677 raft_consensus.cc:385] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.044647 17677 raft_consensus.cc:740] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.045470 17677 consensus_queue.cc:260] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:51.051107 17471 tablet_service.cc:1511] Processing CreateTablet for tablet 22993c01d4634a3a92c5a2903aa07ffe (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:51.052004 17678 ts_tablet_manager.cc:1434] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.075s	user 0.008s	sys 0.004s
I20260709 13:06:51.052827 17471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22993c01d4634a3a92c5a2903aa07ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.053238 17678 tablet_bootstrap.cc:492] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:51.023370 17692 leader_election.cc:290] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:51.063464 17623 tablet_service.cc:1511] Processing CreateTablet for tablet a81d2393d5664464837df0b9d2a0d25b (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:51.063742 17622 tablet_service.cc:1511] Processing CreateTablet for tablet 34919e11221f465cbcef1151a1c5d4b0 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:51.062342 17621 tablet_service.cc:1511] Processing CreateTablet for tablet 22993c01d4634a3a92c5a2903aa07ffe (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:51.065181 17622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34919e11221f465cbcef1151a1c5d4b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.019905 17470 tablet_service.cc:1511] Processing CreateTablet for tablet 050a17ead1c04486b472a25b0bb179ca (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:51.069711 17469 tablet_service.cc:1511] Processing CreateTablet for tablet dcb6246353144b0a8b9648ac089a1da6 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:06:51.071128 17677 ts_tablet_manager.cc:1434] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.111s	user 0.009s	sys 0.001s
I20260709 13:06:51.072057 17470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 050a17ead1c04486b472a25b0bb179ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.072450 17677 tablet_bootstrap.cc:492] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:51.075519 17623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a81d2393d5664464837df0b9d2a0d25b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.075783 17726 consensus_queue.cc:1048] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.078195 17681 tablet_bootstrap.cc:654] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.079524 17469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcb6246353144b0a8b9648ac089a1da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.079885 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7e0518788149c19cf493a65bc598c2" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:51.079746 17545 tablet_service.cc:1511] Processing CreateTablet for tablet 050a17ead1c04486b472a25b0bb179ca (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:51.080824 17558 raft_consensus.cc:2393] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:06:51.081383 17545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 050a17ead1c04486b472a25b0bb179ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.085143 17543 tablet_service.cc:1511] Processing CreateTablet for tablet 34919e11221f465cbcef1151a1c5d4b0 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:51.087319 17621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22993c01d4634a3a92c5a2903aa07ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.091046 17543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34919e11221f465cbcef1151a1c5d4b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.092828 17683 leader_election.cc:290] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:51.094408 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8bf7f6c34a4edfb28a4914ad4ce49e" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:51.095427 17409 raft_consensus.cc:2393] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a9a65d4c22be403682be9bc817650ada in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:06:51.107978 17451 leader_election.cc:304] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:51.109385 17685 raft_consensus.cc:3060] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.109990 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8bf7f6c34a4edfb28a4914ad4ce49e" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:51.112737 17558 raft_consensus.cc:3060] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.114782 17683 raft_consensus.cc:493] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.115638 17683 raft_consensus.cc:515] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:51.119943 17683 leader_election.cc:290] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:51.119704 17730 consensus_queue.cc:1048] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.121579 17544 tablet_service.cc:1511] Processing CreateTablet for tablet dcb6246353144b0a8b9648ac089a1da6 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:06:51.119699 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ccac61a2bab40708cfe17c17ae1de65" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:51.123256 17544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcb6246353144b0a8b9648ac089a1da6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:51.124356 17598 leader_election.cc:343] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.124684 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8bf7f6c34a4edfb28a4914ad4ce49e" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:51.125576 17633 raft_consensus.cc:2468] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:51.124526 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8bf7f6c34a4edfb28a4914ad4ce49e" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:51.127846 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8bf7f6c34a4edfb28a4914ad4ce49e" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:51.128871 17633 raft_consensus.cc:3060] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.120743 17683 raft_consensus.cc:493] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.132181 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ccac61a2bab40708cfe17c17ae1de65" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:51.142143 17485 raft_consensus.cc:2468] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 0.
I20260709 13:06:51.151723 17715 raft_consensus.cc:2749] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.157627 17678 tablet_bootstrap.cc:654] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.161125 17472 tablet_service.cc:1511] Processing CreateTablet for tablet 20df005b7408432da9ac4fcf110a20b9 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:51.162978 17472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20df005b7408432da9ac4fcf110a20b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.163347 17685 raft_consensus.cc:2749] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.179880 17549 tablet_service.cc:1511] Processing CreateTablet for tablet 20df005b7408432da9ac4fcf110a20b9 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:51.181558 17549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20df005b7408432da9ac4fcf110a20b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.186259 17599 leader_election.cc:304] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:51.137374 17683 raft_consensus.cc:515] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:51.203003 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9a47a9cc56480fada4e8084a44682a" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:51.204000 17632 raft_consensus.cc:2468] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:06:51.171778 17715 consensus_queue.cc:1048] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:06:51.220686 17620 tablet_service.cc:1511] Processing CreateTablet for tablet dcb6246353144b0a8b9648ac089a1da6 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:06:51.161125 17399 tablet_service.cc:1511] Processing CreateTablet for tablet 20df005b7408432da9ac4fcf110a20b9 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:51.239661 17620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcb6246353144b0a8b9648ac089a1da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.257773 17682 tablet_bootstrap.cc:492] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:51.264171 17715 consensus_queue.cc:1048] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.258564 17682 ts_tablet_manager.cc:1403] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.345s	user 0.021s	sys 0.017s
I20260709 13:06:51.277107 17681 tablet_bootstrap.cc:492] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:51.277871 17681 ts_tablet_manager.cc:1403] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.382s	user 0.015s	sys 0.012s
I20260709 13:06:51.243729 17633 raft_consensus.cc:2468] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:51.234213 17596 leader_election.cc:304] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:51.288421 17399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20df005b7408432da9ac4fcf110a20b9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:51.268384 17599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 94240.5us
I20260709 13:06:51.286804 17734 raft_consensus.cc:2749] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.316954 17678 tablet_bootstrap.cc:492] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:51.317632 17678 ts_tablet_manager.cc:1403] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.265s	user 0.023s	sys 0.018s
I20260709 13:06:51.304363 17475 tablet_service.cc:1511] Processing CreateTablet for tablet 242531b3d6ad4ea4bdd979f393b59f29 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:51.318090 17375 leader_election.cc:304] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:51.319082 17475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 242531b3d6ad4ea4bdd979f393b59f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.319144 17715 raft_consensus.cc:2804] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.319818 17715 raft_consensus.cc:697] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:06:51.320753 17715 consensus_queue.cc:237] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:51.321641 17678 raft_consensus.cc:359] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:51.322544 17678 raft_consensus.cc:385] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:06:51.309017 17355 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:51.282930 17558 raft_consensus.cc:2468] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:51.310560 17681 raft_consensus.cc:359] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:51.311959 17677 tablet_bootstrap.cc:654] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.315832 17737 raft_consensus.cc:2804] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:06:51.315982 17596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 89710.8us
I20260709 13:06:51.282095 17682 raft_consensus.cc:359] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:51.342003 17737 raft_consensus.cc:493] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.342674 17737 raft_consensus.cc:3060] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.323642 17678 raft_consensus.cc:740] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.333917 17681 raft_consensus.cc:385] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.359834 17400 tablet_service.cc:1511] Processing CreateTablet for tablet 242531b3d6ad4ea4bdd979f393b59f29 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:51.360284 17621 tablet_service.cc:1511] Processing CreateTablet for tablet 242531b3d6ad4ea4bdd979f393b59f29 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:51.343442 17683 leader_election.cc:290] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:51.343637 17682 raft_consensus.cc:385] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.361481 17400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 242531b3d6ad4ea4bdd979f393b59f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.366019 17621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 242531b3d6ad4ea4bdd979f393b59f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.366475 17545 tablet_service.cc:1511] Processing CreateTablet for tablet 05345b561dfb497c8f1e081b05c1bb5d (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:06:51.367549 17618 tablet_service.cc:1511] Processing CreateTablet for tablet 05345b561dfb497c8f1e081b05c1bb5d (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:06:51.367884 17473 tablet_service.cc:1511] Processing CreateTablet for tablet cdcc480cc6534780aa5a81e4c97e8bef (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:06:51.368950 17618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05345b561dfb497c8f1e081b05c1bb5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.369372 17473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdcc480cc6534780aa5a81e4c97e8bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.369215 17394 tablet_service.cc:1511] Processing CreateTablet for tablet 05345b561dfb497c8f1e081b05c1bb5d (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:06:51.375227 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:51.376008 17546 tablet_service.cc:1511] Processing CreateTablet for tablet cdcc480cc6534780aa5a81e4c97e8bef (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:06:51.357511 17681 raft_consensus.cc:740] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.367981 17545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05345b561dfb497c8f1e081b05c1bb5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.383879 17394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05345b561dfb497c8f1e081b05c1bb5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.400012 17524 leader_election.cc:304] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:51.364799 17682 raft_consensus.cc:740] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.410213 17546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdcc480cc6534780aa5a81e4c97e8bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.415787 17398 tablet_service.cc:1511] Processing CreateTablet for tablet def346af33c747faaf6c9cf0414510f0 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:06:51.417881 17398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet def346af33c747faaf6c9cf0414510f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.419332 17682 consensus_queue.cc:260] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:51.422297 17682 ts_tablet_manager.cc:1434] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.151s	user 0.007s	sys 0.000s
I20260709 13:06:51.424224 17685 raft_consensus.cc:493] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.458181 17685 raft_consensus.cc:515] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:51.461107 17685 leader_election.cc:290] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:51.430912 17681 consensus_queue.cc:260] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:06:51.395468 17599 leader_election.cc:343] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.387353 17678 consensus_queue.cc:260] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:51.439248 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:51.439289 17622 tablet_service.cc:1511] Processing CreateTablet for tablet cdcc480cc6534780aa5a81e4c97e8bef (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:06:51.455020 17737 raft_consensus.cc:515] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:51.456243 17691 raft_consensus.cc:2804] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:06:51.457114 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 57230.2us
W20260709 13:06:51.463086 17599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 74323.4us
I20260709 13:06:51.480273 17622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdcc480cc6534780aa5a81e4c97e8bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.483340 17475 tablet_service.cc:1511] Processing CreateTablet for tablet def346af33c747faaf6c9cf0414510f0 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:06:51.485541 17475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet def346af33c747faaf6c9cf0414510f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.466313 17691 raft_consensus.cc:493] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.478897 17682 tablet_bootstrap.cc:492] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:51.499600 17691 raft_consensus.cc:3060] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.502900 17692 raft_consensus.cc:493] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.522707 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9a47a9cc56480fada4e8084a44682a" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:06:51.523430 17596 leader_election.cc:343] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.524055 17632 raft_consensus.cc:2468] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:51.524945 17678 ts_tablet_manager.cc:1434] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.207s	user 0.010s	sys 0.001s
I20260709 13:06:51.524092 17596 leader_election.cc:304] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:51.527328 17450 leader_election.cc:304] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:51.528748 17745 raft_consensus.cc:2749] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.529799 17685 raft_consensus.cc:2804] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.530341 17685 raft_consensus.cc:493] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.530794 17685 raft_consensus.cc:3060] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.548686 17737 leader_election.cc:290] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:51.541091 17692 raft_consensus.cc:515] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:51.551857 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ccac61a2bab40708cfe17c17ae1de65" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:51.552711 17677 tablet_bootstrap.cc:492] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:51.552776 17484 raft_consensus.cc:3060] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.553552 17677 ts_tablet_manager.cc:1403] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.481s	user 0.017s	sys 0.034s
I20260709 13:06:51.550395 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ccac61a2bab40708cfe17c17ae1de65" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:51.554567 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b734b40edd440fad746ea61f858c1b" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:51.526376 17678 tablet_bootstrap.cc:492] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:51.561333 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9a47a9cc56480fada4e8084a44682a" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:06:51.562542 17598 leader_election.cc:343] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.565696 17549 tablet_service.cc:1511] Processing CreateTablet for tablet def346af33c747faaf6c9cf0414510f0 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:06:51.570168 17681 ts_tablet_manager.cc:1434] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.292s	user 0.010s	sys 0.048s
I20260709 13:06:51.570648 17549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet def346af33c747faaf6c9cf0414510f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.572935 17681 tablet_bootstrap.cc:492] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
W20260709 13:06:51.576632 17373 leader_election.cc:343] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.577359 17620 tablet_service.cc:1511] Processing CreateTablet for tablet 0b2d2a20fb474604a1077ce9af500e42 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:06:51.550473 17685 raft_consensus.cc:515] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:51.585263 17620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b2d2a20fb474604a1077ce9af500e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.587062 17399 tablet_service.cc:1511] Processing CreateTablet for tablet 0b2d2a20fb474604a1077ce9af500e42 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:06:51.588819 17399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b2d2a20fb474604a1077ce9af500e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.579766 17737 raft_consensus.cc:493] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.592361 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b734b40edd440fad746ea61f858c1b" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:51.592669 17737 raft_consensus.cc:515] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:51.595589 17737 leader_election.cc:290] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:51.596947 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c85566c0a845438507d71c6936d507" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:51.598723 17692 leader_election.cc:290] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
W20260709 13:06:51.599097 17596 leader_election.cc:343] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.600683 17677 raft_consensus.cc:359] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:51.601617 17677 raft_consensus.cc:385] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.601938 17677 raft_consensus.cc:740] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.604658 17484 raft_consensus.cc:2468] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 1.
I20260709 13:06:51.606810 17596 leader_election.cc:304] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:51.609846 17737 raft_consensus.cc:2804] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.616091 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c85566c0a845438507d71c6936d507" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
W20260709 13:06:51.616454 17376 leader_election.cc:343] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.617077 17376 leader_election.cc:304] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
W20260709 13:06:51.618644 17598 leader_election.cc:343] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.619268 17598 leader_election.cc:304] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:51.622138 17745 raft_consensus.cc:2749] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.623009 17715 raft_consensus.cc:2749] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.632416 17307 catalog_manager.cc:5697] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:51.613682 17737 raft_consensus.cc:697] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [term 1 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:06:51.649849 17691 raft_consensus.cc:515] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:51.655213 17559 raft_consensus.cc:2393] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 1: Already voted for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in this term.
I20260709 13:06:51.658164 17682 tablet_bootstrap.cc:654] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.659826 17691 leader_election.cc:290] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:51.661716 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9a47a9cc56480fada4e8084a44682a" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:51.668962 17484 raft_consensus.cc:2393] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
I20260709 13:06:51.671221 17475 tablet_service.cc:1511] Processing CreateTablet for tablet 0b2d2a20fb474604a1077ce9af500e42 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:06:51.672782 17475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b2d2a20fb474604a1077ce9af500e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.675153 17678 tablet_bootstrap.cc:654] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.682538 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9a47a9cc56480fada4e8084a44682a" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:51.683154 17685 leader_election.cc:290] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:51.683465 17632 raft_consensus.cc:3060] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.684929 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9a47a9cc56480fada4e8084a44682a" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:51.685775 17549 tablet_service.cc:1511] Processing CreateTablet for tablet 307acc498b0c4da7b7953e895b049c2a (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:51.705029 17549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 307acc498b0c4da7b7953e895b049c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.707386 17546 tablet_service.cc:1511] Processing CreateTablet for tablet c295096a121e4940b38125740cea67cb (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:06:51.709036 17546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c295096a121e4940b38125740cea67cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.686893 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9a47a9cc56480fada4e8084a44682a" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:51.711886 17394 tablet_service.cc:1511] Processing CreateTablet for tablet 1b10c75b663e4cbcb3237b59443f77ca (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:51.712651 17559 raft_consensus.cc:2393] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 1: Already voted for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in this term.
I20260709 13:06:51.713568 17394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b10c75b663e4cbcb3237b59443f77ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.690347 17620 tablet_service.cc:1511] Processing CreateTablet for tablet 307acc498b0c4da7b7953e895b049c2a (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:51.691028 17737 consensus_queue.cc:237] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:51.696710 17400 tablet_service.cc:1511] Processing CreateTablet for tablet 307acc498b0c4da7b7953e895b049c2a (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:51.715238 17620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 307acc498b0c4da7b7953e895b049c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.715333 17400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 307acc498b0c4da7b7953e895b049c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.716701 17632 raft_consensus.cc:2468] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:51.697254 17678 tablet_bootstrap.cc:492] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:51.697273 17473 tablet_service.cc:1511] Processing CreateTablet for tablet c295096a121e4940b38125740cea67cb (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:06:51.646363 17677 consensus_queue.cc:260] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:51.720047 17524 leader_election.cc:304] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:51.720638 17473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c295096a121e4940b38125740cea67cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.701964 17681 tablet_bootstrap.cc:654] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.723624 17691 raft_consensus.cc:2804] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.723846 17450 leader_election.cc:304] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:51.724308 17691 raft_consensus.cc:697] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:06:51.724871 17685 raft_consensus.cc:2749] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.722836 17545 tablet_service.cc:1511] Processing CreateTablet for tablet 1b10c75b663e4cbcb3237b59443f77ca (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:51.728344 17545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b10c75b663e4cbcb3237b59443f77ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.729020 17475 tablet_service.cc:1511] Processing CreateTablet for tablet 1b10c75b663e4cbcb3237b59443f77ca (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:51.725443 17691 consensus_queue.cc:237] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:51.729796 17678 ts_tablet_manager.cc:1403] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.193s	user 0.022s	sys 0.006s
I20260709 13:06:51.733927 17475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b10c75b663e4cbcb3237b59443f77ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.763492 17692 consensus_queue.cc:1048] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.736527 17678 raft_consensus.cc:359] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:51.737875 17677 ts_tablet_manager.cc:1434] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.184s	user 0.005s	sys 0.008s
I20260709 13:06:51.738464 17622 tablet_service.cc:1511] Processing CreateTablet for tablet c295096a121e4940b38125740cea67cb (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:06:51.784073 17622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c295096a121e4940b38125740cea67cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.784093 17677 tablet_bootstrap.cc:492] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:51.787843 17307 catalog_manager.cc:5697] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:51.796593 17678 raft_consensus.cc:385] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.797335 17678 raft_consensus.cc:740] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.798834 17682 tablet_bootstrap.cc:492] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:51.799513 17682 ts_tablet_manager.cc:1403] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.341s	user 0.013s	sys 0.017s
I20260709 13:06:51.803290 17677 tablet_bootstrap.cc:654] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.811669 17682 raft_consensus.cc:359] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:51.812685 17682 raft_consensus.cc:385] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.813017 17682 raft_consensus.cc:740] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.814023 17691 raft_consensus.cc:493] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.814980 17682 consensus_queue.cc:260] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:51.814837 17691 raft_consensus.cc:515] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:51.798193 17678 consensus_queue.cc:260] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:51.819751 17691 leader_election.cc:290] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:51.820794 17678 ts_tablet_manager.cc:1434] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.000s
I20260709 13:06:51.823447 17682 ts_tablet_manager.cc:1434] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.002s
I20260709 13:06:51.828276 17692 consensus_queue.cc:1048] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.830533 17618 tablet_service.cc:1511] Processing CreateTablet for tablet cac16fb87c954fd9b4cef5850ffc16e9 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:06:51.832695 17399 tablet_service.cc:1511] Processing CreateTablet for tablet cac16fb87c954fd9b4cef5850ffc16e9 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:06:51.840112 17546 tablet_service.cc:1511] Processing CreateTablet for tablet eea055e03bf849bea8fdc851e10a8cbe (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:51.841753 17546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eea055e03bf849bea8fdc851e10a8cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.841902 17472 tablet_service.cc:1511] Processing CreateTablet for tablet cac16fb87c954fd9b4cef5850ffc16e9 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:06:51.844997 17618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cac16fb87c954fd9b4cef5850ffc16e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.845475 17472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cac16fb87c954fd9b4cef5850ffc16e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.846554 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5414b9916b148bba0932c91f3770f57" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:51.847463 17410 raft_consensus.cc:2468] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:06:51.849329 17677 tablet_bootstrap.cc:492] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:51.849972 17677 ts_tablet_manager.cc:1403] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.066s	user 0.006s	sys 0.012s
I20260709 13:06:51.852558 17682 tablet_bootstrap.cc:492] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:51.852435 17524 leader_election.cc:304] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:06:51.854260 17691 raft_consensus.cc:2804] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.854768 17691 raft_consensus.cc:493] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.855147 17691 raft_consensus.cc:3060] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.856598 17399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cac16fb87c954fd9b4cef5850ffc16e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.864916 17691 raft_consensus.cc:515] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:51.867280 17678 tablet_bootstrap.cc:492] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:51.837810 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5414b9916b148bba0932c91f3770f57" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:51.931847 17632 raft_consensus.cc:2468] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:06:51.944757 17677 raft_consensus.cc:359] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:51.962158 17677 raft_consensus.cc:385] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.962654 17677 raft_consensus.cc:740] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.962296 17620 tablet_service.cc:1511] Processing CreateTablet for tablet eea055e03bf849bea8fdc851e10a8cbe (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:51.963652 17677 consensus_queue.cc:260] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:51.964854 17691 leader_election.cc:290] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:51.964951 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5414b9916b148bba0932c91f3770f57" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:51.965898 17632 raft_consensus.cc:3060] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.966284 17394 tablet_service.cc:1511] Processing CreateTablet for tablet eea055e03bf849bea8fdc851e10a8cbe (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:51.968022 17394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eea055e03bf849bea8fdc851e10a8cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.978677 17750 raft_consensus.cc:493] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.979398 17750 raft_consensus.cc:515] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:51.982987 17620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eea055e03bf849bea8fdc851e10a8cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.985538 17750 leader_election.cc:290] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:51.986644 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:51.987500 17633 raft_consensus.cc:2468] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:51.988999 17450 leader_election.cc:304] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:51.990016 17750 raft_consensus.cc:2804] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.990439 17750 raft_consensus.cc:493] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.992070 17677 ts_tablet_manager.cc:1434] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.142s	user 0.001s	sys 0.017s
I20260709 13:06:51.992930 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:51.990821 17750 raft_consensus.cc:3060] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.996078 17677 tablet_bootstrap.cc:492] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
W20260709 13:06:51.996268 17451 leader_election.cc:343] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.997646 17308 catalog_manager.cc:5697] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 0 to 1, leader changed from <none> to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 1 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.005263 17546 tablet_service.cc:1511] Processing CreateTablet for tablet dfc142508c684e6eb9716cb26e7047b8 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:52.005910 17682 tablet_bootstrap.cc:654] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.006848 17546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfc142508c684e6eb9716cb26e7047b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.013662 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5414b9916b148bba0932c91f3770f57" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:52.014531 17410 raft_consensus.cc:3060] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.993319 17473 tablet_service.cc:1511] Processing CreateTablet for tablet dfc142508c684e6eb9716cb26e7047b8 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:52.016220 17473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfc142508c684e6eb9716cb26e7047b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.016309 17681 tablet_bootstrap.cc:492] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:52.017704 17681 ts_tablet_manager.cc:1403] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.446s	user 0.032s	sys 0.035s
I20260709 13:06:52.014221 17475 tablet_service.cc:1511] Processing CreateTablet for tablet f55f8c5bd47d4a31a9562ab2b75ab56f (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:52.020232 17475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f55f8c5bd47d4a31a9562ab2b75ab56f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.021171 17678 tablet_bootstrap.cc:654] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.021389 17681 raft_consensus.cc:359] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.022167 17681 raft_consensus.cc:385] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.016002 17399 tablet_service.cc:1511] Processing CreateTablet for tablet f55f8c5bd47d4a31a9562ab2b75ab56f (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:52.031560 17399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f55f8c5bd47d4a31a9562ab2b75ab56f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.035065 17621 tablet_service.cc:1511] Processing CreateTablet for tablet 1995e080f6c048fe935afd95f336b109 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:52.036852 17621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1995e080f6c048fe935afd95f336b109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.038548 17677 tablet_bootstrap.cc:654] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.050005 17400 tablet_service.cc:1511] Processing CreateTablet for tablet 1995e080f6c048fe935afd95f336b109 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:52.051766 17400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1995e080f6c048fe935afd95f336b109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.051949 17471 tablet_service.cc:1511] Processing CreateTablet for tablet 1995e080f6c048fe935afd95f336b109 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:52.053476 17471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1995e080f6c048fe935afd95f336b109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.054374 17410 raft_consensus.cc:2468] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:52.055681 17545 tablet_service.cc:1511] Processing CreateTablet for tablet f55f8c5bd47d4a31a9562ab2b75ab56f (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:52.057183 17545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f55f8c5bd47d4a31a9562ab2b75ab56f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.057804 17524 leader_election.cc:304] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:06:52.063138 17691 raft_consensus.cc:2804] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.063683 17691 raft_consensus.cc:697] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:06:52.064194 17623 tablet_service.cc:1511] Processing CreateTablet for tablet abe7eea0c47a4c6998274f4b6d14dfb7 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:52.064566 17691 consensus_queue.cc:237] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.065584 17623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abe7eea0c47a4c6998274f4b6d14dfb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.071969 17549 tablet_service.cc:1511] Processing CreateTablet for tablet abe7eea0c47a4c6998274f4b6d14dfb7 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:52.073432 17549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abe7eea0c47a4c6998274f4b6d14dfb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.078100 17678 tablet_bootstrap.cc:492] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:52.078805 17678 ts_tablet_manager.cc:1403] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.212s	user 0.013s	sys 0.005s
I20260709 13:06:52.078580 17750 raft_consensus.cc:515] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.081046 17632 raft_consensus.cc:2468] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:52.083400 17622 tablet_service.cc:1511] Processing CreateTablet for tablet dfc142508c684e6eb9716cb26e7047b8 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:52.085079 17622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfc142508c684e6eb9716cb26e7047b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.086026 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:52.086745 17398 tablet_service.cc:1511] Processing CreateTablet for tablet abe7eea0c47a4c6998274f4b6d14dfb7 (DEFAULT_TABLE table=test-workload [id=7a746f935c424ee18385b4e312e8a00d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:52.082170 17678 raft_consensus.cc:359] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.111477 17678 raft_consensus.cc:385] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.111800 17678 raft_consensus.cc:740] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.096419 17750 leader_election.cc:290] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:52.096406 17677 tablet_bootstrap.cc:492] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:52.099143 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:52.112632 17678 consensus_queue.cc:260] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.113196 17632 raft_consensus.cc:3060] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.105436 17682 tablet_bootstrap.cc:492] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:52.110198 17685 raft_consensus.cc:493] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.116286 17677 ts_tablet_manager.cc:1403] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.120s	user 0.007s	sys 0.015s
I20260709 13:06:52.022490 17681 raft_consensus.cc:740] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
W20260709 13:06:52.112205 17451 leader_election.cc:343] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.115958 17685 raft_consensus.cc:515] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.118144 17681 consensus_queue.cc:260] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.122165 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c85566c0a845438507d71c6936d507" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:52.123013 17633 raft_consensus.cc:2468] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:52.111847 17398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abe7eea0c47a4c6998274f4b6d14dfb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.122431 17681 ts_tablet_manager.cc:1434] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.104s	user 0.007s	sys 0.003s
I20260709 13:06:52.127281 17681 tablet_bootstrap.cc:492] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:52.128158 17304 catalog_manager.cc:5697] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.135962 17450 leader_election.cc:304] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:52.137141 17750 raft_consensus.cc:2804] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.137570 17750 raft_consensus.cc:493] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.137902 17750 raft_consensus.cc:3060] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.144874 17685 leader_election.cc:290] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:52.147536 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c85566c0a845438507d71c6936d507" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
W20260709 13:06:52.150332 17450 leader_election.cc:343] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.150581 17677 raft_consensus.cc:359] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.117081 17682 ts_tablet_manager.cc:1403] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.265s	user 0.014s	sys 0.010s
I20260709 13:06:52.153715 17678 ts_tablet_manager.cc:1434] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.075s	user 0.010s	sys 0.000s
I20260709 13:06:52.166430 17678 tablet_bootstrap.cc:492] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:52.171406 17632 raft_consensus.cc:2468] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:52.172523 17681 tablet_bootstrap.cc:654] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.156260 17682 raft_consensus.cc:359] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.179443 17682 raft_consensus.cc:385] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.179842 17682 raft_consensus.cc:740] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.180912 17682 consensus_queue.cc:260] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.182595 17750 raft_consensus.cc:515] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.185050 17750 leader_election.cc:290] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:52.187325 17678 tablet_bootstrap.cc:654] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.192909 17450 leader_election.cc:304] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:52.193969 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c85566c0a845438507d71c6936d507" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:52.194478 17681 tablet_bootstrap.cc:492] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:52.194749 17632 raft_consensus.cc:3060] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.195060 17681 ts_tablet_manager.cc:1403] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.068s	user 0.010s	sys 0.013s
I20260709 13:06:52.151415 17677 raft_consensus.cc:385] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.196659 17682 ts_tablet_manager.cc:1434] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.001s
I20260709 13:06:52.196739 17677 raft_consensus.cc:740] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.197774 17677 consensus_queue.cc:260] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.198028 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c85566c0a845438507d71c6936d507" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:52.198885 17685 raft_consensus.cc:2804] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.199229 17409 raft_consensus.cc:3060] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.198770 17681 raft_consensus.cc:359] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.199527 17685 raft_consensus.cc:697] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:52.199837 17681 raft_consensus.cc:385] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.200233 17681 raft_consensus.cc:740] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.200553 17685 consensus_queue.cc:237] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.201077 17681 consensus_queue.cc:260] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.205183 17681 ts_tablet_manager.cc:1434] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:06:52.206422 17681 tablet_bootstrap.cc:492] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
W20260709 13:06:52.207579 17598 consensus_peers.cc:597] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:52.209481 17677 ts_tablet_manager.cc:1434] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.087s	user 0.006s	sys 0.000s
I20260709 13:06:52.210949 17677 tablet_bootstrap.cc:492] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:52.213881 17691 raft_consensus.cc:493] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.214576 17691 raft_consensus.cc:515] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.216550 17682 tablet_bootstrap.cc:492] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:52.217151 17691 leader_election.cc:290] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:52.218127 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d19239b38b1485ca53f239a71a11444" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:52.219588 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d19239b38b1485ca53f239a71a11444" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:06:52.219969 17522 leader_election.cc:343] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.218272 17737 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.220916 17737 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.223292 17737 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:52.224871 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:52.226212 17681 tablet_bootstrap.cc:654] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.226917 17737 raft_consensus.cc:493] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.227607 17737 raft_consensus.cc:515] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:06:52.228082 17598 leader_election.cc:343] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:52.224905 17524 leader_election.cc:343] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.229096 17524 leader_election.cc:304] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:52.230762 17691 raft_consensus.cc:2749] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.231663 17745 consensus_queue.cc:1048] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.232643 17677 tablet_bootstrap.cc:654] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.237761 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6633559a9d2643a29622caab848e5442" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:06:52.239686 17596 leader_election.cc:343] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.240089 17682 tablet_bootstrap.cc:654] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.241060 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:06:52.243049 17599 leader_election.cc:343] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.243587 17599 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:52.245110 17737 leader_election.cc:290] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:52.245788 17737 raft_consensus.cc:2749] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.247301 17677 tablet_bootstrap.cc:492] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:52.248099 17677 ts_tablet_manager.cc:1403] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.003s
I20260709 13:06:52.249270 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6633559a9d2643a29622caab848e5442" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
W20260709 13:06:52.251668 17598 leader_election.cc:343] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.252329 17598 leader_election.cc:304] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:52.253371 17737 raft_consensus.cc:2749] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.253491 17682 tablet_bootstrap.cc:492] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:52.254333 17682 ts_tablet_manager.cc:1403] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.009s
I20260709 13:06:52.257050 17409 raft_consensus.cc:2468] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:52.259218 17632 raft_consensus.cc:2468] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:52.252153 17677 raft_consensus.cc:359] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.263036 17677 raft_consensus.cc:385] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.258994 17682 raft_consensus.cc:359] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.263561 17682 raft_consensus.cc:385] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.263370 17450 leader_election.cc:304] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:52.263902 17682 raft_consensus.cc:740] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.264750 17682 consensus_queue.cc:260] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.265147 17750 raft_consensus.cc:2804] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.265818 17750 raft_consensus.cc:697] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:52.267134 17682 ts_tablet_manager.cc:1434] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:06:52.268987 17750 consensus_queue.cc:237] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.270059 17682 tablet_bootstrap.cc:492] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:52.277504 17682 tablet_bootstrap.cc:654] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.263389 17677 raft_consensus.cc:740] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.290155 17677 consensus_queue.cc:260] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.301545 17677 ts_tablet_manager.cc:1434] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.053s	user 0.011s	sys 0.000s
I20260709 13:06:52.315688 17677 tablet_bootstrap.cc:492] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:52.311314 17681 tablet_bootstrap.cc:492] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:52.318045 17681 ts_tablet_manager.cc:1403] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.112s	user 0.029s	sys 0.023s
I20260709 13:06:52.319847 17682 tablet_bootstrap.cc:492] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:52.320549 17682 ts_tablet_manager.cc:1403] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.003s
I20260709 13:06:52.323211 17677 tablet_bootstrap.cc:654] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.322741 17681 raft_consensus.cc:359] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.323727 17681 raft_consensus.cc:385] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.324177 17681 raft_consensus.cc:740] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.323554 17304 catalog_manager.cc:5697] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.325133 17681 consensus_queue.cc:260] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.326555 17678 tablet_bootstrap.cc:492] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:52.327253 17681 ts_tablet_manager.cc:1434] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:06:52.327347 17678 ts_tablet_manager.cc:1403] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.161s	user 0.017s	sys 0.003s
I20260709 13:06:52.328481 17681 tablet_bootstrap.cc:492] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:52.323725 17682 raft_consensus.cc:359] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.331104 17678 raft_consensus.cc:359] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.332027 17692 raft_consensus.cc:493] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.332597 17692 raft_consensus.cc:515] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.332063 17678 raft_consensus.cc:385] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.334079 17678 raft_consensus.cc:740] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.331331 17682 raft_consensus.cc:385] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.334831 17682 raft_consensus.cc:740] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.334972 17692 leader_election.cc:290] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:52.335723 17682 consensus_queue.cc:260] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.336314 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ccac61a2bab40708cfe17c17ae1de65" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:52.338503 17678 consensus_queue.cc:260] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.339035 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ccac61a2bab40708cfe17c17ae1de65" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:52.344687 17681 tablet_bootstrap.cc:654] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:52.345265 17355 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:52.345481 17373 leader_election.cc:304] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:52.349653 17682 ts_tablet_manager.cc:1434] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 13:06:52.350914 17678 ts_tablet_manager.cc:1434] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:06:52.352092 17678 tablet_bootstrap.cc:492] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:52.355906 17682 tablet_bootstrap.cc:492] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:52.361001 17749 consensus_queue.cc:1048] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.365151 17692 raft_consensus.cc:2749] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.374775 17691 consensus_queue.cc:1048] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.376619 17308 catalog_manager.cc:5697] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:52.379560 17678 tablet_bootstrap.cc:654] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.386258 17682 tablet_bootstrap.cc:654] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.399178 17749 raft_consensus.cc:493] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.399871 17749 raft_consensus.cc:515] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.402381 17749 leader_election.cc:290] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:52.403745 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b734b40edd440fad746ea61f858c1b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:52.405130 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b734b40edd440fad746ea61f858c1b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:52.405510 17485 raft_consensus.cc:2468] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:06:52.405941 17409 raft_consensus.cc:2468] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:06:52.407364 17522 leader_election.cc:304] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:52.409015 17677 tablet_bootstrap.cc:492] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:52.409786 17677 ts_tablet_manager.cc:1403] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.094s	user 0.020s	sys 0.028s
I20260709 13:06:52.413940 17780 raft_consensus.cc:493] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.414099 17677 raft_consensus.cc:359] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.414522 17780 raft_consensus.cc:515] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.415287 17677 raft_consensus.cc:385] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.415720 17677 raft_consensus.cc:740] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.416697 17677 consensus_queue.cc:260] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.420394 17677 ts_tablet_manager.cc:1434] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.005s
I20260709 13:06:52.421350 17691 raft_consensus.cc:2804] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.421545 17677 tablet_bootstrap.cc:492] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:52.421885 17691 raft_consensus.cc:493] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.422286 17691 raft_consensus.cc:3060] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.422971 17678 tablet_bootstrap.cc:492] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:52.423543 17678 ts_tablet_manager.cc:1403] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.072s	user 0.008s	sys 0.015s
I20260709 13:06:52.426117 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d19239b38b1485ca53f239a71a11444" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:52.427160 17678 raft_consensus.cc:359] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.427887 17678 raft_consensus.cc:385] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.428243 17678 raft_consensus.cc:740] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.426895 17559 raft_consensus.cc:2468] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:52.429183 17678 consensus_queue.cc:260] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.429822 17677 tablet_bootstrap.cc:654] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.430142 17451 leader_election.cc:304] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:52.431417 17685 raft_consensus.cc:2804] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.431981 17685 raft_consensus.cc:493] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.432401 17685 raft_consensus.cc:3060] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.432489 17678 ts_tablet_manager.cc:1434] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:06:52.438274 17678 tablet_bootstrap.cc:492] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:52.439224 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d19239b38b1485ca53f239a71a11444" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:52.440068 17632 raft_consensus.cc:2468] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:52.432009 17780 leader_election.cc:290] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:52.452795 17677 tablet_bootstrap.cc:492] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:52.453644 17677 ts_tablet_manager.cc:1403] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.005s
I20260709 13:06:52.457378 17677 raft_consensus.cc:359] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.457495 17685 raft_consensus.cc:515] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.458271 17677 raft_consensus.cc:385] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.458686 17682 tablet_bootstrap.cc:492] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:52.459347 17682 ts_tablet_manager.cc:1403] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.104s	user 0.013s	sys 0.009s
I20260709 13:06:52.461871 17681 tablet_bootstrap.cc:492] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:52.462466 17681 ts_tablet_manager.cc:1403] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.134s	user 0.014s	sys 0.002s
I20260709 13:06:52.458699 17677 raft_consensus.cc:740] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.463034 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d19239b38b1485ca53f239a71a11444" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:52.463721 17559 raft_consensus.cc:3060] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.463539 17677 consensus_queue.cc:260] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.464015 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d19239b38b1485ca53f239a71a11444" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:52.464623 17632 raft_consensus.cc:3060] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.466096 17677 ts_tablet_manager.cc:1434] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:06:52.465700 17681 raft_consensus.cc:359] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.467092 17681 raft_consensus.cc:385] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.467235 17677 tablet_bootstrap.cc:492] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:52.467403 17681 raft_consensus.cc:740] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.468315 17681 consensus_queue.cc:260] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.469105 17691 raft_consensus.cc:515] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.470592 17681 ts_tablet_manager.cc:1434] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.001s
I20260709 13:06:52.471515 17691 leader_election.cc:290] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:52.472718 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b734b40edd440fad746ea61f858c1b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:52.473111 17678 tablet_bootstrap.cc:654] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.473373 17485 raft_consensus.cc:3060] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.474516 17682 raft_consensus.cc:359] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.475347 17682 raft_consensus.cc:385] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.475606 17682 raft_consensus.cc:740] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.476348 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b734b40edd440fad746ea61f858c1b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:52.476332 17682 consensus_queue.cc:260] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.477105 17409 raft_consensus.cc:3060] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.478335 17632 raft_consensus.cc:2468] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:52.478663 17682 ts_tablet_manager.cc:1434] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:06:52.474642 17685 leader_election.cc:290] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:52.484462 17450 leader_election.cc:304] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:52.485419 17677 tablet_bootstrap.cc:654] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.487808 17485 raft_consensus.cc:2468] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:52.488935 17681 tablet_bootstrap.cc:492] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:52.489311 17522 leader_election.cc:304] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:52.490355 17789 raft_consensus.cc:2804] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.490954 17789 raft_consensus.cc:697] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:06:52.491874 17789 consensus_queue.cc:237] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.500224 17681 tablet_bootstrap.cc:654] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.501368 17685 raft_consensus.cc:2804] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.505465 17685 raft_consensus.cc:697] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:52.505739 17677 tablet_bootstrap.cc:492] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:52.507743 17677 ts_tablet_manager.cc:1403] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.041s	user 0.006s	sys 0.006s
I20260709 13:06:52.511566 17559 raft_consensus.cc:2468] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:52.511370 17677 raft_consensus.cc:359] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.515972 17677 raft_consensus.cc:385] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.516542 17677 raft_consensus.cc:740] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.517642 17677 consensus_queue.cc:260] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.512480 17685 consensus_queue.cc:237] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.483853 17682 tablet_bootstrap.cc:492] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:52.514669 17409 raft_consensus.cc:2468] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:52.530299 17304 catalog_manager.cc:5697] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:52.533679 17677 ts_tablet_manager.cc:1434] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.002s
I20260709 13:06:52.539840 17677 tablet_bootstrap.cc:492] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:52.542897 17678 tablet_bootstrap.cc:492] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:52.543013 17789 consensus_queue.cc:1048] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.543708 17678 ts_tablet_manager.cc:1403] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.106s	user 0.018s	sys 0.031s
I20260709 13:06:52.548636 17678 raft_consensus.cc:359] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.549527 17678 raft_consensus.cc:385] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.549875 17678 raft_consensus.cc:740] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.550779 17678 consensus_queue.cc:260] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.553337 17678 ts_tablet_manager.cc:1434] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:06:52.554881 17678 tablet_bootstrap.cc:492] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:52.562278 17682 tablet_bootstrap.cc:654] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.563788 17678 tablet_bootstrap.cc:654] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.569872 17789 consensus_queue.cc:1048] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.573624 17677 tablet_bootstrap.cc:654] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.589205 17681 tablet_bootstrap.cc:492] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:52.593633 17682 tablet_bootstrap.cc:492] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:52.596767 17682 ts_tablet_manager.cc:1403] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.113s	user 0.008s	sys 0.007s
I20260709 13:06:52.599066 17692 raft_consensus.cc:493] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.599699 17692 raft_consensus.cc:515] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.603062 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:52.603860 17692 leader_election.cc:290] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:52.604017 17485 raft_consensus.cc:2468] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:52.603487 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:52.604828 17678 tablet_bootstrap.cc:492] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:52.605572 17678 ts_tablet_manager.cc:1403] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.051s	user 0.019s	sys 0.000s
W20260709 13:06:52.605994 17376 leader_election.cc:343] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.606791 17373 leader_election.cc:304] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:52.609601 17678 raft_consensus.cc:359] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.610725 17678 raft_consensus.cc:385] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.611064 17678 raft_consensus.cc:740] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.612452 17692 raft_consensus.cc:2804] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.612978 17692 raft_consensus.cc:493] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.613379 17682 raft_consensus.cc:359] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.614208 17682 raft_consensus.cc:385] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.614794 17682 raft_consensus.cc:740] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.613400 17692 raft_consensus.cc:3060] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.624768 17678 consensus_queue.cc:260] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.625219 17304 catalog_manager.cc:5697] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.628037 17681 ts_tablet_manager.cc:1403] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.139s	user 0.039s	sys 0.015s
I20260709 13:06:52.641537 17681 raft_consensus.cc:359] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.642350 17681 raft_consensus.cc:385] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.642661 17681 raft_consensus.cc:740] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.642781 17678 ts_tablet_manager.cc:1434] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.002s
I20260709 13:06:52.630729 17682 consensus_queue.cc:260] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.631780 17745 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.633461 17780 consensus_queue.cc:1048] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.643700 17745 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.647032 17678 tablet_bootstrap.cc:492] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:52.646211 17745 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:52.648272 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:52.650832 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:52.650096 17681 consensus_queue.cc:260] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
W20260709 13:06:52.656222 17598 leader_election.cc:343] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.657224 17780 raft_consensus.cc:493] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.667725 17681 ts_tablet_manager.cc:1434] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.002s
I20260709 13:06:52.669436 17681 tablet_bootstrap.cc:492] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
W20260709 13:06:52.659914 17599 leader_election.cc:343] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.683516 17678 tablet_bootstrap.cc:654] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.671486 17780 raft_consensus.cc:515] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.689073 17780 leader_election.cc:290] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:52.683565 17599 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:52.689471 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e559727ff02540749beb944b741d471f" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:52.690366 17745 raft_consensus.cc:2749] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.696851 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e559727ff02540749beb944b741d471f" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:52.697777 17632 raft_consensus.cc:2468] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
W20260709 13:06:52.699414 17451 leader_election.cc:343] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.700093 17559 raft_consensus.cc:3060] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.703976 17682 ts_tablet_manager.cc:1434] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.107s	user 0.016s	sys 0.015s
I20260709 13:06:52.705307 17682 tablet_bootstrap.cc:492] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:52.708477 17450 leader_election.cc:304] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:52.711038 17750 raft_consensus.cc:2804] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.711535 17750 raft_consensus.cc:493] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.711902 17750 raft_consensus.cc:3060] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.721298 17685 consensus_queue.cc:1048] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.707978 17692 raft_consensus.cc:515] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.724779 17750 raft_consensus.cc:515] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.727700 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:52.728703 17485 raft_consensus.cc:3060] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.731966 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:52.732858 17555 raft_consensus.cc:3060] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.739993 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e559727ff02540749beb944b741d471f" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
W20260709 13:06:52.743252 17451 leader_election.cc:343] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.758796 17682 tablet_bootstrap.cc:654] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.760406 17677 tablet_bootstrap.cc:492] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:52.761201 17677 ts_tablet_manager.cc:1403] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.222s	user 0.028s	sys 0.056s
I20260709 13:06:52.742681 17692 leader_election.cc:290] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:52.763914 17750 leader_election.cc:290] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:52.781324 17409 raft_consensus.cc:3060] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.768149 17789 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.783509 17555 raft_consensus.cc:2468] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:52.783202 17789 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.771782 17485 raft_consensus.cc:2468] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:52.779754 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e559727ff02540749beb944b741d471f" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:52.779599 17677 raft_consensus.cc:359] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.785470 17677 raft_consensus.cc:385] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.785967 17677 raft_consensus.cc:740] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.786451 17681 tablet_bootstrap.cc:654] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.787154 17677 consensus_queue.cc:260] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.790539 17376 leader_election.cc:304] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:06:52.792245 17692 raft_consensus.cc:2804] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.792758 17692 raft_consensus.cc:697] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:06:52.781459 17803 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.793852 17789 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:52.793700 17692 consensus_queue.cc:237] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:52.793996 17803 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.794845 17632 raft_consensus.cc:3060] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.797602 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:52.798352 17555 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:52.797267 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:52.799353 17633 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:06:52.799803 17376 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:06:52.800863 17801 raft_consensus.cc:2804] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.801287 17801 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.801633 17801 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.801544 17524 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:52.797425 17408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:52.803365 17789 raft_consensus.cc:2804] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.806514 17789 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.813195 17677 ts_tablet_manager.cc:1434] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.052s	user 0.011s	sys 0.000s
I20260709 13:06:52.816802 17677 tablet_bootstrap.cc:492] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:52.815217 17789 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.816453 17750 raft_consensus.cc:493] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.822188 17737 raft_consensus.cc:493] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.830026 17737 raft_consensus.cc:515] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.836349 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:52.837203 17633 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:52.833096 17803 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:52.826503 17750 raft_consensus.cc:515] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.851516 17407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
W20260709 13:06:52.853955 17598 leader_election.cc:343] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.853763 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:06:52.857520 17596 leader_election.cc:343] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.858168 17596 leader_election.cc:304] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:52.859246 17745 raft_consensus.cc:2749] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.860601 17678 tablet_bootstrap.cc:492] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:52.861268 17678 ts_tablet_manager.cc:1403] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.214s	user 0.013s	sys 0.018s
I20260709 13:06:52.861145 17632 raft_consensus.cc:2468] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:52.851673 17737 leader_election.cc:290] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:52.862881 17450 leader_election.cc:304] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:52.864714 17685 raft_consensus.cc:2804] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.865283 17685 raft_consensus.cc:697] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:52.865136 17678 raft_consensus.cc:359] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.866073 17678 raft_consensus.cc:385] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.866379 17685 consensus_queue.cc:237] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:52.866803 17801 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.867190 17780 consensus_queue.cc:1048] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.868085 17408 raft_consensus.cc:2393] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:06:52.866379 17678 raft_consensus.cc:740] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.871621 17678 consensus_queue.cc:260] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.875367 17678 ts_tablet_manager.cc:1434] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.001s
I20260709 13:06:52.875944 17682 tablet_bootstrap.cc:492] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:52.876552 17682 ts_tablet_manager.cc:1403] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.171s	user 0.013s	sys 0.013s
I20260709 13:06:52.876632 17678 tablet_bootstrap.cc:492] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:52.885691 17682 raft_consensus.cc:359] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.886519 17682 raft_consensus.cc:385] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.886886 17682 raft_consensus.cc:740] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.887029 17677 tablet_bootstrap.cc:654] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.887669 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:52.889525 17750 leader_election.cc:290] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:52.890013 17682 consensus_queue.cc:260] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.891222 17801 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:52.892817 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:52.893596 17632 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.893965 17682 ts_tablet_manager.cc:1434] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:06:52.895169 17682 tablet_bootstrap.cc:492] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:52.896082 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6633559a9d2643a29622caab848e5442" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:52.896842 17633 raft_consensus.cc:2468] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:52.899783 17450 leader_election.cc:304] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:52.901376 17769 raft_consensus.cc:2804] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.901835 17769 raft_consensus.cc:493] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.902187 17769 raft_consensus.cc:3060] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.903862 17750 consensus_queue.cc:1048] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.910398 17681 tablet_bootstrap.cc:492] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:52.911698 17681 ts_tablet_manager.cc:1403] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.243s	user 0.015s	sys 0.004s
I20260709 13:06:52.911873 17408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6633559a9d2643a29622caab848e5442" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:52.912725 17408 raft_consensus.cc:2468] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:52.913136 17307 catalog_manager.cc:5697] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.915001 17681 raft_consensus.cc:359] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.915822 17681 raft_consensus.cc:385] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.919368 17632 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:52.916090 17681 raft_consensus.cc:740] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.921083 17681 consensus_queue.cc:260] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.921219 17375 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:52.922633 17692 raft_consensus.cc:2804] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.922994 17678 tablet_bootstrap.cc:654] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.923200 17692 raft_consensus.cc:697] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:06:52.924015 17789 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.924302 17692 consensus_queue.cc:237] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:52.925557 17557 raft_consensus.cc:2393] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in this term.
I20260709 13:06:52.926252 17682 tablet_bootstrap.cc:654] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.927701 17789 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:52.931419 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:52.935169 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:52.938185 17304 catalog_manager.cc:5697] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.940289 17632 raft_consensus.cc:2393] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:06:52.920807 17769 raft_consensus.cc:515] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.948428 17681 ts_tablet_manager.cc:1434] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 13:06:52.956149 17524 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:52.967759 17677 tablet_bootstrap.cc:492] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:52.970180 17677 ts_tablet_manager.cc:1403] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.154s	user 0.012s	sys 0.005s
I20260709 13:06:52.968848 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6633559a9d2643a29622caab848e5442" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:52.971200 17632 raft_consensus.cc:3060] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.973675 17677 raft_consensus.cc:359] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.974552 17677 raft_consensus.cc:385] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.974884 17677 raft_consensus.cc:740] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.975386 17749 raft_consensus.cc:2749] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.976416 17749 consensus_queue.cc:1048] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.975770 17677 consensus_queue.cc:260] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.981259 17677 ts_tablet_manager.cc:1434] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:06:52.982020 17678 tablet_bootstrap.cc:492] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:52.982679 17678 ts_tablet_manager.cc:1403] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.106s	user 0.018s	sys 0.030s
I20260709 13:06:52.982720 17677 tablet_bootstrap.cc:492] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:52.985929 17750 consensus_queue.cc:1048] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:52.995317 17677 tablet_bootstrap.cc:654] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.986043 17678 raft_consensus.cc:359] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:52.999435 17678 raft_consensus.cc:385] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.999658 17678 raft_consensus.cc:740] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.000242 17678 consensus_queue.cc:260] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.002251 17678 ts_tablet_manager.cc:1434] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.019s	user 0.002s	sys 0.001s
I20260709 13:06:53.003808 17678 tablet_bootstrap.cc:492] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:53.012864 17677 tablet_bootstrap.cc:492] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:53.012930 17678 tablet_bootstrap.cc:654] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.013692 17677 ts_tablet_manager.cc:1403] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.003s
I20260709 13:06:53.018123 17677 raft_consensus.cc:359] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.019330 17677 raft_consensus.cc:385] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.019773 17677 raft_consensus.cc:740] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.020462 17681 tablet_bootstrap.cc:492] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:53.020855 17677 consensus_queue.cc:260] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.025048 17677 ts_tablet_manager.cc:1434] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.001s
I20260709 13:06:53.026535 17678 tablet_bootstrap.cc:492] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:53.027012 17677 tablet_bootstrap.cc:492] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:53.026772 17789 consensus_queue.cc:1048] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.027364 17678 ts_tablet_manager.cc:1403] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.009s
I20260709 13:06:53.033622 17678 raft_consensus.cc:359] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:53.034746 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6633559a9d2643a29622caab848e5442" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:53.035545 17409 raft_consensus.cc:3060] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.037629 17769 leader_election.cc:290] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:53.043633 17677 tablet_bootstrap.cc:654] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.034723 17678 raft_consensus.cc:385] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.044413 17678 raft_consensus.cc:740] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.046716 17678 consensus_queue.cc:260] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:53.045476 17409 raft_consensus.cc:2468] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:53.049454 17450 leader_election.cc:304] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:53.050586 17769 raft_consensus.cc:2804] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.051399 17769 raft_consensus.cc:697] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:53.051541 17678 ts_tablet_manager.cc:1434] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 13:06:53.052815 17678 tablet_bootstrap.cc:492] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:53.052357 17769 consensus_queue.cc:237] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.062299 17692 raft_consensus.cc:493] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.063225 17678 tablet_bootstrap.cc:654] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.063040 17692 raft_consensus.cc:515] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.066965 17750 consensus_queue.cc:1048] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.018358 17632 raft_consensus.cc:2468] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:53.078578 17307 catalog_manager.cc:5697] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.086369 17682 tablet_bootstrap.cc:492] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:53.087150 17682 ts_tablet_manager.cc:1403] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.192s	user 0.013s	sys 0.006s
I20260709 13:06:53.092233 17678 tablet_bootstrap.cc:492] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:53.093093 17678 ts_tablet_manager.cc:1403] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.006s
I20260709 13:06:53.091723 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61235f863ca44ed3b6c4715dfd68e223" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:53.097411 17557 raft_consensus.cc:2468] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:53.092372 17692 leader_election.cc:290] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:53.098461 17677 tablet_bootstrap.cc:492] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:53.099200 17677 ts_tablet_manager.cc:1403] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.072s	user 0.015s	sys 0.007s
I20260709 13:06:53.103124 17678 raft_consensus.cc:359] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.103806 17677 raft_consensus.cc:359] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.104355 17678 raft_consensus.cc:385] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.104780 17677 raft_consensus.cc:385] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.104909 17678 raft_consensus.cc:740] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.105208 17677 raft_consensus.cc:740] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.105940 17678 consensus_queue.cc:260] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.108176 17678 ts_tablet_manager.cc:1434] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:06:53.109414 17682 raft_consensus.cc:359] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.110457 17682 raft_consensus.cc:385] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.110960 17682 raft_consensus.cc:740] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.106029 17677 consensus_queue.cc:260] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.115370 17682 consensus_queue.cc:260] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.116195 17681 tablet_bootstrap.cc:654] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.117997 17376 leader_election.cc:304] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:06:53.123284 17678 tablet_bootstrap.cc:492] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:53.123445 17803 raft_consensus.cc:2804] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.124509 17803 raft_consensus.cc:493] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.111218 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61235f863ca44ed3b6c4715dfd68e223" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:53.124904 17803 raft_consensus.cc:3060] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.131069 17750 raft_consensus.cc:493] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.131990 17750 raft_consensus.cc:515] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.134959 17632 raft_consensus.cc:2468] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:53.137395 17682 ts_tablet_manager.cc:1434] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 13:06:53.137908 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d047496a466464ba1777cf63c5c84b0" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:53.144832 17410 raft_consensus.cc:2468] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:53.146286 17450 leader_election.cc:304] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:53.140219 17678 tablet_bootstrap.cc:654] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.147787 17780 raft_consensus.cc:2804] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.148262 17780 raft_consensus.cc:493] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.148773 17780 raft_consensus.cc:3060] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.144680 17803 raft_consensus.cc:515] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.155381 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61235f863ca44ed3b6c4715dfd68e223" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:53.156371 17632 raft_consensus.cc:3060] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.156738 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d047496a466464ba1777cf63c5c84b0" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:06:53.158731 17451 leader_election.cc:343] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.164232 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61235f863ca44ed3b6c4715dfd68e223" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:53.165166 17557 raft_consensus.cc:3060] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.166869 17632 raft_consensus.cc:2468] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:53.143762 17750 leader_election.cc:290] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:53.168411 17375 leader_election.cc:304] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:53.169483 17692 raft_consensus.cc:2804] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.170030 17692 raft_consensus.cc:697] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:06:53.171006 17692 consensus_queue.cc:237] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.172640 17677 ts_tablet_manager.cc:1434] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.073s	user 0.008s	sys 0.001s
I20260709 13:06:53.173880 17677 tablet_bootstrap.cc:492] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:53.145592 17682 tablet_bootstrap.cc:492] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:53.162410 17803 leader_election.cc:290] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:53.174022 17780 raft_consensus.cc:515] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.199229 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d047496a466464ba1777cf63c5c84b0" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:53.202766 17678 tablet_bootstrap.cc:492] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
W20260709 13:06:53.203281 17451 leader_election.cc:343] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.203482 17678 ts_tablet_manager.cc:1403] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.080s	user 0.016s	sys 0.006s
I20260709 13:06:53.205461 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d047496a466464ba1777cf63c5c84b0" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:53.206234 17410 raft_consensus.cc:3060] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.207386 17678 raft_consensus.cc:359] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:53.208258 17678 raft_consensus.cc:385] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.208532 17678 raft_consensus.cc:740] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.209753 17678 consensus_queue.cc:260] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:53.210940 17745 raft_consensus.cc:493] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.211524 17745 raft_consensus.cc:515] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.212283 17678 ts_tablet_manager.cc:1434] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:53.213003 17681 tablet_bootstrap.cc:492] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:53.213524 17678 tablet_bootstrap.cc:492] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:53.213675 17681 ts_tablet_manager.cc:1403] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.193s	user 0.015s	sys 0.003s
I20260709 13:06:53.214931 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:06:53.216557 17599 leader_election.cc:343] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.218592 17677 tablet_bootstrap.cc:654] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.216666 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:53.217056 17769 raft_consensus.cc:493] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.219494 17769 raft_consensus.cc:515] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.210206 17557 raft_consensus.cc:2468] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:53.221504 17678 tablet_bootstrap.cc:654] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.221947 17769 leader_election.cc:290] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
W20260709 13:06:53.222991 17598 leader_election.cc:343] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.215083 17745 leader_election.cc:290] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:53.228333 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:53.233429 17598 leader_election.cc:304] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:53.233934 17632 raft_consensus.cc:2468] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:53.228539 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:53.236563 17745 raft_consensus.cc:2749] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.236876 17678 tablet_bootstrap.cc:492] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:53.236977 17410 raft_consensus.cc:2468] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:53.238294 17450 leader_election.cc:304] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:53.246906 17682 tablet_bootstrap.cc:654] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.247318 17769 raft_consensus.cc:2804] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:06:53.247406 17450 leader_election.cc:343] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.248034 17769 raft_consensus.cc:493] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.248513 17769 raft_consensus.cc:3060] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.249141 17450 leader_election.cc:304] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:53.247531 17305 catalog_manager.cc:5697] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.238063 17780 leader_election.cc:290] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:53.243553 17678 ts_tablet_manager.cc:1403] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.006s
I20260709 13:06:53.246207 17677 tablet_bootstrap.cc:492] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:53.255257 17677 ts_tablet_manager.cc:1403] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.001s
I20260709 13:06:53.239280 17681 raft_consensus.cc:359] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:53.257467 17769 raft_consensus.cc:515] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.258147 17681 raft_consensus.cc:385] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.258643 17681 raft_consensus.cc:740] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.260049 17769 leader_election.cc:290] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:53.259672 17681 consensus_queue.cc:260] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:53.262513 17681 ts_tablet_manager.cc:1434] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.048s	user 0.009s	sys 0.000s
I20260709 13:06:53.263245 17829 raft_consensus.cc:2804] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.262892 17304 catalog_manager.cc:5697] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.264042 17829 raft_consensus.cc:697] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:53.265437 17677 raft_consensus.cc:359] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.270121 17677 raft_consensus.cc:385] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.267308 17678 raft_consensus.cc:359] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.268311 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:53.271034 17632 raft_consensus.cc:3060] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.271191 17681 tablet_bootstrap.cc:492] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:53.269446 17682 tablet_bootstrap.cc:492] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:53.273535 17682 ts_tablet_manager.cc:1403] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.128s	user 0.008s	sys 0.020s
I20260709 13:06:53.276973 17682 raft_consensus.cc:359] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.277853 17682 raft_consensus.cc:385] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.278141 17682 raft_consensus.cc:740] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.269964 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:53.265290 17829 consensus_queue.cc:237] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.279346 17678 raft_consensus.cc:385] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.279688 17678 raft_consensus.cc:740] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.280120 17681 tablet_bootstrap.cc:654] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.280503 17678 consensus_queue.cc:260] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.283067 17678 ts_tablet_manager.cc:1434] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:06:53.278952 17682 consensus_queue.cc:260] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.284276 17410 raft_consensus.cc:3060] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.286036 17682 ts_tablet_manager.cc:1434] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 13:06:53.287272 17682 tablet_bootstrap.cc:492] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:53.294225 17682 tablet_bootstrap.cc:654] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.296116 17745 raft_consensus.cc:493] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.296846 17745 raft_consensus.cc:515] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:53.301178 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bc57cc906f4179a8f6ff86c42b0247" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:06:53.304004 17596 leader_election.cc:343] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.270502 17677 raft_consensus.cc:740] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.310365 17745 leader_election.cc:290] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:53.312275 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bc57cc906f4179a8f6ff86c42b0247" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:06:53.314446 17599 leader_election.cc:343] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.315022 17599 leader_election.cc:304] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:53.319227 17745 raft_consensus.cc:2749] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.321939 17678 tablet_bootstrap.cc:492] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:53.323910 17804 consensus_queue.cc:1048] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.309754 17677 consensus_queue.cc:260] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.325343 17632 raft_consensus.cc:2468] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:53.337143 17780 consensus_queue.cc:1048] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.343111 17450 leader_election.cc:304] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:53.351967 17678 tablet_bootstrap.cc:654] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.332455 17677 ts_tablet_manager.cc:1434] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.002s
I20260709 13:06:53.359215 17829 raft_consensus.cc:2804] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.360090 17829 raft_consensus.cc:697] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:53.361150 17829 consensus_queue.cc:237] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.354645 17410 raft_consensus.cc:2468] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
W20260709 13:06:53.371520 17355 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:53.375744 17804 consensus_queue.cc:1048] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.390857 17305 catalog_manager.cc:5697] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.392524 17557 raft_consensus.cc:3060] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.359819 17677 tablet_bootstrap.cc:492] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:53.410550 17677 tablet_bootstrap.cc:654] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.415005 17829 consensus_queue.cc:1048] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:53.419118 17682 tablet_bootstrap.cc:492] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:53.421663 17681 tablet_bootstrap.cc:492] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:53.432742 17681 ts_tablet_manager.cc:1403] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.162s	user 0.029s	sys 0.035s
I20260709 13:06:53.432633 17682 ts_tablet_manager.cc:1403] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.146s	user 0.033s	sys 0.073s
I20260709 13:06:53.436297 17682 raft_consensus.cc:359] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.436450 17681 raft_consensus.cc:359] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.437157 17682 raft_consensus.cc:385] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.437340 17681 raft_consensus.cc:385] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.437542 17682 raft_consensus.cc:740] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.437796 17780 raft_consensus.cc:493] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.437908 17681 raft_consensus.cc:740] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.438580 17780 raft_consensus.cc:515] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.439040 17681 consensus_queue.cc:260] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.440959 17769 raft_consensus.cc:493] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.441594 17681 ts_tablet_manager.cc:1434] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:06:53.441570 17769 raft_consensus.cc:515] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.441823 17780 leader_election.cc:290] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:53.443077 17804 consensus_queue.cc:1048] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.444020 17769 leader_election.cc:290] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:53.443231 17682 consensus_queue.cc:260] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.446497 17745 raft_consensus.cc:493] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.447299 17745 raft_consensus.cc:515] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.451496 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a74eee34cc514b79b981a6ef6c45d7e2" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:53.452369 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36168ca382614c3f9061a116e5d12df5" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:53.452818 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36168ca382614c3f9061a116e5d12df5" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:53.453130 17485 raft_consensus.cc:2468] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 0.
W20260709 13:06:53.453157 17451 leader_election.cc:343] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:53.455188 17598 leader_election.cc:343] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.456638 17596 leader_election.cc:304] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:53.457754 17737 raft_consensus.cc:2804] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.457842 17745 leader_election.cc:290] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:53.458232 17737 raft_consensus.cc:493] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.458606 17682 ts_tablet_manager.cc:1434] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:06:53.458667 17737 raft_consensus.cc:3060] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.460225 17681 tablet_bootstrap.cc:492] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:53.460379 17682 tablet_bootstrap.cc:492] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:53.460623 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36168ca382614c3f9061a116e5d12df5" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:53.463001 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36168ca382614c3f9061a116e5d12df5" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:53.464370 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a74eee34cc514b79b981a6ef6c45d7e2" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
W20260709 13:06:53.467877 17450 leader_election.cc:343] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:53.468706 17450 leader_election.cc:343] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.469327 17450 leader_election.cc:304] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:53.470192 17769 raft_consensus.cc:2749] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.472723 17682 tablet_bootstrap.cc:654] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.477061 17681 tablet_bootstrap.cc:654] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.489950 17682 tablet_bootstrap.cc:492] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:53.490273 17801 consensus_queue.cc:1048] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.490919 17682 ts_tablet_manager.cc:1403] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.008s
I20260709 13:06:53.497639 17737 raft_consensus.cc:515] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.500337 17737 leader_election.cc:290] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:53.503006 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36168ca382614c3f9061a116e5d12df5" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:53.503785 17485 raft_consensus.cc:3060] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.506755 17632 raft_consensus.cc:2393] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 1: Already voted for candidate a9a65d4c22be403682be9bc817650ada in this term.
I20260709 13:06:53.509239 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36168ca382614c3f9061a116e5d12df5" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
W20260709 13:06:53.511160 17598 leader_election.cc:343] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.517654 17678 tablet_bootstrap.cc:492] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:53.518537 17678 ts_tablet_manager.cc:1403] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.197s	user 0.014s	sys 0.003s
I20260709 13:06:53.528429 17682 raft_consensus.cc:359] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.530234 17682 raft_consensus.cc:385] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.533596 17678 raft_consensus.cc:359] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.534952 17678 raft_consensus.cc:385] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.535362 17678 raft_consensus.cc:740] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.537439 17682 raft_consensus.cc:740] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.540834 17682 consensus_queue.cc:260] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.543713 17678 consensus_queue.cc:260] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.546171 17834 raft_consensus.cc:493] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.546907 17834 raft_consensus.cc:515] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.548653 17801 raft_consensus.cc:493] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.549604 17801 raft_consensus.cc:515] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.551110 17450 leader_election.cc:304] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:53.553031 17801 leader_election.cc:290] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:53.553769 17834 leader_election.cc:290] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:53.553305 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:53.555596 17682 ts_tablet_manager.cc:1434] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.002s
I20260709 13:06:53.556383 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:53.556785 17682 tablet_bootstrap.cc:492] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:06:53.558040 17451 leader_election.cc:343] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:53.559517 17376 leader_election.cc:343] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.564080 17678 ts_tablet_manager.cc:1434] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.045s	user 0.011s	sys 0.000s
I20260709 13:06:53.566075 17678 tablet_bootstrap.cc:492] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:53.576110 17306 catalog_manager.cc:5697] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.584606 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:53.584972 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:53.585562 17483 raft_consensus.cc:2468] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:53.585774 17410 raft_consensus.cc:2468] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:53.586688 17485 raft_consensus.cc:2468] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 1.
I20260709 13:06:53.587373 17769 raft_consensus.cc:2749] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.593385 17450 leader_election.cc:304] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:53.599406 17745 raft_consensus.cc:493] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.599843 17834 raft_consensus.cc:2804] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.600484 17834 raft_consensus.cc:493] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.600873 17834 raft_consensus.cc:3060] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.599989 17745 raft_consensus.cc:515] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.603231 17737 raft_consensus.cc:493] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.604074 17737 raft_consensus.cc:515] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:53.609900 17678 tablet_bootstrap.cc:654] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.593971 17596 leader_election.cc:304] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:53.618132 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deeb668bf4814b1bb7e01791d6f96914" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:53.623484 17681 tablet_bootstrap.cc:492] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:53.624223 17681 ts_tablet_manager.cc:1403] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.164s	user 0.025s	sys 0.028s
I20260709 13:06:53.627146 17682 tablet_bootstrap.cc:654] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.631879 17681 raft_consensus.cc:359] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.632963 17681 raft_consensus.cc:385] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.633335 17681 raft_consensus.cc:740] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.634320 17681 consensus_queue.cc:260] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.636456 17829 consensus_queue.cc:1048] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.636902 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f330683b8d4b0696fad129a8f2ce23" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:53.638891 17677 tablet_bootstrap.cc:492] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:53.639081 17737 leader_election.cc:290] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:53.631938 17745 leader_election.cc:290] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:53.640201 17737 raft_consensus.cc:2804] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.640748 17737 raft_consensus.cc:697] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [term 1 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:06:53.598716 17373 leader_election.cc:304] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:53.639609 17677 ts_tablet_manager.cc:1403] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.280s	user 0.047s	sys 0.082s
I20260709 13:06:53.636798 17681 ts_tablet_manager.cc:1434] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
W20260709 13:06:53.645134 17373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 50108us
I20260709 13:06:53.646850 17801 raft_consensus.cc:2804] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.647450 17801 raft_consensus.cc:493] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.646705 17681 tablet_bootstrap.cc:492] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:53.647948 17801 raft_consensus.cc:3060] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.648850 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deeb668bf4814b1bb7e01791d6f96914" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:06:53.650564 17599 leader_election.cc:343] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.651875 17749 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.652447 17749 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.654592 17749 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:53.657336 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:53.658157 17409 raft_consensus.cc:2468] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:06:53.656001 17677 raft_consensus.cc:359] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:53.664188 17677 raft_consensus.cc:385] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.664706 17677 raft_consensus.cc:740] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.665875 17677 consensus_queue.cc:260] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:53.666965 17853 raft_consensus.cc:493] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.667583 17853 raft_consensus.cc:515] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.659256 17692 consensus_queue.cc:1048] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:06:53.679090 17524 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:06:53.679129 17677 ts_tablet_manager.cc:1434] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.001s
I20260709 13:06:53.680812 17677 tablet_bootstrap.cc:492] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:53.679617 17780 consensus_queue.cc:1048] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:06:53.641707 17737 consensus_queue.cc:237] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.687126 17749 raft_consensus.cc:2804] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.708170 17749 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.725453 17749 raft_consensus.cc:3060] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.719926 17834 raft_consensus.cc:515] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.735801 17780 consensus_queue.cc:1048] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
W20260709 13:06:53.752513 17598 leader_election.cc:343] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.753291 17853 leader_election.cc:290] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:53.753427 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b1eabbf0fc942fabea0c59e6997ad23" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:53.754453 17678 tablet_bootstrap.cc:492] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:53.755038 17678 ts_tablet_manager.cc:1403] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.189s	user 0.021s	sys 0.017s
I20260709 13:06:53.762514 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f330683b8d4b0696fad129a8f2ce23" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:53.767016 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b1eabbf0fc942fabea0c59e6997ad23" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:53.766662 17678 raft_consensus.cc:359] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.769464 17678 raft_consensus.cc:385] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.769743 17677 tablet_bootstrap.cc:654] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.769888 17678 raft_consensus.cc:740] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.778190 17678 consensus_queue.cc:260] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:06:53.767059 17599 leader_election.cc:343] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.780866 17682 tablet_bootstrap.cc:492] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:53.781108 17678 ts_tablet_manager.cc:1434] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:06:53.781656 17682 ts_tablet_manager.cc:1403] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.225s	user 0.025s	sys 0.025s
I20260709 13:06:53.782263 17678 tablet_bootstrap.cc:492] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:53.783181 17692 consensus_queue.cc:1048] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:53.791386 17598 leader_election.cc:343] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.792050 17598 leader_election.cc:304] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:53.793020 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:53.794644 17853 raft_consensus.cc:2749] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.795126 17834 leader_election.cc:290] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:53.799103 17681 tablet_bootstrap.cc:654] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.799320 17678 tablet_bootstrap.cc:654] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.803222 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
W20260709 13:06:53.803858 17451 consensus_peers.cc:597] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e -> Peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Couldn't send request to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:53.804064 17557 raft_consensus.cc:3060] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.798563 17682 raft_consensus.cc:359] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.806456 17682 raft_consensus.cc:385] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.806973 17682 raft_consensus.cc:740] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.808866 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:53.809674 17632 raft_consensus.cc:2468] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
W20260709 13:06:53.812561 17596 leader_election.cc:343] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.814476 17780 consensus_queue.cc:1048] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.813127 17596 leader_election.cc:304] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:53.816103 17854 raft_consensus.cc:2749] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:53.818536 17596 leader_election.cc:343] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.819579 17596 leader_election.cc:304] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:53.821694 17854 raft_consensus.cc:2749] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.841470 17682 consensus_queue.cc:260] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.862831 17803 raft_consensus.cc:493] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.873512 17682 ts_tablet_manager.cc:1434] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.091s	user 0.008s	sys 0.001s
I20260709 13:06:53.875445 17682 tablet_bootstrap.cc:492] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:53.873559 17304 catalog_manager.cc:5697] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 0 to 1, leader changed from <none> to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 1 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.866902 17801 raft_consensus.cc:515] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:53.866597 17803 raft_consensus.cc:515] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.886835 17410 raft_consensus.cc:2393] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:06:53.892057 17737 raft_consensus.cc:493] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.868463 17749 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.922345 17749 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:53.924068 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:53.922391 17678 tablet_bootstrap.cc:492] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:53.930570 17678 ts_tablet_manager.cc:1403] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.149s	user 0.033s	sys 0.019s
I20260709 13:06:53.924973 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:53.937644 17677 tablet_bootstrap.cc:492] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:53.938532 17677 ts_tablet_manager.cc:1403] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.258s	user 0.010s	sys 0.014s
I20260709 13:06:53.933745 17633 raft_consensus.cc:3060] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.946949 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:53.919402 17737 raft_consensus.cc:515] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:53.938558 17410 raft_consensus.cc:3060] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.951521 17681 tablet_bootstrap.cc:492] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:53.952275 17681 ts_tablet_manager.cc:1403] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.306s	user 0.007s	sys 0.013s
I20260709 13:06:53.953668 17485 raft_consensus.cc:2393] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
I20260709 13:06:53.955076 17682 tablet_bootstrap.cc:654] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.956125 17681 raft_consensus.cc:359] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.956944 17681 raft_consensus.cc:385] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.957245 17681 raft_consensus.cc:740] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.958010 17681 consensus_queue.cc:260] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.959782 17681 ts_tablet_manager.cc:1434] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:53.962407 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:53.965790 17681 tablet_bootstrap.cc:492] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:53.966590 17678 raft_consensus.cc:359] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:53.967482 17678 raft_consensus.cc:385] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.967887 17678 raft_consensus.cc:740] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.972306 17737 leader_election.cc:290] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:53.973295 17408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
W20260709 13:06:53.975250 17598 leader_election.cc:343] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.976328 17633 raft_consensus.cc:2468] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:53.976444 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:53.980581 17373 leader_election.cc:304] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:53.968801 17678 consensus_queue.cc:260] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:53.983801 17524 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:53.944190 17801 leader_election.cc:290] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:53.985024 17678 ts_tablet_manager.cc:1434] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.054s	user 0.000s	sys 0.007s
I20260709 13:06:53.985620 17692 raft_consensus.cc:2749] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.986166 17678 tablet_bootstrap.cc:492] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:53.986414 17749 raft_consensus.cc:2804] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.987591 17749 raft_consensus.cc:697] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:06:53.988623 17749 consensus_queue.cc:237] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:53.989367 17410 raft_consensus.cc:2468] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:53.990566 17803 leader_election.cc:290] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
W20260709 13:06:53.991423 17596 leader_election.cc:343] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.992012 17596 leader_election.cc:304] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:53.992820 17737 raft_consensus.cc:2749] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.992611 17677 raft_consensus.cc:359] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:53.993602 17677 raft_consensus.cc:385] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.993919 17677 raft_consensus.cc:740] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.994772 17677 consensus_queue.cc:260] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.000402 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:54.003293 17829 consensus_queue.cc:1048] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:06:54.007266 17631 raft_consensus.cc:2468] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:54.009641 17375 leader_election.cc:304] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:54.013953 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:54.014894 17559 raft_consensus.cc:2468] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:54.022414 17678 tablet_bootstrap.cc:654] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.027457 17677 ts_tablet_manager.cc:1434] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.088s	user 0.008s	sys 0.001s
I20260709 13:06:54.029652 17677 tablet_bootstrap.cc:492] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:54.031944 17803 raft_consensus.cc:2804] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.032492 17803 raft_consensus.cc:493] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.032910 17803 raft_consensus.cc:3060] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.038053 17682 tablet_bootstrap.cc:492] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:54.038827 17682 ts_tablet_manager.cc:1403] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.164s	user 0.025s	sys 0.005s
I20260709 13:06:54.042164 17682 raft_consensus.cc:359] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.043474 17682 raft_consensus.cc:385] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.043848 17682 raft_consensus.cc:740] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.991477 17557 raft_consensus.cc:2468] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:54.045064 17682 consensus_queue.cc:260] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.047192 17803 raft_consensus.cc:515] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:54.048822 17682 ts_tablet_manager.cc:1434] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:06:54.054854 17451 leader_election.cc:304] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:54.056881 17769 raft_consensus.cc:2804] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.057610 17769 raft_consensus.cc:697] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:54.059202 17769 consensus_queue.cc:237] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:54.061266 17682 tablet_bootstrap.cc:492] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:54.083431 17678 tablet_bootstrap.cc:492] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:54.084204 17678 ts_tablet_manager.cc:1403] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.098s	user 0.028s	sys 0.010s
I20260709 13:06:54.090261 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:54.091135 17633 raft_consensus.cc:3060] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.092286 17677 tablet_bootstrap.cc:654] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.093245 17303 catalog_manager.cc:5697] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.091156 17678 raft_consensus.cc:359] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:54.096750 17678 raft_consensus.cc:385] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.097107 17678 raft_consensus.cc:740] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.098011 17678 consensus_queue.cc:260] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:54.102646 17678 ts_tablet_manager.cc:1434] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:06:54.084712 17682 tablet_bootstrap.cc:654] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.085321 17803 leader_election.cc:290] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:54.086215 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:54.104516 17557 raft_consensus.cc:3060] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.107551 17633 raft_consensus.cc:2468] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:54.087356 17681 tablet_bootstrap.cc:654] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.112061 17375 leader_election.cc:304] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:54.113811 17867 raft_consensus.cc:2804] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.114593 17867 raft_consensus.cc:697] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:06:54.115953 17867 consensus_queue.cc:237] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:54.117869 17557 raft_consensus.cc:2468] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:54.120184 17306 catalog_manager.cc:5697] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.126922 17829 raft_consensus.cc:493] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.129915 17829 raft_consensus.cc:515] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.133011 17829 leader_election.cc:290] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:54.103811 17678 tablet_bootstrap.cc:492] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:54.141764 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bc57cc906f4179a8f6ff86c42b0247" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:54.143041 17633 raft_consensus.cc:2468] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:54.136693 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bc57cc906f4179a8f6ff86c42b0247" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:54.146098 17450 leader_election.cc:304] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
W20260709 13:06:54.147786 17451 leader_election.cc:343] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.148202 17769 raft_consensus.cc:2804] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.148706 17769 raft_consensus.cc:493] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.149092 17769 raft_consensus.cc:3060] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.151090 17682 tablet_bootstrap.cc:492] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:54.151873 17682 ts_tablet_manager.cc:1403] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.102s	user 0.016s	sys 0.021s
I20260709 13:06:54.172762 17682 raft_consensus.cc:359] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.173696 17682 raft_consensus.cc:385] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.174026 17682 raft_consensus.cc:740] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.174777 17678 tablet_bootstrap.cc:654] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.174980 17682 consensus_queue.cc:260] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.178359 17682 ts_tablet_manager.cc:1434] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.010s
I20260709 13:06:54.179320 17681 tablet_bootstrap.cc:492] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:54.179651 17682 tablet_bootstrap.cc:492] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:54.180124 17681 ts_tablet_manager.cc:1403] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.215s	user 0.021s	sys 0.004s
I20260709 13:06:54.180594 17677 tablet_bootstrap.cc:492] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:54.181170 17677 ts_tablet_manager.cc:1403] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.152s	user 0.016s	sys 0.005s
I20260709 13:06:54.184281 17677 raft_consensus.cc:359] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.184154 17681 raft_consensus.cc:359] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:54.185178 17677 raft_consensus.cc:385] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.185684 17681 raft_consensus.cc:385] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.186147 17681 raft_consensus.cc:740] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.187196 17681 consensus_queue.cc:260] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:54.188338 17677 raft_consensus.cc:740] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.190075 17677 consensus_queue.cc:260] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.192013 17682 tablet_bootstrap.cc:654] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.198158 17677 ts_tablet_manager.cc:1434] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 13:06:54.202340 17677 tablet_bootstrap.cc:492] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:54.195549 17308 catalog_manager.cc:5697] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.200969 17681 ts_tablet_manager.cc:1434] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.003s
I20260709 13:06:54.210925 17681 tablet_bootstrap.cc:492] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:54.220091 17803 raft_consensus.cc:493] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.220896 17803 raft_consensus.cc:515] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.222474 17769 raft_consensus.cc:515] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.224786 17677 tablet_bootstrap.cc:654] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.226755 17803 leader_election.cc:290] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:54.227583 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f330683b8d4b0696fad129a8f2ce23" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:54.228575 17485 raft_consensus.cc:2468] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:54.229141 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f330683b8d4b0696fad129a8f2ce23" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:54.229903 17633 raft_consensus.cc:2468] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:54.230223 17681 tablet_bootstrap.cc:654] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.230168 17373 leader_election.cc:304] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:54.232033 17871 raft_consensus.cc:2804] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.232436 17871 raft_consensus.cc:493] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.232784 17871 raft_consensus.cc:3060] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.248631 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bc57cc906f4179a8f6ff86c42b0247" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:54.249475 17633 raft_consensus.cc:3060] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.258359 17769 leader_election.cc:290] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:54.260918 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bc57cc906f4179a8f6ff86c42b0247" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:54.263406 17737 consensus_queue.cc:1048] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:54.265717 17633 raft_consensus.cc:2468] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:54.266355 17871 raft_consensus.cc:515] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.267359 17450 leader_election.cc:304] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:54.272722 17678 tablet_bootstrap.cc:492] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:54.273495 17678 ts_tablet_manager.cc:1403] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.170s	user 0.021s	sys 0.035s
I20260709 13:06:54.273303 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f330683b8d4b0696fad129a8f2ce23" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:54.274127 17633 raft_consensus.cc:3060] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:54.276584 17451 leader_election.cc:343] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.278600 17829 raft_consensus.cc:2804] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.278962 17678 raft_consensus.cc:359] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.280221 17678 raft_consensus.cc:385] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.280781 17678 raft_consensus.cc:740] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.279752 17829 raft_consensus.cc:697] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:54.282019 17678 consensus_queue.cc:260] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.282968 17829 consensus_queue.cc:237] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.285838 17678 ts_tablet_manager.cc:1434] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:06:54.287356 17871 leader_election.cc:290] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:54.288713 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f330683b8d4b0696fad129a8f2ce23" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:54.289527 17484 raft_consensus.cc:3060] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.292277 17678 tablet_bootstrap.cc:492] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:54.294899 17682 tablet_bootstrap.cc:492] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:54.295614 17682 ts_tablet_manager.cc:1403] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.116s	user 0.016s	sys 0.005s
I20260709 13:06:54.298508 17633 raft_consensus.cc:2468] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:54.300170 17375 leader_election.cc:304] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:54.302400 17871 raft_consensus.cc:2804] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.303390 17871 raft_consensus.cc:697] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:06:54.302553 17678 tablet_bootstrap.cc:654] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.304662 17871 consensus_queue.cc:237] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.305253 17484 raft_consensus.cc:2468] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:54.300434 17682 raft_consensus.cc:359] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:54.308475 17682 raft_consensus.cc:385] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.309338 17682 raft_consensus.cc:740] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.309752 17677 tablet_bootstrap.cc:492] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:54.310658 17677 ts_tablet_manager.cc:1403] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.109s	user 0.008s	sys 0.011s
I20260709 13:06:54.310511 17682 consensus_queue.cc:260] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:54.314715 17682 ts_tablet_manager.cc:1434] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.019s	user 0.011s	sys 0.000s
I20260709 13:06:54.316013 17682 tablet_bootstrap.cc:492] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:54.314338 17677 raft_consensus.cc:359] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:54.321079 17677 raft_consensus.cc:385] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.323515 17682 tablet_bootstrap.cc:654] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.327399 17769 raft_consensus.cc:493] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.328087 17769 raft_consensus.cc:515] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.334717 17681 tablet_bootstrap.cc:492] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:54.335438 17681 ts_tablet_manager.cc:1403] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.130s	user 0.008s	sys 0.015s
I20260709 13:06:54.337317 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deeb668bf4814b1bb7e01791d6f96914" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:54.338811 17681 raft_consensus.cc:359] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.339530 17308 catalog_manager.cc:5697] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.339653 17681 raft_consensus.cc:385] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.341656 17681 raft_consensus.cc:740] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.342389 17681 consensus_queue.cc:260] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.345274 17681 ts_tablet_manager.cc:1434] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:06:54.321552 17677 raft_consensus.cc:740] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.347748 17677 consensus_queue.cc:260] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:54.325394 17678 tablet_bootstrap.cc:492] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:54.349191 17678 ts_tablet_manager.cc:1403] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.057s	user 0.015s	sys 0.000s
I20260709 13:06:54.352818 17678 raft_consensus.cc:359] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.353641 17678 raft_consensus.cc:385] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.353945 17678 raft_consensus.cc:740] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.358412 17681 tablet_bootstrap.cc:492] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
W20260709 13:06:54.360148 17451 leader_election.cc:343] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.371217 17769 leader_election.cc:290] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:54.375342 17681 tablet_bootstrap.cc:654] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.364148 17304 catalog_manager.cc:5697] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.361476 17677 ts_tablet_manager.cc:1434] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.001s
I20260709 13:06:54.389364 17682 tablet_bootstrap.cc:492] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:54.390007 17682 ts_tablet_manager.cc:1403] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.074s	user 0.012s	sys 0.000s
I20260709 13:06:54.393352 17682 raft_consensus.cc:359] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:54.394117 17682 raft_consensus.cc:385] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.394418 17682 raft_consensus.cc:740] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.394696 17677 tablet_bootstrap.cc:492] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:54.395220 17682 consensus_queue.cc:260] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:54.396873 17678 consensus_queue.cc:260] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.397826 17682 ts_tablet_manager.cc:1434] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:06:54.399125 17682 tablet_bootstrap.cc:492] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:54.399788 17678 ts_tablet_manager.cc:1434] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.001s
I20260709 13:06:54.400979 17678 tablet_bootstrap.cc:492] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:54.406267 17682 tablet_bootstrap.cc:654] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.408440 17677 tablet_bootstrap.cc:654] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.411918 17682 tablet_bootstrap.cc:492] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:54.412613 17682 ts_tablet_manager.cc:1403] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 13:06:54.415962 17682 raft_consensus.cc:359] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.416792 17678 tablet_bootstrap.cc:654] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.416891 17682 raft_consensus.cc:385] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.417430 17682 raft_consensus.cc:740] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.418977 17749 consensus_queue.cc:1048] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:54.421286 17864 consensus_queue.cc:1048] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:54.418510 17682 consensus_queue.cc:260] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.427902 17682 ts_tablet_manager.cc:1434] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:06:54.428138 17737 raft_consensus.cc:493] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.429150 17682 tablet_bootstrap.cc:492] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:54.428932 17737 raft_consensus.cc:515] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.435832 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626dcd42c0f74fccb9c2a42ee7d443ee" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:54.382907 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deeb668bf4814b1bb7e01791d6f96914" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:54.437404 17633 raft_consensus.cc:2468] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:54.380440 17871 raft_consensus.cc:493] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:54.438663 17596 leader_election.cc:343] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.439038 17450 leader_election.cc:304] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:54.439105 17871 raft_consensus.cc:515] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.440605 17829 raft_consensus.cc:2804] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.441031 17829 raft_consensus.cc:493] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.441398 17829 raft_consensus.cc:3060] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.446058 17682 tablet_bootstrap.cc:654] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.441184 17737 leader_election.cc:290] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:54.441849 17871 leader_election.cc:290] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:54.447468 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626dcd42c0f74fccb9c2a42ee7d443ee" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:54.443152 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36168ca382614c3f9061a116e5d12df5" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:06:54.451743 17599 leader_election.cc:343] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.461891 17408 raft_consensus.cc:3060] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.462258 17678 tablet_bootstrap.cc:492] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:54.466837 17678 ts_tablet_manager.cc:1403] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.066s	user 0.019s	sys 0.001s
I20260709 13:06:54.443638 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36168ca382614c3f9061a116e5d12df5" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:54.477769 17737 raft_consensus.cc:493] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.479637 17737 raft_consensus.cc:515] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:54.469986 17599 leader_election.cc:304] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:54.491056 17854 raft_consensus.cc:2749] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.485209 17678 raft_consensus.cc:359] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:54.492815 17678 raft_consensus.cc:385] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.493180 17678 raft_consensus.cc:740] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.497092 17373 leader_election.cc:304] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:54.493983 17678 consensus_queue.cc:260] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:54.524166 17678 ts_tablet_manager.cc:1434] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.057s	user 0.010s	sys 0.000s
W20260709 13:06:54.525360 17355 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:54.528292 17678 tablet_bootstrap.cc:492] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:54.546821 17678 tablet_bootstrap.cc:654] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.556432 17677 tablet_bootstrap.cc:492] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:54.557112 17677 ts_tablet_manager.cc:1403] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.172s	user 0.024s	sys 0.044s
I20260709 13:06:54.562533 17677 raft_consensus.cc:359] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.571000 17677 raft_consensus.cc:385] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.564571 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e84a923caa9c4755bcb78a35143dfbd9" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:54.571624 17677 raft_consensus.cc:740] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.572495 17677 consensus_queue.cc:260] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.576452 17682 tablet_bootstrap.cc:492] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:54.577258 17682 ts_tablet_manager.cc:1403] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.148s	user 0.028s	sys 0.019s
W20260709 13:06:54.579679 17599 leader_election.cc:343] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.581254 17682 raft_consensus.cc:359] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:54.582206 17682 raft_consensus.cc:385] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.582535 17682 raft_consensus.cc:740] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.583454 17682 consensus_queue.cc:260] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:54.586057 17677 ts_tablet_manager.cc:1434] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.001s
I20260709 13:06:54.587404 17677 tablet_bootstrap.cc:492] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:54.567229 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e84a923caa9c4755bcb78a35143dfbd9" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:54.585362 17682 ts_tablet_manager.cc:1434] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
W20260709 13:06:54.594892 17598 leader_election.cc:343] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.595490 17598 leader_election.cc:304] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:54.596633 17854 raft_consensus.cc:2749] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.566668 17737 leader_election.cc:290] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:54.599022 17897 raft_consensus.cc:493] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.599615 17897 raft_consensus.cc:515] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:54.602015 17897 leader_election.cc:290] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:54.595497 17682 tablet_bootstrap.cc:492] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:54.603410 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d047496a466464ba1777cf63c5c84b0" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:54.605348 17737 raft_consensus.cc:493] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.606112 17737 raft_consensus.cc:515] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.611783 17407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
W20260709 13:06:54.614073 17598 leader_election.cc:343] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.614091 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d047496a466464ba1777cf63c5c84b0" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:54.614370 17682 tablet_bootstrap.cc:654] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.617573 17829 raft_consensus.cc:515] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.621517 17854 raft_consensus.cc:493] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.622118 17854 raft_consensus.cc:515] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:54.623119 17681 tablet_bootstrap.cc:492] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:54.623723 17681 ts_tablet_manager.cc:1403] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.266s	user 0.018s	sys 0.042s
I20260709 13:06:54.624476 17854 leader_election.cc:290] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:54.628861 17407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032dc48c2ae44160841c456abd340e40" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:54.629240 17692 raft_consensus.cc:493] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.630142 17692 raft_consensus.cc:515] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:06:54.630748 17598 leader_election.cc:343] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.631119 17871 raft_consensus.cc:2749] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.636271 17834 consensus_queue.cc:1048] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:54.639225 17854 consensus_queue.cc:1048] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:54.642284 17681 raft_consensus.cc:359] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.643246 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:54.643532 17681 raft_consensus.cc:385] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.644085 17681 raft_consensus.cc:740] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
W20260709 13:06:54.645361 17596 leader_election.cc:343] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.645149 17681 consensus_queue.cc:260] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.642148 17829 leader_election.cc:290] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:54.645985 17596 leader_election.cc:304] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:54.647874 17854 raft_consensus.cc:2749] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.639288 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deeb668bf4814b1bb7e01791d6f96914" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:54.648877 17631 raft_consensus.cc:3060] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.642078 17737 leader_election.cc:290] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:54.652741 17681 ts_tablet_manager.cc:1434] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:06:54.654525 17681 tablet_bootstrap.cc:492] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:54.637848 17524 leader_election.cc:304] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:54.655414 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b1eabbf0fc942fabea0c59e6997ad23" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:54.655689 17749 raft_consensus.cc:2749] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.656286 17633 raft_consensus.cc:2468] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:54.657559 17692 leader_election.cc:290] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:54.661686 17375 leader_election.cc:304] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:54.663131 17871 raft_consensus.cc:2804] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.663879 17871 raft_consensus.cc:493] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.664017 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b1eabbf0fc942fabea0c59e6997ad23" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:54.664289 17871 raft_consensus.cc:3060] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.669613 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032dc48c2ae44160841c456abd340e40" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:54.671664 17692 raft_consensus.cc:493] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.672463 17692 raft_consensus.cc:515] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:54.676496 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a74eee34cc514b79b981a6ef6c45d7e2" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:54.677296 17485 raft_consensus.cc:2468] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:54.684628 17749 raft_consensus.cc:493] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.685853 17749 raft_consensus.cc:515] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.690474 17749 leader_election.cc:290] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:54.690675 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bc57cc906f4179a8f6ff86c42b0247" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:54.693507 17867 consensus_queue.cc:1048] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.678997 17373 leader_election.cc:304] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:54.696132 17867 raft_consensus.cc:2804] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.696573 17867 raft_consensus.cc:493] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.696935 17867 raft_consensus.cc:3060] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.699170 17677 tablet_bootstrap.cc:654] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.705408 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deeb668bf4814b1bb7e01791d6f96914" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:54.719151 17737 raft_consensus.cc:493] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.719834 17737 raft_consensus.cc:515] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.731285 17692 leader_election.cc:290] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:54.731916 17678 tablet_bootstrap.cc:492] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:54.732745 17678 ts_tablet_manager.cc:1403] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.205s	user 0.048s	sys 0.027s
I20260709 13:06:54.733620 17557 raft_consensus.cc:3060] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.739131 17678 raft_consensus.cc:359] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.740051 17678 raft_consensus.cc:385] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.740355 17678 raft_consensus.cc:740] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.741197 17682 tablet_bootstrap.cc:492] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:54.744369 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bc57cc906f4179a8f6ff86c42b0247" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:54.744916 17737 leader_election.cc:290] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:54.745085 17682 ts_tablet_manager.cc:1403] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.150s	user 0.019s	sys 0.035s
I20260709 13:06:54.745395 17630 raft_consensus.cc:2393] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
I20260709 13:06:54.747536 17681 tablet_bootstrap.cc:654] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.737322 17871 raft_consensus.cc:515] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:06:54.743707 17599 leader_election.cc:343] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.753654 17599 leader_election.cc:304] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:54.754551 17737 raft_consensus.cc:2749] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.754730 17871 leader_election.cc:290] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:54.758579 17834 raft_consensus.cc:493] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.741142 17678 consensus_queue.cc:260] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
W20260709 13:06:54.748847 17451 leader_election.cc:343] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.759599 17834 raft_consensus.cc:515] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.749382 17692 raft_consensus.cc:493] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.749629 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4b5ba400ad420f89c16a97b71e3b70" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:54.761101 17829 consensus_queue.cc:1048] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.761792 17678 ts_tablet_manager.cc:1434] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:06:54.763883 17678 tablet_bootstrap.cc:492] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:54.764048 17834 leader_election.cc:290] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:54.786464 17678 tablet_bootstrap.cc:654] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.774782 17682 raft_consensus.cc:359] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:54.803777 17682 raft_consensus.cc:385] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.797655 17524 leader_election.cc:304] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:54.804159 17682 raft_consensus.cc:740] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.805171 17682 consensus_queue.cc:260] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:54.808086 17677 tablet_bootstrap.cc:492] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:54.809044 17682 ts_tablet_manager.cc:1434] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.064s	user 0.014s	sys 0.002s
I20260709 13:06:54.809489 17677 ts_tablet_manager.cc:1403] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.222s	user 0.018s	sys 0.001s
I20260709 13:06:54.810849 17682 tablet_bootstrap.cc:492] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:54.816354 17631 raft_consensus.cc:2468] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:54.818193 17450 leader_election.cc:304] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:54.772938 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b1eabbf0fc942fabea0c59e6997ad23" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:54.814024 17677 raft_consensus.cc:359] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.769661 17406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:54.772074 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:54.821077 17749 raft_consensus.cc:2749] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.832499 17630 raft_consensus.cc:2468] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:54.833236 17834 raft_consensus.cc:493] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.784544 17871 raft_consensus.cc:493] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.833854 17834 raft_consensus.cc:515] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:54.824474 17829 raft_consensus.cc:2804] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.828639 17559 raft_consensus.cc:3060] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.828963 17769 consensus_queue.cc:1048] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:54.834169 17871 raft_consensus.cc:515] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:54.834604 17632 raft_consensus.cc:3060] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.835314 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4b5ba400ad420f89c16a97b71e3b70" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:54.826151 17677 raft_consensus.cc:385] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.854231 17677 raft_consensus.cc:740] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.857066 17681 tablet_bootstrap.cc:492] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:54.857975 17681 ts_tablet_manager.cc:1403] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.204s	user 0.017s	sys 0.013s
W20260709 13:06:54.774212 17596 leader_election.cc:343] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.836858 17450 leader_election.cc:304] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:54.841763 17749 raft_consensus.cc:493] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.801505 17692 raft_consensus.cc:515] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:54.863732 17871 leader_election.cc:290] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:54.851063 17780 consensus_queue.cc:1048] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:06:54.767735 17867 raft_consensus.cc:515] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:54.845677 17834 leader_election.cc:290] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:54.866111 17692 leader_election.cc:290] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:54.859462 17677 consensus_queue.cc:260] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.860074 17803 consensus_queue.cc:1048] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.042s
W20260709 13:06:54.882054 17596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 107882us
I20260709 13:06:54.912462 17677 ts_tablet_manager.cc:1434] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.102s	user 0.006s	sys 0.004s
W20260709 13:06:54.917295 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 80523.2us
I20260709 13:06:54.918464 17677 tablet_bootstrap.cc:492] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:54.885200 17406 raft_consensus.cc:2468] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:54.880549 17749 raft_consensus.cc:515] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.900831 17829 raft_consensus.cc:697] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:54.903716 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a74eee34cc514b79b981a6ef6c45d7e2" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:54.905079 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:06:54.913182 17376 leader_election.cc:343] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.930895 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:54.931782 17485 raft_consensus.cc:2468] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:54.931993 17749 leader_election.cc:290] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:54.932920 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:54.933660 17484 raft_consensus.cc:2468] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:54.935199 17682 tablet_bootstrap.cc:654] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.937340 17769 raft_consensus.cc:2804] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.937805 17769 raft_consensus.cc:493] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.938134 17769 raft_consensus.cc:3060] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.941252 17681 raft_consensus.cc:359] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:54.942139 17681 raft_consensus.cc:385] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.922987 17867 leader_election.cc:290] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:54.942425 17681 raft_consensus.cc:740] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.948328 17681 consensus_queue.cc:260] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:54.953282 17681 ts_tablet_manager.cc:1434] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.095s	user 0.010s	sys 0.000s
W20260709 13:06:54.924577 17599 leader_election.cc:343] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.955559 17678 tablet_bootstrap.cc:492] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:54.956477 17678 ts_tablet_manager.cc:1403] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.193s	user 0.024s	sys 0.032s
I20260709 13:06:54.946144 17867 consensus_queue.cc:1048] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:06:54.956770 17829 consensus_queue.cc:237] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:54.959086 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a74eee34cc514b79b981a6ef6c45d7e2" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:54.968645 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:54.959779 17599 leader_election.cc:304] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:54.975215 17485 raft_consensus.cc:3060] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.991110 17737 raft_consensus.cc:493] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.013754 17410 raft_consensus.cc:2468] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:55.018352 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626dcd42c0f74fccb9c2a42ee7d443ee" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:06:55.013080 17596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 61119.6us
I20260709 13:06:55.042954 17919 raft_consensus.cc:2749] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.024197 17864 raft_consensus.cc:493] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.049621 17864 raft_consensus.cc:515] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:55.052178 17864 leader_election.cc:290] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:55.053298 17681 tablet_bootstrap.cc:492] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
W20260709 13:06:55.021420 17599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 98499.7us
I20260709 13:06:55.063674 17632 raft_consensus.cc:2468] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:55.065023 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:55.065907 17630 raft_consensus.cc:2468] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:55.019860 17737 raft_consensus.cc:515] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.069862 17737 leader_election.cc:290] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
W20260709 13:06:55.071950 17451 leader_election.cc:343] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.073508 17485 raft_consensus.cc:2468] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
W20260709 13:06:55.022367 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 83107us
I20260709 13:06:55.080479 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b1eabbf0fc942fabea0c59e6997ad23" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
W20260709 13:06:55.082713 17522 leader_election.cc:343] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.083024 17554 raft_consensus.cc:3060] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.082197 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b1eabbf0fc942fabea0c59e6997ad23" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:55.087806 17410 raft_consensus.cc:2393] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:06:55.090266 17678 raft_consensus.cc:359] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:55.096539 17678 raft_consensus.cc:385] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.097985 17678 raft_consensus.cc:740] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.091008 17553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:55.098942 17678 consensus_queue.cc:260] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:55.100818 17677 tablet_bootstrap.cc:654] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.101964 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626dcd42c0f74fccb9c2a42ee7d443ee" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:55.102943 17630 raft_consensus.cc:2468] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:06:55.104102 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:55.105914 17678 ts_tablet_manager.cc:1434] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.148s	user 0.009s	sys 0.000s
I20260709 13:06:55.105686 17524 leader_election.cc:304] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:55.107231 17678 tablet_bootstrap.cc:492] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:55.107476 17749 raft_consensus.cc:2804] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.108137 17749 raft_consensus.cc:493] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.108634 17749 raft_consensus.cc:3060] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.115108 17375 leader_election.cc:304] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:55.117640 17375 leader_election.cc:304] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:55.119243 17867 raft_consensus.cc:2804] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.120087 17867 raft_consensus.cc:697] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:06:55.120294 17803 raft_consensus.cc:2804] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.120975 17803 raft_consensus.cc:493] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.121495 17803 raft_consensus.cc:3060] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.121140 17867 consensus_queue.cc:237] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:55.130057 17553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:55.141968 17769 raft_consensus.cc:515] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:55.145407 17769 leader_election.cc:290] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:55.164698 17678 tablet_bootstrap.cc:654] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:55.165716 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 221046us
I20260709 13:06:55.172534 17450 leader_election.cc:304] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:55.195772 17373 leader_election.cc:304] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:55.202557 17373 leader_election.cc:304] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:55.202859 17914 raft_consensus.cc:2804] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.203532 17914 raft_consensus.cc:493] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.204013 17914 raft_consensus.cc:3060] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.204082 17692 raft_consensus.cc:2804] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.204663 17692 raft_consensus.cc:493] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.205099 17692 raft_consensus.cc:3060] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.206105 17906 raft_consensus.cc:2804] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.206938 17871 raft_consensus.cc:493] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.207649 17871 raft_consensus.cc:515] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:06:55.176473 17599 leader_election.cc:343] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:55.164362 17596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 52576.1us
I20260709 13:06:55.195632 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b1eabbf0fc942fabea0c59e6997ad23" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:55.223886 17632 raft_consensus.cc:2393] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
W20260709 13:06:55.174396 17376 leader_election.cc:343] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.141907 17803 raft_consensus.cc:515] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:55.228341 17306 catalog_manager.cc:5697] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.216492 17626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:55.236188 17871 leader_election.cc:290] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:55.251155 17678 tablet_bootstrap.cc:492] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:55.260922 17678 ts_tablet_manager.cc:1403] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.154s	user 0.019s	sys 0.000s
I20260709 13:06:55.256556 17407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:55.262686 17407 raft_consensus.cc:2393] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:06:55.262413 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e84a923caa9c4755bcb78a35143dfbd9" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:55.263303 17632 raft_consensus.cc:2468] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:55.267047 17375 leader_election.cc:304] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:55.270044 17916 raft_consensus.cc:2804] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.270598 17916 raft_consensus.cc:493] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.271029 17916 raft_consensus.cc:3060] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.274142 17678 raft_consensus.cc:359] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.275228 17678 raft_consensus.cc:385] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.275506 17682 tablet_bootstrap.cc:492] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:55.275701 17678 raft_consensus.cc:740] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.276417 17682 ts_tablet_manager.cc:1403] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.466s	user 0.024s	sys 0.012s
I20260709 13:06:55.282354 17681 tablet_bootstrap.cc:654] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.282552 17678 consensus_queue.cc:260] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
W20260709 13:06:55.237967 17596 leader_election.cc:343] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.286864 17780 raft_consensus.cc:493] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.291078 17596 leader_election.cc:304] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:55.291486 17780 raft_consensus.cc:515] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
W20260709 13:06:55.292934 17596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 56631.9us
I20260709 13:06:55.295037 17919 raft_consensus.cc:2749] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.296684 17524 leader_election.cc:304] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:55.299474 17803 leader_election.cc:290] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:55.300544 17780 leader_election.cc:290] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:55.301460 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:55.308656 17485 raft_consensus.cc:2393] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
I20260709 13:06:55.260332 17626 raft_consensus.cc:3060] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.311611 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4b5ba400ad420f89c16a97b71e3b70" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:55.314049 17632 raft_consensus.cc:2468] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:55.315590 17450 leader_election.cc:304] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:55.322240 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4b5ba400ad420f89c16a97b71e3b70" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:55.306131 17682 raft_consensus.cc:359] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.326069 17682 raft_consensus.cc:385] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.326771 17682 raft_consensus.cc:740] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.331388 17918 raft_consensus.cc:2804] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.331997 17918 raft_consensus.cc:493] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.332368 17918 raft_consensus.cc:3060] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.289719 17906 raft_consensus.cc:697] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:06:55.344328 17554 raft_consensus.cc:2468] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
W20260709 13:06:55.248282 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 76008.2us
I20260709 13:06:55.345333 17899 raft_consensus.cc:2749] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:55.346596 17376 leader_election.cc:343] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.327907 17682 consensus_queue.cc:260] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.357261 17678 ts_tablet_manager.cc:1434] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.000s
I20260709 13:06:55.359241 17678 tablet_bootstrap.cc:492] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:55.380961 17682 ts_tablet_manager.cc:1434] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.104s	user 0.007s	sys 0.002s
I20260709 13:06:55.356107 17919 raft_consensus.cc:493] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.342351 17692 raft_consensus.cc:515] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.361363 17749 raft_consensus.cc:515] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.392521 17677 tablet_bootstrap.cc:492] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:55.392791 17692 leader_election.cc:290] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:55.393297 17677 ts_tablet_manager.cc:1403] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.479s	user 0.014s	sys 0.022s
I20260709 13:06:55.400324 17914 raft_consensus.cc:515] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.403239 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:55.348942 17916 raft_consensus.cc:515] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:55.384312 17682 tablet_bootstrap.cc:492] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:55.394415 17749 leader_election.cc:290] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:55.350507 17906 consensus_queue.cc:237] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.450476 17681 tablet_bootstrap.cc:492] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:55.451514 17681 ts_tablet_manager.cc:1403] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.399s	user 0.022s	sys 0.012s
I20260709 13:06:55.419989 17626 raft_consensus.cc:2468] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:55.408099 17919 raft_consensus.cc:515] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:55.459472 17677 raft_consensus.cc:359] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.460220 17677 raft_consensus.cc:385] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.460476 17677 raft_consensus.cc:740] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.462139 17677 consensus_queue.cc:260] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.467653 17678 tablet_bootstrap.cc:654] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.468446 17677 ts_tablet_manager.cc:1434] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.075s	user 0.007s	sys 0.000s
I20260709 13:06:55.475641 17677 tablet_bootstrap.cc:492] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:55.476408 17916 leader_election.cc:290] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:55.478569 17854 raft_consensus.cc:493] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.479547 17854 raft_consensus.cc:515] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.483320 17919 leader_election.cc:290] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:55.483989 17677 tablet_bootstrap.cc:654] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.483826 17854 leader_election.cc:290] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:55.484433 17914 leader_election.cc:290] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:55.505462 17681 raft_consensus.cc:359] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.506086 17682 tablet_bootstrap.cc:654] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.506353 17681 raft_consensus.cc:385] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.507301 17918 raft_consensus.cc:515] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.509949 17681 raft_consensus.cc:740] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.514088 17681 consensus_queue.cc:260] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.516038 17681 ts_tablet_manager.cc:1434] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.000s
I20260709 13:06:55.516753 17678 tablet_bootstrap.cc:492] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:55.517175 17681 tablet_bootstrap.cc:492] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:55.517400 17678 ts_tablet_manager.cc:1403] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.159s	user 0.027s	sys 0.007s
I20260709 13:06:55.518210 17677 tablet_bootstrap.cc:492] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:55.518875 17677 ts_tablet_manager.cc:1403] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.044s	user 0.015s	sys 0.012s
I20260709 13:06:55.521085 17678 raft_consensus.cc:359] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:55.521989 17678 raft_consensus.cc:385] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.522313 17678 raft_consensus.cc:740] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.523630 17918 leader_election.cc:290] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:55.523350 17677 raft_consensus.cc:359] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:55.525558 17681 tablet_bootstrap.cc:654] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.525641 17677 raft_consensus.cc:385] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.526571 17677 raft_consensus.cc:740] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.527176 17678 consensus_queue.cc:260] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:55.528944 17677 consensus_queue.cc:260] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:55.532727 17677 ts_tablet_manager.cc:1434] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:06:55.529918 17678 ts_tablet_manager.cc:1434] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:06:55.536813 17678 tablet_bootstrap.cc:492] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:55.536926 17677 tablet_bootstrap.cc:492] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:55.542582 17681 tablet_bootstrap.cc:492] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:55.543494 17681 ts_tablet_manager.cc:1403] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.027s	user 0.018s	sys 0.006s
I20260709 13:06:55.547474 17681 raft_consensus.cc:359] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.548425 17681 raft_consensus.cc:385] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.548880 17681 raft_consensus.cc:740] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.549959 17681 consensus_queue.cc:260] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.550993 17678 tablet_bootstrap.cc:654] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.552649 17681 ts_tablet_manager.cc:1434] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:06:55.554381 17681 tablet_bootstrap.cc:492] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:55.563094 17677 tablet_bootstrap.cc:654] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.608835 17681 tablet_bootstrap.cc:654] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.625672 17677 tablet_bootstrap.cc:492] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:55.626292 17677 ts_tablet_manager.cc:1403] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.090s	user 0.014s	sys 0.041s
I20260709 13:06:55.628652 17682 tablet_bootstrap.cc:492] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:55.629343 17682 ts_tablet_manager.cc:1403] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.247s	user 0.023s	sys 0.026s
I20260709 13:06:55.635306 17677 raft_consensus.cc:359] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.636840 17677 raft_consensus.cc:385] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.637275 17677 raft_consensus.cc:740] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.638324 17681 tablet_bootstrap.cc:492] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:55.635859 17682 raft_consensus.cc:359] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.638859 17682 raft_consensus.cc:385] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.638980 17678 tablet_bootstrap.cc:492] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:55.639148 17682 raft_consensus.cc:740] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.640110 17682 consensus_queue.cc:260] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.639731 17678 ts_tablet_manager.cc:1403] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.103s	user 0.017s	sys 0.028s
I20260709 13:06:55.642347 17682 ts_tablet_manager.cc:1434] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:06:55.643420 17682 tablet_bootstrap.cc:492] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:55.638384 17677 consensus_queue.cc:260] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.644611 17678 raft_consensus.cc:359] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:55.645396 17678 raft_consensus.cc:385] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.645728 17678 raft_consensus.cc:740] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.646602 17677 ts_tablet_manager.cc:1434] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:06:55.646550 17678 consensus_queue.cc:260] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:55.647475 17681 ts_tablet_manager.cc:1403] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.093s	user 0.016s	sys 0.005s
I20260709 13:06:55.648254 17677 tablet_bootstrap.cc:492] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:55.653826 17682 tablet_bootstrap.cc:654] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.654016 17681 raft_consensus.cc:359] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:55.654866 17681 raft_consensus.cc:385] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.655261 17681 raft_consensus.cc:740] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.656122 17681 consensus_queue.cc:260] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:55.649256 17678 ts_tablet_manager.cc:1434] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:55.659459 17681 ts_tablet_manager.cc:1434] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:06:55.659739 17678 tablet_bootstrap.cc:492] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:55.660977 17681 tablet_bootstrap.cc:492] T 2e4b5ba400ad420f89c16a97b71e3b70 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:55.668260 17914 raft_consensus.cc:493] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.668565 17677 tablet_bootstrap.cc:654] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.668987 17914 raft_consensus.cc:515] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.671432 17681 tablet_bootstrap.cc:654] T 2e4b5ba400ad420f89c16a97b71e3b70 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.672391 17914 leader_election.cc:290] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:55.668300 17678 tablet_bootstrap.cc:654] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.675851 17682 tablet_bootstrap.cc:492] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:55.676483 17682 ts_tablet_manager.cc:1403] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.008s
I20260709 13:06:55.688405 17677 tablet_bootstrap.cc:492] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:55.688150 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81d2393d5664464837df0b9d2a0d25b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:55.689038 17677 ts_tablet_manager.cc:1403] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.015s
I20260709 13:06:55.690090 17482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:55.692639 17677 raft_consensus.cc:359] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.693509 17677 raft_consensus.cc:385] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.693804 17677 raft_consensus.cc:740] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.694530 17677 consensus_queue.cc:260] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.671181 17626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e84a923caa9c4755bcb78a35143dfbd9" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:55.697062 17677 ts_tablet_manager.cc:1434] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:06:55.697338 17626 raft_consensus.cc:3060] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.698155 17677 tablet_bootstrap.cc:492] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:55.704563 17918 raft_consensus.cc:493] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.705287 17918 raft_consensus.cc:515] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:55.706604 17677 tablet_bootstrap.cc:654] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.707720 17918 leader_election.cc:290] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:55.713692 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4b5ba400ad420f89c16a97b71e3b70" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:55.714906 17628 raft_consensus.cc:3060] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.725554 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626dcd42c0f74fccb9c2a42ee7d443ee" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:55.726449 17485 raft_consensus.cc:3060] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.740195 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:55.737766 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:55.741271 17484 raft_consensus.cc:2393] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
I20260709 13:06:55.745761 17682 raft_consensus.cc:359] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:55.746603 17682 raft_consensus.cc:385] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.750056 17682 raft_consensus.cc:740] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.757006 17682 consensus_queue.cc:260] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:55.765861 17450 leader_election.cc:304] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:55.773088 17803 raft_consensus.cc:493] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.774013 17913 raft_consensus.cc:493] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.774729 17867 consensus_queue.cc:1048] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.774858 17913 raft_consensus.cc:515] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.773888 17914 raft_consensus.cc:2804] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.778673 17914 raft_consensus.cc:697] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:55.780385 17408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:55.780189 17914 consensus_queue.cc:237] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:55.773864 17803 raft_consensus.cc:515] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.755900 17916 raft_consensus.cc:493] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.786459 17916 raft_consensus.cc:515] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.786744 17375 leader_election.cc:304] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:55.789173 17934 raft_consensus.cc:2749] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:06:55.866817 17375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 80314.7us
I20260709 13:06:55.877923 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:55.879418 17409 raft_consensus.cc:2393] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:06:55.882571 17485 raft_consensus.cc:2468] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
W20260709 13:06:55.889257 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 104248us
I20260709 13:06:55.890164 17522 leader_election.cc:304] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
W20260709 13:06:55.897410 17450 leader_election.cc:343] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.900486 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:55.901515 17409 raft_consensus.cc:2468] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:55.908525 17955 raft_consensus.cc:493] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.911142 17481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:55.913290 17481 raft_consensus.cc:2468] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:55.914466 17955 raft_consensus.cc:515] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.924420 17450 leader_election.cc:304] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:55.924536 17373 leader_election.cc:304] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:55.927170 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deeb668bf4814b1bb7e01791d6f96914" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:55.927702 17953 raft_consensus.cc:2804] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.928272 17953 raft_consensus.cc:493] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.928649 17953 raft_consensus.cc:3060] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.928809 17934 raft_consensus.cc:2804] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.930334 17934 raft_consensus.cc:493] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.933770 17934 raft_consensus.cc:3060] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.951697 17953 raft_consensus.cc:515] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.957345 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:06:55.959679 17373 leader_election.cc:343] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.955454 17803 leader_election.cc:290] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:55.960300 17957 raft_consensus.cc:2804] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.961112 17934 raft_consensus.cc:515] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:55.941558 17626 raft_consensus.cc:2468] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:55.963080 17955 leader_election.cc:290] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:55.963883 17955 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:06:55.964493 17405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:55.964427 17955 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:55.965404 17405 raft_consensus.cc:2393] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:06:55.967231 17955 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:55.970480 17934 leader_election.cc:290] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:55.971083 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:55.972081 17484 raft_consensus.cc:2393] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
I20260709 13:06:55.975107 17953 leader_election.cc:290] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:55.976454 17957 raft_consensus.cc:697] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:06:55.980304 17957 consensus_queue.cc:237] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.981838 17681 tablet_bootstrap.cc:492] T 2e4b5ba400ad420f89c16a97b71e3b70 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:55.982537 17681 ts_tablet_manager.cc:1403] T 2e4b5ba400ad420f89c16a97b71e3b70 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.322s	user 0.088s	sys 0.083s
I20260709 13:06:55.986240 17681 raft_consensus.cc:359] T 2e4b5ba400ad420f89c16a97b71e3b70 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.988236 17681 raft_consensus.cc:385] T 2e4b5ba400ad420f89c16a97b71e3b70 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.988621 17681 raft_consensus.cc:740] T 2e4b5ba400ad420f89c16a97b71e3b70 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.989392 17681 consensus_queue.cc:260] T 2e4b5ba400ad420f89c16a97b71e3b70 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:55.991983 17681 ts_tablet_manager.cc:1434] T 2e4b5ba400ad420f89c16a97b71e3b70 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:06:55.993287 17681 tablet_bootstrap.cc:492] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:56.002283 17681 tablet_bootstrap.cc:654] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.008945 17681 tablet_bootstrap.cc:492] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:56.009591 17681 ts_tablet_manager.cc:1403] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.000s
I20260709 13:06:56.013933 17681 raft_consensus.cc:359] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:56.014671 17681 raft_consensus.cc:385] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.014995 17681 raft_consensus.cc:740] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.015720 17681 consensus_queue.cc:260] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:56.017333 17681 ts_tablet_manager.cc:1434] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:56.017937 17913 leader_election.cc:290] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:56.018240 17681 tablet_bootstrap.cc:492] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:55.789484 17916 leader_election.cc:290] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:56.025507 17681 tablet_bootstrap.cc:654] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.028141 17957 raft_consensus.cc:493] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.028823 17957 raft_consensus.cc:515] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:56.028124 17405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:56.032481 17681 tablet_bootstrap.cc:492] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:56.033552 17957 leader_election.cc:290] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:56.034695 17681 ts_tablet_manager.cc:1403] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.001s
I20260709 13:06:56.035006 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a74eee34cc514b79b981a6ef6c45d7e2" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:56.036130 17484 raft_consensus.cc:2393] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:06:56.038797 17955 raft_consensus.cc:493] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.038657 17681 raft_consensus.cc:359] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:56.039922 17681 raft_consensus.cc:385] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.039743 17955 raft_consensus.cc:515] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:56.040290 17681 raft_consensus.cc:740] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.041113 17681 consensus_queue.cc:260] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:56.043555 17681 ts_tablet_manager.cc:1434] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:06:56.042279 17955 leader_election.cc:290] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:56.045135 17681 tablet_bootstrap.cc:492] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:56.048492 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:56.049397 17557 raft_consensus.cc:3060] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.055096 17678 tablet_bootstrap.cc:492] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:56.056048 17678 ts_tablet_manager.cc:1403] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.397s	user 0.076s	sys 0.076s
I20260709 13:06:56.062158 17555 raft_consensus.cc:3060] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.062101 17678 raft_consensus.cc:359] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:56.063411 17678 raft_consensus.cc:385] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.063879 17678 raft_consensus.cc:740] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.065004 17678 consensus_queue.cc:260] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:56.069787 17681 tablet_bootstrap.cc:654] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.071085 17553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4b5ba400ad420f89c16a97b71e3b70" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:56.071722 17964 raft_consensus.cc:493] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:06:56.071918 17553 raft_consensus.cc:3060] T 2e4b5ba400ad420f89c16a97b71e3b70 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.072422 17964 raft_consensus.cc:515] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:56.073323 17932 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:06:56.073977 17932 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:56.076272 17932 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:56.077008 17964 leader_election.cc:290] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:56.078866 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deeb668bf4814b1bb7e01791d6f96914" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:56.079398 17627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626dcd42c0f74fccb9c2a42ee7d443ee" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:56.080545 17627 raft_consensus.cc:3060] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.081485 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:06:56.082510 17596 leader_election.cc:343] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.083109 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:06:56.083449 17596 leader_election.cc:343] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.083874 17631 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:56.079943 17633 raft_consensus.cc:2393] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
I20260709 13:06:56.089957 17628 raft_consensus.cc:2468] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
W20260709 13:06:56.098510 17451 leader_election.cc:343] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.115595 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:56.117256 17678 ts_tablet_manager.cc:1434] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.061s	user 0.011s	sys 0.001s
I20260709 13:06:56.120612 17678 tablet_bootstrap.cc:492] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:56.135936 17681 tablet_bootstrap.cc:492] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:56.136559 17681 ts_tablet_manager.cc:1403] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.092s	user 0.034s	sys 0.017s
W20260709 13:06:56.138952 17355 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:56.139665 17964 raft_consensus.cc:493] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:06:56.140349 17964 raft_consensus.cc:515] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:56.142553 17964 leader_election.cc:290] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:56.121093 17906 raft_consensus.cc:493] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.121142 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:56.121220 17552 raft_consensus.cc:2468] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:56.149001 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81d2393d5664464837df0b9d2a0d25b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:56.153180 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:56.134814 17627 raft_consensus.cc:2468] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:56.160483 17633 raft_consensus.cc:2468] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:56.169735 17375 leader_election.cc:304] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:56.171207 17626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:56.150352 17906 raft_consensus.cc:515] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:56.171633 17871 raft_consensus.cc:2804] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:56.160863 17966 raft_consensus.cc:493] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.183670 17871 raft_consensus.cc:697] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:06:56.184291 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:56.185081 17554 raft_consensus.cc:3060] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.184964 17918 consensus_queue.cc:1048] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:56.186561 17450 leader_election.cc:304] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:56.186228 17306 catalog_manager.cc:5697] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:56.187686 17450 leader_election.cc:304] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:56.188458 17913 raft_consensus.cc:2804] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:56.188951 17913 raft_consensus.cc:697] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:56.189265 17681 raft_consensus.cc:359] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:56.193466 17681 raft_consensus.cc:385] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.192935 17953 raft_consensus.cc:2804] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:56.195222 17953 raft_consensus.cc:493] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.195633 17953 raft_consensus.cc:3060] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.189826 17913 consensus_queue.cc:237] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:56.193929 17681 raft_consensus.cc:740] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
W20260709 13:06:56.162184 17598 leader_election.cc:343] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.161370 17557 raft_consensus.cc:2468] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:56.203529 17681 consensus_queue.cc:260] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:56.208112 17916 consensus_queue.cc:1048] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:56.212322 17677 tablet_bootstrap.cc:492] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:56.213107 17677 ts_tablet_manager.cc:1403] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.515s	user 0.026s	sys 0.032s
I20260709 13:06:56.215186 17678 tablet_bootstrap.cc:654] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.218652 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e84a923caa9c4755bcb78a35143dfbd9" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:56.190843 17626 raft_consensus.cc:2468] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:56.224882 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a74eee34cc514b79b981a6ef6c45d7e2" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:56.191663 17966 raft_consensus.cc:515] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:56.202260 17553 raft_consensus.cc:2468] T 2e4b5ba400ad420f89c16a97b71e3b70 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
W20260709 13:06:56.233598 17451 leader_election.cc:343] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.235983 17405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e84a923caa9c4755bcb78a35143dfbd9" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:56.237205 17964 raft_consensus.cc:493] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.237615 17408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61235f863ca44ed3b6c4715dfd68e223" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:56.237797 17964 raft_consensus.cc:515] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:56.240037 17964 leader_election.cc:290] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:56.235103 17553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:56.248631 17524 leader_election.cc:304] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:56.219554 17552 raft_consensus.cc:2468] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:56.254144 17681 ts_tablet_manager.cc:1434] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.117s	user 0.007s	sys 0.002s
I20260709 13:06:56.259555 17524 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:56.260782 17681 tablet_bootstrap.cc:492] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:56.255694 17677 raft_consensus.cc:359] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:56.262212 17677 raft_consensus.cc:385] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.262652 17677 raft_consensus.cc:740] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.263620 17677 consensus_queue.cc:260] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:56.184732 17871 consensus_queue.cc:237] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:56.215672 17598 leader_election.cc:304] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:56.273417 17955 raft_consensus.cc:2804] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:56.274914 17906 leader_election.cc:290] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:56.275229 17955 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:06:56.258379 17964 raft_consensus.cc:3060] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.275760 17955 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:56.277577 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:56.279074 17677 ts_tablet_manager.cc:1434] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 13:06:56.267417 17524 leader_election.cc:304] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:56.280413 17677 tablet_bootstrap.cc:492] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:56.285827 17957 raft_consensus.cc:493] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:06:56.289417 17971 raft_consensus.cc:493] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.290176 17899 raft_consensus.cc:2749] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.290680 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e84a923caa9c4755bcb78a35143dfbd9" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:56.251195 17966 leader_election.cc:290] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:56.265255 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:56.294956 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e84a923caa9c4755bcb78a35143dfbd9" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:56.296648 17969 raft_consensus.cc:2749] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.298498 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8bf7f6c34a4edfb28a4914ad4ce49e" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:56.300761 17551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:56.307498 17628 raft_consensus.cc:2393] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:06:56.308736 17557 raft_consensus.cc:2393] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
W20260709 13:06:56.312668 17599 leader_election.cc:343] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.313593 17599 leader_election.cc:304] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:56.314777 17969 raft_consensus.cc:2749] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.315089 17682 ts_tablet_manager.cc:1434] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.636s	user 0.015s	sys 0.000s
W20260709 13:06:56.315523 17682 ts_tablet_manager.cc:1467] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c: Tablet startup took 636ms
I20260709 13:06:56.286550 17957 raft_consensus.cc:515] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:56.321080 17450 leader_election.cc:304] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:56.329979 17914 raft_consensus.cc:2804] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:56.330564 17914 raft_consensus.cc:493] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.330976 17914 raft_consensus.cc:3060] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:56.331056 17973 log.cc:927] Time spent T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:06:56.334092 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:56.341224 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:56.342185 17625 raft_consensus.cc:2468] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
W20260709 13:06:56.343024 17598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 181695us
I20260709 13:06:56.319903 17558 raft_consensus.cc:3060] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.316152 17971 raft_consensus.cc:515] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:56.352331 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:56.360867 17626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61235f863ca44ed3b6c4715dfd68e223" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:06:56.316032 17682 ts_tablet_manager.cc:1469] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c: Trace:
0709 13:06:50.756940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.757158 (+   218us) service_pool.cc:224] Handling call
0709 13:06:50.759513 (+  2355us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:50.759638 (+   125us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:50.944924 (+185286us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:51.092379 (+147455us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:51.092934 (+   555us) inbound_call.cc:177] Queueing success response
0709 13:06:55.643120 (+4550186us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:06:55.679345 (+ 36225us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:06:55.680222 (+   877us) tablet_replica.cc:261] Starting instrumentation
0709 13:06:55.681148 (+   926us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":70573,"raft.queue_time_us":8907,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":16718,"tablet-open.queue_time_us":4550311}
I20260709 13:06:56.369593 17682 tablet_bootstrap.cc:492] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:56.370736 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8bf7f6c34a4edfb28a4914ad4ce49e" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:06:56.369380 17598 leader_election.cc:343] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.384155 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:56.386029 17957 leader_election.cc:290] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:56.388048 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a74eee34cc514b79b981a6ef6c45d7e2" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
W20260709 13:06:56.390910 17596 leader_election.cc:343] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.391574 17596 leader_election.cc:304] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:56.396514 17919 raft_consensus.cc:2749] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.398483 17681 tablet_bootstrap.cc:654] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.378679 17914 raft_consensus.cc:515] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
W20260709 13:06:56.408456 17451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 73456.1us
W20260709 13:06:56.407616 17375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 69537.9us
I20260709 13:06:56.411399 17913 consensus_queue.cc:1048] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 13:06:56.415005 17871 consensus_queue.cc:1048] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
W20260709 13:06:56.457136 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 189889us
I20260709 13:06:56.421633 17780 consensus_queue.cc:1048] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 13:06:56.430487 17953 raft_consensus.cc:515] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:56.463411 17971 leader_election.cc:290] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:56.464432 17914 leader_election.cc:290] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:56.467032 17554 raft_consensus.cc:2468] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:56.462296 17524 leader_election.cc:304] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:56.484720 17558 raft_consensus.cc:2468] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:56.486063 17407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:56.486413 17481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:56.489178 17485 raft_consensus.cc:2393] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
W20260709 13:06:56.489645 17598 leader_election.cc:343] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.490216 17953 leader_election.cc:290] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:56.492841 17375 leader_election.cc:304] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:56.495515 17928 raft_consensus.cc:2804] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:56.496064 17928 raft_consensus.cc:493] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.496474 17928 raft_consensus.cc:3060] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.501109 17980 raft_consensus.cc:493] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:06:56.502260 17980 raft_consensus.cc:515] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:56.508405 17964 raft_consensus.cc:2749] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.510244 17978 raft_consensus.cc:2749] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.512145 17955 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:06:56.515586 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 101440us
I20260709 13:06:56.529834 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d047496a466464ba1777cf63c5c84b0" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:56.515553 17677 tablet_bootstrap.cc:654] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:56.514034 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 51837.4us
I20260709 13:06:56.532938 17524 leader_election.cc:304] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:56.536844 17682 tablet_bootstrap.cc:654] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.535061 17899 raft_consensus.cc:2749] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:56.534104 17957 consensus_queue.cc:1048] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:06:56.545202 17408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:56.546490 17955 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:56.532421 17980 leader_election.cc:290] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:56.551702 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:56.552611 17631 raft_consensus.cc:3060] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.565497 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:56.583552 17681 tablet_bootstrap.cc:492] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:56.584295 17681 ts_tablet_manager.cc:1403] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.324s	user 0.027s	sys 0.005s
I20260709 13:06:56.586100 17678 tablet_bootstrap.cc:492] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:56.586894 17678 ts_tablet_manager.cc:1403] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.467s	user 0.014s	sys 0.013s
I20260709 13:06:56.600184 17451 leader_election.cc:304] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:56.602893 17829 raft_consensus.cc:2804] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:56.603575 17829 raft_consensus.cc:697] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:56.604637 17829 consensus_queue.cc:237] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
W20260709 13:06:56.605916 17376 leader_election.cc:343] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.604894 17626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:56.609779 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:56.617532 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:56.619524 17555 raft_consensus.cc:1240] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Rejecting Update request from peer 337bbc857b9b4e329c817b81bfb0971c for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:56.622815 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:56.624189 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:56.589265 17631 raft_consensus.cc:2468] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:56.630246 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:56.621992 17678 raft_consensus.cc:359] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:56.635062 17678 raft_consensus.cc:385] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.635563 17678 raft_consensus.cc:740] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.636575 17678 consensus_queue.cc:260] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:56.639786 17524 leader_election.cc:304] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:56.644344 17524 leader_election.cc:304] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:56.610744 17628 raft_consensus.cc:3060] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.648450 17899 raft_consensus.cc:2749] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:56.615629 17955 raft_consensus.cc:493] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.651404 17955 raft_consensus.cc:515] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:56.653769 17955 leader_election.cc:290] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:56.654455 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:56.657429 17450 leader_election.cc:304] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:56.659350 17987 raft_consensus.cc:2804] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:56.659916 17987 raft_consensus.cc:697] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:56.660317 17406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d047496a466464ba1777cf63c5c84b0" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:56.660760 17980 raft_consensus.cc:2749] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:56.660862 17987 consensus_queue.cc:237] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:56.662010 17913 raft_consensus.cc:493] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.606585 17376 leader_election.cc:304] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:56.646853 17451 leader_election.cc:304] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:56.663471 17803 raft_consensus.cc:2749] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.683295 17524 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:56.684712 17899 raft_consensus.cc:2749] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:56.685366 17524 leader_election.cc:304] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:56.662644 17913 raft_consensus.cc:515] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:56.690337 17678 ts_tablet_manager.cc:1434] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.103s	user 0.009s	sys 0.001s
I20260709 13:06:56.692672 17678 tablet_bootstrap.cc:492] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
W20260709 13:06:56.663302 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 57435.6us
I20260709 13:06:56.690869 17899 raft_consensus.cc:2749] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:06:56.694247 17451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 59822.7us
I20260709 13:06:56.657514 17599 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:56.715663 17677 tablet_bootstrap.cc:492] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:56.716318 17677 ts_tablet_manager.cc:1403] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.436s	user 0.019s	sys 0.013s
I20260709 13:06:56.724889 17954 raft_consensus.cc:2804] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:56.725546 17954 raft_consensus.cc:697] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:56.726577 17954 consensus_queue.cc:237] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:56.728384 17404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032dc48c2ae44160841c456abd340e40" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:56.722538 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032dc48c2ae44160841c456abd340e40" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:56.731129 17982 consensus_queue.cc:1048] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.050s
I20260709 13:06:56.731784 17633 raft_consensus.cc:2468] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:06:56.731943 17376 leader_election.cc:304] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:56.691771 17928 raft_consensus.cc:515] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:56.734899 17524 leader_election.cc:304] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:56.736146 17803 raft_consensus.cc:2749] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.736213 17780 consensus_queue.cc:1048] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:06:56.739233 17934 consensus_queue.cc:1048] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
W20260709 13:06:56.746305 17524 leader_election.cc:343] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.723881 17971 raft_consensus.cc:2749] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:56.757442 17681 raft_consensus.cc:359] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:56.740265 17982 raft_consensus.cc:2804] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:56.760146 17982 raft_consensus.cc:493] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.760572 17982 raft_consensus.cc:3060] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.758911 17681 raft_consensus.cc:385] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.768900 17628 raft_consensus.cc:2468] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:56.773555 17913 leader_election.cc:290] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:56.776332 17928 leader_election.cc:290] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:56.776620 17677 raft_consensus.cc:359] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:56.779109 17677 raft_consensus.cc:385] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.763703 17681 raft_consensus.cc:740] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.780413 17681 consensus_queue.cc:260] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:56.780588 17682 tablet_bootstrap.cc:492] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:56.779526 17677 raft_consensus.cc:740] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.782256 17682 ts_tablet_manager.cc:1403] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.413s	user 0.023s	sys 0.008s
W20260709 13:06:56.738828 17599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 81432us
I20260709 13:06:56.783557 17971 raft_consensus.cc:493] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.784631 17681 ts_tablet_manager.cc:1434] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.200s	user 0.008s	sys 0.003s
I20260709 13:06:56.784247 17971 raft_consensus.cc:515] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:56.787516 17681 tablet_bootstrap.cc:492] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:56.787935 17971 leader_election.cc:290] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:56.794296 17550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34919e11221f465cbcef1151a1c5d4b0" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:56.796958 17682 raft_consensus.cc:359] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:56.797946 17682 raft_consensus.cc:385] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.798463 17682 raft_consensus.cc:740] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.799283 17682 consensus_queue.cc:260] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:06:56.803650 17599 leader_election.cc:343] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.806893 17599 leader_election.cc:304] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
W20260709 13:06:56.869109 17580 debug-util.cc:398] Leaking SignalData structure 0x7b0800275e80 after lost signal to thread 7529
I20260709 13:06:56.870028 17408 raft_consensus.cc:2393] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:06:56.870594 17404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34919e11221f465cbcef1151a1c5d4b0" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
W20260709 13:06:56.871380 17451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 160392us
W20260709 13:06:56.960831 17580 debug-util.cc:398] Leaking SignalData structure 0x7b080016aa20 after lost signal to thread 17342
I20260709 13:06:56.810021 17982 raft_consensus.cc:515] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:06:56.869110 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 127970us
I20260709 13:06:56.881377 17899 raft_consensus.cc:493] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.964403 17899 raft_consensus.cc:515] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:56.924779 17677 consensus_queue.cc:260] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:56.932741 17406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:56.942983 17681 tablet_bootstrap.cc:654] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.943929 17678 tablet_bootstrap.cc:654] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.957005 17682 ts_tablet_manager.cc:1434] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.174s	user 0.008s	sys 0.000s
I20260709 13:06:56.964834 17376 leader_election.cc:304] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:56.888401 17407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
W20260709 13:06:56.978008 17580 debug-util.cc:398] Leaking SignalData structure 0x7b0800195680 after lost signal to thread 17434
I20260709 13:06:56.980965 17899 leader_election.cc:290] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:56.990188 17524 leader_election.cc:304] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:56.993564 17404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:56.999626 17407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032dc48c2ae44160841c456abd340e40" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
W20260709 13:06:56.982966 17598 leader_election.cc:343] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.985332 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032dc48c2ae44160841c456abd340e40" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
W20260709 13:06:56.981410 17580 debug-util.cc:398] Leaking SignalData structure 0x7b08002677e0 after lost signal to thread 17509
I20260709 13:06:56.993748 17406 raft_consensus.cc:2468] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:57.003014 17633 raft_consensus.cc:3060] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.005048 17969 raft_consensus.cc:2749] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:57.005438 17580 debug-util.cc:398] Leaking SignalData structure 0x7b080026d6e0 after lost signal to thread 17583
W20260709 13:06:57.006031 17580 debug-util.cc:398] Leaking SignalData structure 0x7b080029b340 after lost signal to thread 17584
I20260709 13:06:56.989531 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:56.988488 17803 raft_consensus.cc:2749] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.007432 17485 raft_consensus.cc:2468] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
W20260709 13:06:57.007876 17599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 205146us
I20260709 13:06:57.010165 17522 leader_election.cc:304] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:57.011576 17407 raft_consensus.cc:3060] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.013011 17955 raft_consensus.cc:2804] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.013615 17677 ts_tablet_manager.cc:1434] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.297s	user 0.007s	sys 0.014s
I20260709 13:06:57.013883 17955 raft_consensus.cc:493] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.965723 17982 leader_election.cc:290] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:57.000928 17899 raft_consensus.cc:2749] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:57.021906 17955 raft_consensus.cc:3060] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.027254 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:57.028801 17485 raft_consensus.cc:2393] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
W20260709 13:06:56.962354 17451 leader_election.cc:343] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:57.042639 17451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 80515.6us
W20260709 13:06:57.009313 17599 leader_election.cc:343] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.044739 17599 leader_election.cc:304] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:57.045233 17633 raft_consensus.cc:2468] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:57.045919 17969 raft_consensus.cc:2749] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.047221 17524 leader_election.cc:304] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:57.051535 17899 raft_consensus.cc:2804] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:57.051857 17928 consensus_queue.cc:1048] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:57.054491 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 61201us
I20260709 13:06:57.057919 17407 raft_consensus.cc:2468] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:57.059615 17899 raft_consensus.cc:697] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:06:57.062903 17969 raft_consensus.cc:493] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.083048 17969 raft_consensus.cc:515] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:57.090653 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb6246353144b0a8b9648ac089a1da6" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:06:57.094205 17596 leader_election.cc:343] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.094004 17934 consensus_queue.cc:1059] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:57.094971 17682 tablet_bootstrap.cc:492] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:57.095268 17899 consensus_queue.cc:237] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:57.096133 17803 raft_consensus.cc:3055] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:57.096661 17803 raft_consensus.cc:740] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:06:57.098284 17803 consensus_queue.cc:260] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:57.100230 17803 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.110369 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:57.110747 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:57.112077 17633 raft_consensus.cc:2393] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
I20260709 13:06:57.087724 17969 leader_election.cc:290] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:57.135140 17681 tablet_bootstrap.cc:492] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:57.136126 17681 ts_tablet_manager.cc:1403] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.349s	user 0.014s	sys 0.017s
I20260709 13:06:57.140470 17682 tablet_bootstrap.cc:654] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.140648 17955 raft_consensus.cc:515] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:57.143934 17552 raft_consensus.cc:2393] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 1: Already voted for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in this term.
I20260709 13:06:57.153580 17450 leader_election.cc:304] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:57.155278 17987 raft_consensus.cc:2804] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:57.156044 17987 raft_consensus.cc:697] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
W20260709 13:06:57.156980 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 87077.1us
I20260709 13:06:57.163307 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:57.164294 17409 raft_consensus.cc:3060] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.163712 17987 consensus_queue.cc:237] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:57.167325 17677 tablet_bootstrap.cc:492] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:57.176909 17934 consensus_queue.cc:1048] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:57.192879 17871 consensus_queue.cc:1048] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:57.195408 17678 tablet_bootstrap.cc:492] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:57.204191 17678 ts_tablet_manager.cc:1403] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.512s	user 0.027s	sys 0.011s
I20260709 13:06:57.221751 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:57.222602 17485 raft_consensus.cc:3060] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.218245 17955 leader_election.cc:290] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:57.234864 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb6246353144b0a8b9648ac089a1da6" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:57.243673 17307 catalog_manager.cc:5697] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:57.247763 17303 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40062 (request call id 50) took 1222 ms. Trace:
I20260709 13:06:57.248914 17303 rpcz_store.cc:276] 0709 13:06:56.024913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.025118 (+   205us) service_pool.cc:224] Handling call
0709 13:06:57.247441 (+1222323us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102174,"reported_tablets":4}
W20260709 13:06:57.255425 17599 leader_election.cc:343] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.256341 17599 leader_election.cc:304] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:57.257748 17981 raft_consensus.cc:2749] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:57.274636 17580 debug-util.cc:398] Leaking SignalData structure 0x7b0800042800 after lost signal to thread 17657
W20260709 13:06:57.275915 17580 debug-util.cc:398] Leaking SignalData structure 0x7b08002484a0 after lost signal to thread 17769
W20260709 13:06:57.276479 17580 debug-util.cc:398] Leaking SignalData structure 0x7b080019bd20 after lost signal to thread 17834
W20260709 13:06:57.277058 17580 debug-util.cc:398] Leaking SignalData structure 0x7b08001fb4a0 after lost signal to thread 17877
I20260709 13:06:57.254094 17307 catalog_manager.cc:5697] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:57.284152 17485 raft_consensus.cc:2468] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:57.291855 17678 raft_consensus.cc:359] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:57.293322 17678 raft_consensus.cc:385] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.293870 17678 raft_consensus.cc:740] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
W20260709 13:06:57.294878 17451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 76888.6us
I20260709 13:06:57.299396 17914 raft_consensus.cc:493] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.300467 17522 leader_election.cc:304] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:57.320115 17373 leader_election.cc:304] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:57.337466 17682 tablet_bootstrap.cc:492] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:57.338421 17682 ts_tablet_manager.cc:1403] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.244s	user 0.060s	sys 0.064s
I20260709 13:06:57.290107 17681 raft_consensus.cc:359] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:57.341866 17681 raft_consensus.cc:385] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.342361 17681 raft_consensus.cc:740] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.353377 17954 consensus_queue.cc:1048] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:06:57.351449 17681 consensus_queue.cc:260] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:57.346444 17982 raft_consensus.cc:2804] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:57.350427 17678 consensus_queue.cc:260] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:57.350840 17409 raft_consensus.cc:2468] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:57.360936 17928 raft_consensus.cc:2749] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.344529 17914 raft_consensus.cc:515] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:06:57.371711 17373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 51882.6us
I20260709 13:06:57.373375 17914 leader_election.cc:290] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:57.364584 17982 raft_consensus.cc:697] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:06:57.384513 17681 ts_tablet_manager.cc:1434] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.248s	user 0.014s	sys 0.000s
I20260709 13:06:57.386551 17982 consensus_queue.cc:237] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:57.388365 17678 ts_tablet_manager.cc:1434] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.184s	user 0.010s	sys 0.001s
I20260709 13:06:57.390206 17678 tablet_bootstrap.cc:492] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:57.365083 17954 consensus_queue.cc:1048] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:06:57.399328 17682 raft_consensus.cc:359] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:57.400236 17682 raft_consensus.cc:385] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.400527 17682 raft_consensus.cc:740] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.407122 17554 raft_consensus.cc:3060] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.409854 17914 consensus_queue.cc:1048] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:57.386478 17681 tablet_bootstrap.cc:492] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
W20260709 13:06:57.420408 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 83479.4us
I20260709 13:06:57.424835 17682 consensus_queue.cc:260] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:57.432737 17450 leader_election.cc:304] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:57.440001 17682 ts_tablet_manager.cc:1434] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.101s	user 0.009s	sys 0.002s
W20260709 13:06:57.448280 17355 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:57.444321 17954 raft_consensus.cc:2764] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:06:57.451944 17928 raft_consensus.cc:493] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.452983 17928 raft_consensus.cc:515] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:57.458827 17677 tablet_bootstrap.cc:654] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.462101 17928 leader_election.cc:290] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:57.470351 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:57.471496 17631 raft_consensus.cc:2468] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:57.471695 17981 raft_consensus.cc:493] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.472313 17981 raft_consensus.cc:515] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:06:57.473152 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 49800.7us
I20260709 13:06:57.473294 17375 leader_election.cc:304] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:57.476454 17871 raft_consensus.cc:2804] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.476763 17681 tablet_bootstrap.cc:654] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.477017 17871 raft_consensus.cc:493] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.477497 17871 raft_consensus.cc:3060] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.482674 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05345b561dfb497c8f1e081b05c1bb5d" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
W20260709 13:06:57.485111 17598 leader_election.cc:343] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.485615 17682 tablet_bootstrap.cc:492] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:57.487413 17981 leader_election.cc:290] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:57.492005 17678 tablet_bootstrap.cc:654] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.500948 17871 raft_consensus.cc:515] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:57.504560 17550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05345b561dfb497c8f1e081b05c1bb5d" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:57.511936 17982 consensus_queue.cc:1048] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:57.513509 17954 consensus_queue.cc:1048] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:57.515858 17677 tablet_bootstrap.cc:492] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:57.516639 17677 ts_tablet_manager.cc:1403] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.360s	user 0.009s	sys 0.013s
I20260709 13:06:57.520403 17677 raft_consensus.cc:359] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:57.521402 17677 raft_consensus.cc:385] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.521759 17677 raft_consensus.cc:740] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.522766 17677 consensus_queue.cc:260] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:57.524742 17681 tablet_bootstrap.cc:492] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:57.525385 17681 ts_tablet_manager.cc:1403] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.139s	user 0.014s	sys 0.001s
I20260709 13:06:57.523780 17871 leader_election.cc:290] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:57.528312 17677 ts_tablet_manager.cc:1434] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:06:57.531955 17677 tablet_bootstrap.cc:492] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:57.529160 17681 raft_consensus.cc:359] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:57.537416 17681 raft_consensus.cc:385] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.537781 17681 raft_consensus.cc:740] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.534643 17682 tablet_bootstrap.cc:654] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.539026 17681 consensus_queue.cc:260] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:06:57.540149 17599 leader_election.cc:343] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.540771 17599 leader_election.cc:304] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:57.546365 17957 consensus_queue.cc:1048] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:06:57.553840 17981 raft_consensus.cc:2749] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.553844 17481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:57.557407 17681 ts_tablet_manager.cc:1434] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.001s
I20260709 13:06:57.558583 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:57.559244 17681 tablet_bootstrap.cc:492] T a81d2393d5664464837df0b9d2a0d25b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:57.559784 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:57.561580 17633 raft_consensus.cc:3060] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.565519 17677 tablet_bootstrap.cc:654] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.621697 17633 raft_consensus.cc:2468] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:57.623075 17404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:57.630158 17306 catalog_manager.cc:5697] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:57.612404 17899 raft_consensus.cc:493] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.649473 17954 consensus_queue.cc:1048] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:57.651039 17307 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40054 (request call id 52) took 1270 ms. Trace:
I20260709 13:06:57.664053 17682 tablet_bootstrap.cc:492] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:57.664921 17682 ts_tablet_manager.cc:1403] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.180s	user 0.021s	sys 0.019s
I20260709 13:06:57.668336 17306 catalog_manager.cc:5697] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:57.660399 17307 rpcz_store.cc:276] 0709 13:06:56.380702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.381008 (+   306us) service_pool.cc:224] Handling call
0709 13:06:57.650966 (+1269958us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:57.478539 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:57.479346 (+   807us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:57.479413 (+    67us) write_op.cc:435] Acquired schema lock
0709 13:06:57.479455 (+    42us) tablet.cc:662] Decoding operations
0709 13:06:57.480044 (+   589us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:57.480156 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:57.480189 (+    33us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:06:57.480877 (+   688us) tablet.cc:701] Row locks acquired
0709 13:06:57.480921 (+    44us) write_op.cc:260] PREPARE: finished
0709 13:06:57.481107 (+   186us) write_op.cc:270] Start()
0709 13:06:57.481277 (+   170us) write_op.cc:276] Timestamp: P: 1783602417481051 usec, L: 0
0709 13:06:57.481321 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:06:57.482192 (+   871us) log.cc:844] Serialized 922 byte log entry
0709 13:06:57.604727 (+122535us) op_driver.cc:464] REPLICATION: finished
0709 13:06:57.607069 (+  2342us) write_op.cc:301] APPLY: starting
0709 13:06:57.607180 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:57.607350 (+   170us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:57.607526 (+   176us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:57.619086 (+ 11560us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:57.619433 (+   347us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:06:57.619500 (+    67us) write_op.cc:312] APPLY: finished
0709 13:06:57.619918 (+   418us) log.cc:844] Serialized 28 byte log entry
0709 13:06:57.620133 (+   215us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:57.620249 (+   116us) write_op.cc:454] Released schema lock
0709 13:06:57.620331 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:57.620959 (+   628us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:57.621003 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":79238,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":1984,"apply.run_cpu_time_us":3072,"apply.run_wall_time_us":13850,"num_ops":2,"prepare.queue_time_us":143932,"prepare.run_cpu_time_us":4185,"prepare.run_wall_time_us":65686,"replication_time_us":123147,"thread_start_us":82758,"threads_started":2}]]}
I20260709 13:06:57.686125 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:57.687284 17625 raft_consensus.cc:2468] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:57.716385 17682 raft_consensus.cc:359] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:57.717612 17682 raft_consensus.cc:385] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.717993 17682 raft_consensus.cc:740] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.662382 17899 raft_consensus.cc:515] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:57.727146 17678 tablet_bootstrap.cc:492] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:57.729822 17678 ts_tablet_manager.cc:1403] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.340s	user 0.015s	sys 0.045s
I20260709 13:06:57.728602 17682 consensus_queue.cc:260] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:57.700932 17954 consensus_queue.cc:1048] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:06:57.742120 17682 ts_tablet_manager.cc:1434] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.077s	user 0.011s	sys 0.000s
I20260709 13:06:57.704638 17375 leader_election.cc:304] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:57.789345 17681 tablet_bootstrap.cc:654] T a81d2393d5664464837df0b9d2a0d25b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.797319 17677 tablet_bootstrap.cc:492] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:57.798048 17677 ts_tablet_manager.cc:1403] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.266s	user 0.016s	sys 0.038s
I20260709 13:06:57.802632 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:57.807608 17682 tablet_bootstrap.cc:492] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:57.798089 17899 leader_election.cc:290] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:57.816013 18041 raft_consensus.cc:2804] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:57.816859 18041 raft_consensus.cc:697] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
W20260709 13:06:57.821293 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 118336us
I20260709 13:06:57.823761 17450 leader_election.cc:304] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
W20260709 13:06:57.781448 17451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 66835.4us
W20260709 13:06:57.812050 17375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 107544us
I20260709 13:06:57.818034 18041 consensus_queue.cc:237] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:57.870193 17677 raft_consensus.cc:359] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:57.871416 17677 raft_consensus.cc:385] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.871982 17677 raft_consensus.cc:740] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.844791 17829 raft_consensus.cc:2804] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.842834 17981 raft_consensus.cc:493] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.889082 17677 consensus_queue.cc:260] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:57.875818 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:06:57.896174 17450 leader_election.cc:343] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.881065 17829 raft_consensus.cc:493] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.845012 18037 consensus_queue.cc:1048] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.067s
I20260709 13:06:57.899953 17829 raft_consensus.cc:3060] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.909688 17677 ts_tablet_manager.cc:1434] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.111s	user 0.013s	sys 0.000s
I20260709 13:06:57.902216 17633 raft_consensus.cc:2468] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:57.900130 17981 raft_consensus.cc:515] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:57.941219 17678 raft_consensus.cc:359] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:57.942111 17678 raft_consensus.cc:385] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.942376 17678 raft_consensus.cc:740] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
W20260709 13:06:57.938912 17522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 113744us
W20260709 13:06:57.953266 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 97490.6us
I20260709 13:06:57.943091 17957 consensus_queue.cc:1048] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:57.956513 17677 tablet_bootstrap.cc:492] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:57.959344 18038 raft_consensus.cc:493] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.960170 17981 leader_election.cc:290] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:57.960104 18038 raft_consensus.cc:515] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:57.981520 17678 consensus_queue.cc:260] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:57.953761 18037 raft_consensus.cc:493] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.987136 17682 tablet_bootstrap.cc:654] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:57.990949 18044 log.cc:927] Time spent T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:06:57.991765 17678 ts_tablet_manager.cc:1434] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.262s	user 0.009s	sys 0.001s
I20260709 13:06:57.993295 17678 tablet_bootstrap.cc:492] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:57.998920 18038 leader_election.cc:290] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:57.991044 18037 raft_consensus.cc:515] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:58.004565 17677 tablet_bootstrap.cc:654] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.005945 17524 leader_election.cc:304] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 4df2ca35be5341afa5a621233a4da94e
W20260709 13:06:57.982877 17373 leader_election.cc:343] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.019313 17480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81d2393d5664464837df0b9d2a0d25b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:58.020651 17480 raft_consensus.cc:2468] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 0.
W20260709 13:06:58.040915 17373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 59780.8us
I20260709 13:06:58.041028 18043 raft_consensus.cc:493] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.042649 18043 raft_consensus.cc:515] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:58.045369 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:58.047708 17304 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40072 (request call id 60) took 1595 ms. Trace:
I20260709 13:06:58.048527 18043 leader_election.cc:290] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:58.048305 17304 rpcz_store.cc:276] 0709 13:06:56.461651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.470670 (+  9019us) service_pool.cc:224] Handling call
0709 13:06:56.521383 (+ 50713us) spinlock_profiling.cc:243] Waited 47.8 ms on lock 0x7b54003697d4. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e2463090 00007f32e2501f3a 00007f32e82b81cc 00007f32e83b637e 00007f32e68772a4 00007f32e6877232 00007f32e6877161 00007f32e68770dc 00007f32e6876392 00007f32e083448c 00007f32e08338c6
0709 13:06:58.047475 (+1526092us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19520,"reported_tablets":8,"spinlock_wait_cycles":110049024}
I20260709 13:06:58.032356 18038 raft_consensus.cc:2804] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:58.062760 18038 raft_consensus.cc:493] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:06:58.063290 18038 raft_consensus.cc:3060] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.062774 18040 consensus_queue.cc:1048] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
W20260709 13:06:58.058461 17524 leader_election.cc:343] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.060761 17303 catalog_manager.cc:5697] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } }
W20260709 13:06:58.070765 17373 leader_election.cc:343] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.070408 17303 catalog_manager.cc:5697] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } }
I20260709 13:06:58.072086 17596 leader_election.cc:304] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:58.073352 18047 raft_consensus.cc:2804] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.073796 18047 raft_consensus.cc:493] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.074124 18047 raft_consensus.cc:3060] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.074581 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:58.075464 17628 raft_consensus.cc:2468] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:06:58.076504 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:58.077922 18001 consensus_queue.cc:1048] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.073101 17303 catalog_manager.cc:5697] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.084734 17303 catalog_manager.cc:5697] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.055321 18037 leader_election.cc:290] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:58.080466 17829 raft_consensus.cc:515] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:58.066104 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81d2393d5664464837df0b9d2a0d25b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:58.092155 17484 raft_consensus.cc:2393] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
I20260709 13:06:58.093319 17829 leader_election.cc:290] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:58.089360 17524 leader_election.cc:304] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:58.095917 17303 catalog_manager.cc:5697] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.099380 18040 raft_consensus.cc:2804] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.100339 18040 raft_consensus.cc:493] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.100863 18040 raft_consensus.cc:3060] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.100968 17682 tablet_bootstrap.cc:492] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:58.101868 17682 ts_tablet_manager.cc:1403] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.295s	user 0.013s	sys 0.007s
I20260709 13:06:58.132562 17681 tablet_bootstrap.cc:492] T a81d2393d5664464837df0b9d2a0d25b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:58.133540 17681 ts_tablet_manager.cc:1403] T a81d2393d5664464837df0b9d2a0d25b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.575s	user 0.037s	sys 0.014s
I20260709 13:06:58.117636 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81d2393d5664464837df0b9d2a0d25b" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:58.155661 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:06:58.160290 17599 leader_election.cc:343] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.157795 17829 consensus_queue.cc:1048] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:06:58.158083 17632 raft_consensus.cc:2468] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:58.144011 18038 raft_consensus.cc:515] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:58.170060 17681 raft_consensus.cc:359] T a81d2393d5664464837df0b9d2a0d25b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:58.179716 17681 raft_consensus.cc:385] T a81d2393d5664464837df0b9d2a0d25b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.181485 17678 tablet_bootstrap.cc:654] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.180215 17681 raft_consensus.cc:740] T a81d2393d5664464837df0b9d2a0d25b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.180385 17682 raft_consensus.cc:359] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:58.189760 18047 raft_consensus.cc:515] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:58.194702 17627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:58.182540 17682 raft_consensus.cc:385] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.207978 17682 raft_consensus.cc:740] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.196981 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81d2393d5664464837df0b9d2a0d25b" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:58.208973 17627 raft_consensus.cc:3060] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.218475 17681 consensus_queue.cc:260] T a81d2393d5664464837df0b9d2a0d25b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:58.205081 18040 raft_consensus.cc:515] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:58.237362 17914 consensus_queue.cc:1048] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.242650 18040 leader_election.cc:290] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:58.244100 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:58.251354 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:58.251677 17409 raft_consensus.cc:3060] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.252159 17625 raft_consensus.cc:3060] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.253837 18047 leader_election.cc:290] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
W20260709 13:06:58.247504 18051 log.cc:927] Time spent T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:06:58.256639 17631 raft_consensus.cc:2393] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 1: Already voted for candidate a9a65d4c22be403682be9bc817650ada in this term.
I20260709 13:06:58.251608 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:58.256896 17404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:58.257719 17556 raft_consensus.cc:1240] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Rejecting Update request from peer 4df2ca35be5341afa5a621233a4da94e for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:58.244908 18038 leader_election.cc:290] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:58.237226 17682 consensus_queue.cc:260] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:58.260790 17375 leader_election.cc:304] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:58.263610 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:58.265666 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81d2393d5664464837df0b9d2a0d25b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:58.267560 17677 tablet_bootstrap.cc:492] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:58.268172 17677 ts_tablet_manager.cc:1403] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.333s	user 0.016s	sys 0.032s
I20260709 13:06:58.234048 17559 raft_consensus.cc:2468] T a81d2393d5664464837df0b9d2a0d25b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:58.273267 17681 ts_tablet_manager.cc:1434] T a81d2393d5664464837df0b9d2a0d25b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.139s	user 0.006s	sys 0.004s
I20260709 13:06:58.277290 17409 raft_consensus.cc:2468] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:58.277962 17928 raft_consensus.cc:2764] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:06:58.280951 17681 tablet_bootstrap.cc:492] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:58.300146 17524 leader_election.cc:304] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:58.301755 17677 raft_consensus.cc:359] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:58.308950 17954 consensus_queue.cc:1059] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.311264 17918 raft_consensus.cc:493] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.282932 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81d2393d5664464837df0b9d2a0d25b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:58.318167 18043 consensus_queue.cc:1048] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.323812 17677 raft_consensus.cc:385] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.324903 17682 ts_tablet_manager.cc:1434] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.201s	user 0.010s	sys 0.005s
I20260709 13:06:58.327687 17485 raft_consensus.cc:3060] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.329660 17677 raft_consensus.cc:740] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.335470 17682 tablet_bootstrap.cc:492] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:58.339763 17677 consensus_queue.cc:260] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:58.327292 17899 raft_consensus.cc:2749] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:58.348865 17677 ts_tablet_manager.cc:1434] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.003s
I20260709 13:06:58.360512 17450 leader_election.cc:304] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:58.333683 17918 raft_consensus.cc:515] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:58.363984 17485 raft_consensus.cc:2468] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 1.
W20260709 13:06:58.367614 17524 leader_election.cc:343] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.372289 17677 tablet_bootstrap.cc:492] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:58.366420 18039 raft_consensus.cc:2804] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.396660 17304 catalog_manager.cc:5697] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } }
I20260709 13:06:58.403718 17955 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.401453 17596 leader_election.cc:304] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:58.404363 17955 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:58.405112 17969 raft_consensus.cc:2804] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.405637 17969 raft_consensus.cc:697] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 1 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:06:58.406724 17969 consensus_queue.cc:237] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:58.423658 17678 tablet_bootstrap.cc:492] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:58.424292 17678 ts_tablet_manager.cc:1403] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.431s	user 0.046s	sys 0.046s
I20260709 13:06:58.435925 17955 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:58.441864 17451 leader_election.cc:304] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: a9a65d4c22be403682be9bc817650ada
I20260709 13:06:58.443536 18047 raft_consensus.cc:493] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.444164 18047 raft_consensus.cc:515] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:58.447814 18047 leader_election.cc:290] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:58.448179 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:06:58.450155 17596 leader_election.cc:343] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.450973 17677 tablet_bootstrap.cc:654] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.427336 18039 raft_consensus.cc:697] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:58.456085 17954 raft_consensus.cc:2764] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:06:58.456888 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:58.470445 17407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:58.471985 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:06:58.475013 17598 leader_election.cc:343] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.475622 17598 leader_election.cc:304] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:58.464620 17918 leader_election.cc:290] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:58.478334 18039 consensus_queue.cc:237] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:58.481572 17450 leader_election.cc:304] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:58.487905 17918 raft_consensus.cc:2749] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.479146 17829 raft_consensus.cc:3055] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:58.491824 17829 raft_consensus.cc:740] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:58.492303 17485 raft_consensus.cc:3060] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.492779 17829 consensus_queue.cc:260] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:58.494143 17682 tablet_bootstrap.cc:654] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.514456 17681 tablet_bootstrap.cc:654] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.516372 18043 consensus_queue.cc:1048] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.494347 17829 raft_consensus.cc:3060] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.479310 18047 raft_consensus.cc:2749] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.537891 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:58.538704 17404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:58.555476 17404 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 2.
I20260709 13:06:58.555958 17306 catalog_manager.cc:5697] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.558853 17678 raft_consensus.cc:359] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:58.562606 17678 raft_consensus.cc:385] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.562678 17524 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:06:58.563035 17678 raft_consensus.cc:740] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.568558 17899 raft_consensus.cc:2804] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:58.569130 17899 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.569536 17899 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:58.573637 17628 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:58.579049 17682 tablet_bootstrap.cc:492] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:58.579746 17682 ts_tablet_manager.cc:1403] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.245s	user 0.013s	sys 0.009s
I20260709 13:06:58.580765 17678 consensus_queue.cc:260] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:58.586344 17625 raft_consensus.cc:2468] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:06:58.584287 17627 raft_consensus.cc:2468] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:58.604552 17678 ts_tablet_manager.cc:1434] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.180s	user 0.009s	sys 0.000s
I20260709 13:06:58.605834 17678 tablet_bootstrap.cc:492] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:58.611548 17524 leader_election.cc:304] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:06:58.614182 17681 tablet_bootstrap.cc:492] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:58.617710 17678 tablet_bootstrap.cc:654] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.621624 17681 ts_tablet_manager.cc:1403] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.347s	user 0.009s	sys 0.010s
I20260709 13:06:58.625171 18038 raft_consensus.cc:2804] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.628386 17682 raft_consensus.cc:359] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:58.640235 18038 raft_consensus.cc:697] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:06:58.640271 17682 raft_consensus.cc:385] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.641315 18038 consensus_queue.cc:237] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:58.640904 17682 raft_consensus.cc:740] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.649312 17682 consensus_queue.cc:260] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:58.652860 17677 tablet_bootstrap.cc:492] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:58.653522 17677 ts_tablet_manager.cc:1403] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.288s	user 0.013s	sys 0.012s
W20260709 13:06:58.655443 17355 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:58.668601 18001 raft_consensus.cc:493] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.669572 18001 raft_consensus.cc:515] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:58.672690 17682 ts_tablet_manager.cc:1434] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.093s	user 0.009s	sys 0.000s
I20260709 13:06:58.674292 18001 leader_election.cc:290] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:58.677114 17682 tablet_bootstrap.cc:492] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:58.679665 17408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050a17ead1c04486b472a25b0bb179ca" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:58.680577 17408 raft_consensus.cc:2468] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:58.683120 17450 leader_election.cc:304] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:58.668466 17677 raft_consensus.cc:359] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:58.685843 17677 raft_consensus.cc:385] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.686244 17677 raft_consensus.cc:740] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.686558 17954 raft_consensus.cc:2804] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.689945 17954 raft_consensus.cc:493] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.690372 17954 raft_consensus.cc:3060] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.690276 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050a17ead1c04486b472a25b0bb179ca" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:58.691890 17681 raft_consensus.cc:359] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:58.695773 17681 raft_consensus.cc:385] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.696173 17681 raft_consensus.cc:740] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.696962 17681 consensus_queue.cc:260] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:58.700409 17954 raft_consensus.cc:515] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:58.698850 17304 catalog_manager.cc:5697] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } }
I20260709 13:06:58.702368 17803 raft_consensus.cc:493] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.702143 17304 catalog_manager.cc:5697] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 0 to 1, leader changed from <none> to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 1 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.702934 17803 raft_consensus.cc:515] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:58.706559 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34919e11221f465cbcef1151a1c5d4b0" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:58.707494 17625 raft_consensus.cc:2468] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:58.709003 17375 leader_election.cc:304] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:58.709940 18043 raft_consensus.cc:2804] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.710327 18043 raft_consensus.cc:493] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.710604 17681 ts_tablet_manager.cc:1434] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.002s
I20260709 13:06:58.710676 18043 raft_consensus.cc:3060] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.712046 17681 tablet_bootstrap.cc:492] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:58.693004 17899 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:58.705467 17954 leader_election.cc:290] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:58.716123 17803 leader_election.cc:290] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:58.723021 17677 consensus_queue.cc:260] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:58.728619 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050a17ead1c04486b472a25b0bb179ca" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:58.729457 17558 raft_consensus.cc:3060] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.732858 17408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050a17ead1c04486b472a25b0bb179ca" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:58.733579 17408 raft_consensus.cc:3060] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:58.740396 17451 leader_election.cc:343] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.749387 17899 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:58.751751 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:58.752198 17682 tablet_bootstrap.cc:654] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.752544 17625 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:06:58.761834 18065 log.cc:927] Time spent T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:06:58.763302 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34919e11221f465cbcef1151a1c5d4b0" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:58.772243 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:06:58.773007 17409 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:58.780121 17558 raft_consensus.cc:2468] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
W20260709 13:06:58.783425 17376 leader_election.cc:343] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.785054 17451 leader_election.cc:304] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:06:58.787259 17681 tablet_bootstrap.cc:654] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.786273 18039 raft_consensus.cc:2804] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.770267 17408 raft_consensus.cc:2468] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:58.792248 18039 raft_consensus.cc:697] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:58.803808 18039 consensus_queue.cc:237] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:58.809435 17678 tablet_bootstrap.cc:492] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:58.810355 17678 ts_tablet_manager.cc:1403] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.205s	user 0.025s	sys 0.005s
I20260709 13:06:58.823146 17677 ts_tablet_manager.cc:1434] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.169s	user 0.010s	sys 0.002s
I20260709 13:06:58.824966 17677 tablet_bootstrap.cc:492] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:58.826692 18043 raft_consensus.cc:515] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:58.814591 17678 raft_consensus.cc:359] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:58.828826 17678 raft_consensus.cc:385] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.829202 17678 raft_consensus.cc:740] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.831151 17678 consensus_queue.cc:260] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:58.853886 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34919e11221f465cbcef1151a1c5d4b0" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:58.857832 18043 leader_election.cc:290] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:58.858475 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34919e11221f465cbcef1151a1c5d4b0" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:58.859256 17631 raft_consensus.cc:3060] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:58.863969 17376 leader_election.cc:343] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.868913 17559 raft_consensus.cc:3060] T a81d2393d5664464837df0b9d2a0d25b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.876399 17625 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 3.
I20260709 13:06:58.831190 17409 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 3.
I20260709 13:06:58.881619 17524 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:58.882519 18038 raft_consensus.cc:2804] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:58.883104 18038 raft_consensus.cc:697] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:06:58.884112 18038 consensus_queue.cc:237] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:58.891629 17677 tablet_bootstrap.cc:654] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.901145 17303 catalog_manager.cc:5697] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.904392 17829 raft_consensus.cc:493] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.905165 17829 raft_consensus.cc:515] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:58.906349 17682 tablet_bootstrap.cc:492] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:58.914906 17682 ts_tablet_manager.cc:1403] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.239s	user 0.011s	sys 0.027s
I20260709 13:06:58.908170 17678 ts_tablet_manager.cc:1434] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.002s
I20260709 13:06:58.913854 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb6246353144b0a8b9648ac089a1da6" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:58.919077 17829 leader_election.cc:290] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:58.925462 17631 raft_consensus.cc:2468] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:58.929898 17628 raft_consensus.cc:2468] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:58.931661 17450 leader_election.cc:304] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:58.933137 18039 raft_consensus.cc:2804] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.933604 18039 raft_consensus.cc:493] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.933993 18039 raft_consensus.cc:3060] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.936930 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb6246353144b0a8b9648ac089a1da6" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:58.933167 17375 leader_election.cc:304] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
W20260709 13:06:58.938975 17451 leader_election.cc:343] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.939730 17803 raft_consensus.cc:2804] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.940290 17803 raft_consensus.cc:697] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:06:58.941182 18047 raft_consensus.cc:493] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.941154 17803 consensus_queue.cc:237] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:58.941812 18047 raft_consensus.cc:515] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:58.945456 17682 raft_consensus.cc:359] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:58.946305 17682 raft_consensus.cc:385] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.946583 17682 raft_consensus.cc:740] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.948168 18047 leader_election.cc:290] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:06:58.949472 17678 tablet_bootstrap.cc:492] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:58.949638 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b2d2a20fb474604a1077ce9af500e42" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:58.949656 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b2d2a20fb474604a1077ce9af500e42" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:06:58.953089 17596 leader_election.cc:343] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:58.954476 17598 leader_election.cc:343] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.955020 17598 leader_election.cc:304] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:58.955891 18047 raft_consensus.cc:2749] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.947360 17682 consensus_queue.cc:260] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:58.966543 17304 catalog_manager.cc:5697] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.968585 17304 catalog_manager.cc:5697] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.980113 17682 ts_tablet_manager.cc:1434] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.001s
I20260709 13:06:58.979841 18039 raft_consensus.cc:515] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:58.984586 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb6246353144b0a8b9648ac089a1da6" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:58.985378 17969 consensus_queue.cc:1048] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:58.990921 17451 leader_election.cc:343] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.993381 17682 tablet_bootstrap.cc:492] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:58.994841 17969 consensus_queue.cc:1048] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:59.003173 17303 catalog_manager.cc:5697] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 1 to 3, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.006850 17682 tablet_bootstrap.cc:654] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.012140 17677 tablet_bootstrap.cc:492] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:59.013043 17677 ts_tablet_manager.cc:1403] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.188s	user 0.006s	sys 0.014s
I20260709 13:06:59.035483 18039 leader_election.cc:290] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:59.038149 17681 tablet_bootstrap.cc:492] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:59.037964 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb6246353144b0a8b9648ac089a1da6" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:06:59.038898 17628 raft_consensus.cc:3060] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.038856 17681 ts_tablet_manager.cc:1403] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.327s	user 0.012s	sys 0.011s
I20260709 13:06:59.068050 17681 raft_consensus.cc:359] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:59.068919 17681 raft_consensus.cc:385] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.069336 17681 raft_consensus.cc:740] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.082665 17628 raft_consensus.cc:2468] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:06:59.086211 17450 leader_election.cc:304] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:59.091132 17681 consensus_queue.cc:260] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:59.088075 17829 raft_consensus.cc:2804] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:59.096004 17829 raft_consensus.cc:697] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:06:59.097355 17829 consensus_queue.cc:237] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:59.102742 17981 raft_consensus.cc:493] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.103415 17981 raft_consensus.cc:515] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:59.116364 17677 raft_consensus.cc:359] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:59.117798 17677 raft_consensus.cc:385] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.118141 17677 raft_consensus.cc:740] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.127300 18001 consensus_queue.cc:1048] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:06:59.133962 17682 tablet_bootstrap.cc:492] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:59.134697 17682 ts_tablet_manager.cc:1403] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.142s	user 0.010s	sys 0.031s
I20260709 13:06:59.136533 17304 catalog_manager.cc:5697] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:59.125356 17677 consensus_queue.cc:260] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:59.142273 17954 consensus_queue.cc:1048] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:59.143251 18074 log.cc:927] Time spent T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:06:59.145944 17981 leader_election.cc:290] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:59.148126 17682 raft_consensus.cc:359] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:59.149497 17682 raft_consensus.cc:385] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.149891 17682 raft_consensus.cc:740] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.150640 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcc480cc6534780aa5a81e4c97e8bef" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:59.152204 17682 consensus_queue.cc:260] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:59.155480 17682 ts_tablet_manager.cc:1434] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
W20260709 13:06:59.155457 17596 leader_election.cc:343] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.157294 17682 tablet_bootstrap.cc:492] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:59.157994 17678 tablet_bootstrap.cc:654] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.159749 17681 ts_tablet_manager.cc:1434] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.110s	user 0.008s	sys 0.002s
I20260709 13:06:59.164186 17681 tablet_bootstrap.cc:492] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:59.167629 17677 ts_tablet_manager.cc:1434] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.154s	user 0.010s	sys 0.001s
I20260709 13:06:59.169147 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcc480cc6534780aa5a81e4c97e8bef" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:59.173271 17677 tablet_bootstrap.cc:492] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
W20260709 13:06:59.174577 17599 leader_election.cc:343] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.178576 17599 leader_election.cc:304] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:06:59.190909 17969 raft_consensus.cc:493] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.193073 17969 raft_consensus.cc:515] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:59.199903 17969 leader_election.cc:290] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:59.200021 17955 raft_consensus.cc:493] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.200802 17955 raft_consensus.cc:515] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:59.204371 17955 leader_election.cc:290] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:59.204232 17408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05345b561dfb497c8f1e081b05c1bb5d" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:59.205740 17677 tablet_bootstrap.cc:654] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.181855 17981 raft_consensus.cc:2749] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:59.220160 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05345b561dfb497c8f1e081b05c1bb5d" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:06:59.223886 17599 leader_election.cc:343] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:59.230535 17598 leader_election.cc:343] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.230984 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34919e11221f465cbcef1151a1c5d4b0" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:59.231181 17598 leader_election.cc:304] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:06:59.231909 17631 raft_consensus.cc:2393] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:06:59.232861 18047 raft_consensus.cc:2749] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:59.226536 17408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34919e11221f465cbcef1151a1c5d4b0" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:59.234889 17681 tablet_bootstrap.cc:654] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.241515 17803 raft_consensus.cc:493] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.248706 17803 raft_consensus.cc:515] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:59.268940 17524 leader_election.cc:304] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:59.270045 17899 raft_consensus.cc:3060] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.270911 17677 tablet_bootstrap.cc:492] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:59.271741 17677 ts_tablet_manager.cc:1403] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.099s	user 0.028s	sys 0.016s
I20260709 13:06:59.271606 17307 catalog_manager.cc:5697] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.273870 17803 leader_election.cc:290] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:59.275270 17677 raft_consensus.cc:359] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:59.276122 17677 raft_consensus.cc:385] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.276463 17677 raft_consensus.cc:740] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.277230 17677 consensus_queue.cc:260] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:59.280172 17677 ts_tablet_manager.cc:1434] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:06:59.281689 17677 tablet_bootstrap.cc:492] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:59.290835 17677 tablet_bootstrap.cc:654] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.303896 17678 tablet_bootstrap.cc:492] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:59.304515 17678 ts_tablet_manager.cc:1403] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.355s	user 0.019s	sys 0.007s
I20260709 13:06:59.303203 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20df005b7408432da9ac4fcf110a20b9" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:59.314173 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20df005b7408432da9ac4fcf110a20b9" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:59.318801 17408 raft_consensus.cc:3060] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.318849 17484 raft_consensus.cc:2468] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:59.325104 17373 leader_election.cc:304] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
W20260709 13:06:59.327940 17376 leader_election.cc:343] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.330492 18038 consensus_queue.cc:1048] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:59.335111 17803 raft_consensus.cc:2804] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:59.344466 18041 raft_consensus.cc:493] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.350694 18041 raft_consensus.cc:515] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:59.354039 17682 tablet_bootstrap.cc:654] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.344611 17803 raft_consensus.cc:493] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.359134 17677 tablet_bootstrap.cc:492] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:59.364805 17677 ts_tablet_manager.cc:1403] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.084s	user 0.017s	sys 0.018s
I20260709 13:06:59.359190 17678 raft_consensus.cc:359] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:59.366408 17678 raft_consensus.cc:385] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.366878 17678 raft_consensus.cc:740] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.368798 17803 raft_consensus.cc:3060] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.370999 17678 consensus_queue.cc:260] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:59.374405 17678 ts_tablet_manager.cc:1434] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.070s	user 0.009s	sys 0.000s
I20260709 13:06:59.374717 17899 raft_consensus.cc:2749] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:59.390786 18041 leader_election.cc:290] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:59.398739 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:59.412248 18039 consensus_queue.cc:1048] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:59.412766 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:59.412873 17678 tablet_bootstrap.cc:492] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:59.424536 17677 raft_consensus.cc:359] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:59.426126 17677 raft_consensus.cc:385] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.426420 17677 raft_consensus.cc:740] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.427076 17677 consensus_queue.cc:260] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:59.431934 18038 consensus_queue.cc:1048] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:59.436623 17678 tablet_bootstrap.cc:654] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.441224 17677 ts_tablet_manager.cc:1434] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.076s	user 0.009s	sys 0.000s
I20260709 13:06:59.442083 18043 consensus_queue.cc:1048] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:06:59.445163 17376 leader_election.cc:304] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:59.447198 18043 raft_consensus.cc:2749] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:59.458051 17682 tablet_bootstrap.cc:492] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:06:59.462030 17682 ts_tablet_manager.cc:1403] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.305s	user 0.013s	sys 0.009s
I20260709 13:06:59.464221 17677 tablet_bootstrap.cc:492] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:06:59.468338 17631 raft_consensus.cc:1275] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:59.469374 17954 consensus_queue.cc:1048] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:59.485081 17681 tablet_bootstrap.cc:492] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:59.485850 17681 ts_tablet_manager.cc:1403] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.322s	user 0.018s	sys 0.010s
I20260709 13:06:59.487008 17803 raft_consensus.cc:515] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:59.487658 18038 consensus_queue.cc:1048] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:59.489168 17681 raft_consensus.cc:359] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:59.500001 17681 raft_consensus.cc:385] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.500368 17681 raft_consensus.cc:740] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.503563 18039 raft_consensus.cc:493] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.492280 17682 raft_consensus.cc:359] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:59.511551 17682 raft_consensus.cc:385] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.511997 17682 raft_consensus.cc:740] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.541410 17681 consensus_queue.cc:260] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:59.489908 17803 leader_election.cc:290] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:59.505398 18039 raft_consensus.cc:515] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:59.542966 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20df005b7408432da9ac4fcf110a20b9" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:59.546355 17484 raft_consensus.cc:3060] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.545023 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20df005b7408432da9ac4fcf110a20b9" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:06:59.543859 17682 consensus_queue.cc:260] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:59.490929 18043 consensus_queue.cc:1048] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.053s
I20260709 13:06:59.566949 17677 tablet_bootstrap.cc:654] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.567878 17681 ts_tablet_manager.cc:1434] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.082s	user 0.009s	sys 0.001s
W20260709 13:06:59.562923 17376 leader_election.cc:343] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.598742 17681 tablet_bootstrap.cc:492] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:06:59.607244 17682 ts_tablet_manager.cc:1434] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.145s	user 0.008s	sys 0.004s
W20260709 13:06:59.613097 18098 log.cc:927] Time spent T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:06:59.616806 17678 tablet_bootstrap.cc:492] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:06:59.617468 17678 ts_tablet_manager.cc:1403] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.205s	user 0.030s	sys 0.026s
I20260709 13:06:59.633424 17682 tablet_bootstrap.cc:492] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:06:59.640786 18038 raft_consensus.cc:493] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.652055 18038 raft_consensus.cc:515] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:59.658825 18038 leader_election.cc:290] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:59.646310 17404 raft_consensus.cc:1275] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:59.703176 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcc480cc6534780aa5a81e4c97e8bef" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:59.716112 17678 raft_consensus.cc:359] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:59.720256 17981 raft_consensus.cc:493] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.720883 17981 raft_consensus.cc:515] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:59.718433 17681 tablet_bootstrap.cc:654] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.721792 17678 raft_consensus.cc:385] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.726107 17678 raft_consensus.cc:740] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
W20260709 13:06:59.744395 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 49152us
I20260709 13:06:59.744925 17678 consensus_queue.cc:260] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:59.745429 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb6246353144b0a8b9648ac089a1da6" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:59.747493 17633 raft_consensus.cc:2393] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
I20260709 13:06:59.748387 18037 raft_consensus.cc:493] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.749020 18037 raft_consensus.cc:515] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:06:59.763167 18037 leader_election.cc:290] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:59.764218 17484 raft_consensus.cc:2468] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:06:59.765341 17955 consensus_queue.cc:1048] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:59.770872 17678 ts_tablet_manager.cc:1434] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.153s	user 0.008s	sys 0.003s
I20260709 13:06:59.744735 17969 raft_consensus.cc:493] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:06:59.769611 18039 leader_election.cc:290] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:06:59.817771 17969 raft_consensus.cc:515] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
W20260709 13:06:59.828074 17451 leader_election.cc:343] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.828933 17969 leader_election.cc:290] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:06:59.837978 17682 tablet_bootstrap.cc:654] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:59.853734 17981 leader_election.cc:290] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:06:59.855027 17681 tablet_bootstrap.cc:492] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:06:59.855989 17681 ts_tablet_manager.cc:1403] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.269s	user 0.021s	sys 0.007s
I20260709 13:06:59.879230 17803 raft_consensus.cc:493] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.880007 17803 raft_consensus.cc:515] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:59.892133 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:59.902557 17485 raft_consensus.cc:2468] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:06:59.886204 17404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:59.883536 17403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307acc498b0c4da7b7953e895b049c2a" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:06:59.910787 17480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb6246353144b0a8b9648ac089a1da6" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:59.917666 17677 tablet_bootstrap.cc:492] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:06:59.918340 17677 ts_tablet_manager.cc:1403] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.475s	user 0.053s	sys 0.060s
I20260709 13:06:59.921654 17677 raft_consensus.cc:359] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:59.887208 17550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def346af33c747faaf6c9cf0414510f0" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:59.927526 17681 raft_consensus.cc:359] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:59.928440 17681 raft_consensus.cc:385] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.928715 17681 raft_consensus.cc:740] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.929417 17681 consensus_queue.cc:260] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:06:59.882555 17803 leader_election.cc:290] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
W20260709 13:06:59.931047 17598 leader_election.cc:343] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.922466 17677 raft_consensus.cc:385] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.935084 17677 raft_consensus.cc:740] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.935830 17677 consensus_queue.cc:260] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:06:59.935845 17522 leader_election.cc:304] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:06:59.937749 17596 leader_election.cc:304] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:06:59.938678 18113 raft_consensus.cc:2804] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:59.939116 18113 raft_consensus.cc:493] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:06:59.939507 18113 raft_consensus.cc:3060] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.938676 17678 tablet_bootstrap.cc:492] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:06:59.948302 17677 ts_tablet_manager.cc:1434] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
W20260709 13:06:59.949559 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 125521us
W20260709 13:06:59.951494 17599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 180857us
I20260709 13:06:59.963702 17681 ts_tablet_manager.cc:1434] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.107s	user 0.008s	sys 0.001s
I20260709 13:06:59.965397 17677 tablet_bootstrap.cc:492] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
W20260709 13:06:59.930945 17376 leader_election.cc:343] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:59.966058 17355 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:59.964963 17955 raft_consensus.cc:3060] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:59.974884 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 44002.9us
W20260709 13:06:59.976557 17450 leader_election.cc:343] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:59.974728 18113 raft_consensus.cc:515] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:06:59.986167 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcc480cc6534780aa5a81e4c97e8bef" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:06:59.987278 17631 raft_consensus.cc:2468] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:06:59.989204 17481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def346af33c747faaf6c9cf0414510f0" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:06:59.990150 17481 raft_consensus.cc:2468] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:06:59.968063 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:59.993146 17555 raft_consensus.cc:2393] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a9a65d4c22be403682be9bc817650ada in current term 2: Already voted for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in this term.
I20260709 13:06:59.994135 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:06:59.972147 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307acc498b0c4da7b7953e895b049c2a" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:06:59.999847 17678 tablet_bootstrap.cc:654] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:00.003657 17681 tablet_bootstrap.cc:492] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:07:00.013803 17681 tablet_bootstrap.cc:654] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:00.033430 17599 leader_election.cc:343] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:00.036827 17599 leader_election.cc:304] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:00.040429 17955 raft_consensus.cc:2749] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:00.040429 18113 leader_election.cc:290] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:00.042435 17485 raft_consensus.cc:2468] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:00.023095 17969 raft_consensus.cc:493] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.052690 17969 raft_consensus.cc:515] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:00.054214 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:00.055120 17631 raft_consensus.cc:2468] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:07:00.041857 17373 leader_election.cc:304] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:00.057036 17969 leader_election.cc:290] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:00.057436 17481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c295096a121e4940b38125740cea67cb" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:07:00.059938 17596 leader_election.cc:343] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:00.060379 17373 leader_election.cc:304] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:00.060767 18041 raft_consensus.cc:2804] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:00.062814 17596 leader_election.cc:304] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:00.062911 17450 leader_election.cc:304] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:00.064859 18113 raft_consensus.cc:2804] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:00.063885 18114 consensus_queue.cc:1048] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:07:00.066388 18115 raft_consensus.cc:2804] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:00.066874 18115 raft_consensus.cc:493] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.067242 18115 raft_consensus.cc:3060] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:00.067494 18106 raft_consensus.cc:2804] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:00.067978 18106 raft_consensus.cc:493] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.068303 18106 raft_consensus.cc:3060] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:00.070380 18041 raft_consensus.cc:697] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:00.071905 18041 consensus_queue.cc:237] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:00.076129 17677 tablet_bootstrap.cc:654] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:00.067202 18113 raft_consensus.cc:697] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:00.078819 17969 raft_consensus.cc:2749] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:00.079008 17599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 45647.7us
I20260709 13:07:00.078274 18113 consensus_queue.cc:237] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:00.097909 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:00.099366 17552 raft_consensus.cc:2393] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a9a65d4c22be403682be9bc817650ada in current term 2: Already voted for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in this term.
I20260709 13:07:00.099885 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c295096a121e4940b38125740cea67cb" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:07:00.088183 17375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 58410us
I20260709 13:07:00.107589 17450 leader_election.cc:304] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:00.111982 18107 raft_consensus.cc:2804] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:00.112496 18107 raft_consensus.cc:493] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.112841 18107 raft_consensus.cc:3060] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:00.114645 17829 consensus_queue.cc:1048] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:07:00.121510 17599 leader_election.cc:343] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:00.122040 17599 leader_election.cc:304] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:00.125384 18047 raft_consensus.cc:2749] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:00.134307 17682 tablet_bootstrap.cc:492] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:07:00.134747 17678 tablet_bootstrap.cc:492] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:07:00.135442 17678 ts_tablet_manager.cc:1403] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.234s	user 0.024s	sys 0.000s
I20260709 13:07:00.135442 17682 ts_tablet_manager.cc:1403] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.506s	user 0.031s	sys 0.031s
I20260709 13:07:00.167291 17681 tablet_bootstrap.cc:492] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:07:00.168143 18106 raft_consensus.cc:515] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:00.168143 17681 ts_tablet_manager.cc:1403] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.165s	user 0.016s	sys 0.008s
I20260709 13:07:00.178054 17678 raft_consensus.cc:359] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:00.179116 17678 raft_consensus.cc:385] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:00.179491 17678 raft_consensus.cc:740] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.180388 17678 consensus_queue.cc:260] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:00.192306 17678 ts_tablet_manager.cc:1434] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.056s	user 0.009s	sys 0.000s
I20260709 13:07:00.214959 17682 raft_consensus.cc:359] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:00.215876 17682 raft_consensus.cc:385] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:00.216146 17682 raft_consensus.cc:740] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.227041 17678 tablet_bootstrap.cc:492] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:07:00.221534 17682 consensus_queue.cc:260] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:00.235891 17681 raft_consensus.cc:359] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:00.236846 17681 raft_consensus.cc:385] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:00.237130 17681 raft_consensus.cc:740] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.239147 17681 consensus_queue.cc:260] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:00.247534 18106 leader_election.cc:290] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:00.274230 17682 ts_tablet_manager.cc:1434] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.137s	user 0.008s	sys 0.002s
I20260709 13:07:00.276458 17681 ts_tablet_manager.cc:1434] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.107s	user 0.007s	sys 0.002s
I20260709 13:07:00.279112 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcc480cc6534780aa5a81e4c97e8bef" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
W20260709 13:07:00.292878 17451 leader_election.cc:343] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:00.318918 18105 raft_consensus.cc:493] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.319713 18105 raft_consensus.cc:515] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:00.230005 18115 raft_consensus.cc:515] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:00.333982 17627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcc480cc6534780aa5a81e4c97e8bef" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:00.335197 17627 raft_consensus.cc:3060] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:00.326475 18105 leader_election.cc:290] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:00.343793 18107 raft_consensus.cc:515] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:00.348498 17629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:00.349407 17629 raft_consensus.cc:3060] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:00.356747 17678 tablet_bootstrap.cc:654] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:00.355099 17304 catalog_manager.cc:5697] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 2 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:00.361254 17681 tablet_bootstrap.cc:492] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:07:00.371801 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def346af33c747faaf6c9cf0414510f0" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:00.372746 17485 raft_consensus.cc:3060] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:00.391070 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05345b561dfb497c8f1e081b05c1bb5d" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:00.409508 17682 tablet_bootstrap.cc:492] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:00.394783 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05345b561dfb497c8f1e081b05c1bb5d" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:00.424723 18115 leader_election.cc:290] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:00.428447 17955 raft_consensus.cc:493] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.430596 17955 raft_consensus.cc:515] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:00.442121 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:00.443838 17481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:00.444713 17481 raft_consensus.cc:2468] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 2.
I20260709 13:07:00.450979 17522 leader_election.cc:304] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:00.452293 18038 raft_consensus.cc:2804] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:00.452759 18038 raft_consensus.cc:493] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.453120 18038 raft_consensus.cc:3060] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:00.431486 17628 raft_consensus.cc:2468] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:07:00.464099 18107 leader_election.cc:290] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:00.465973 17629 raft_consensus.cc:2468] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:07:00.466854 17955 leader_election.cc:290] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:00.467196 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
W20260709 13:07:00.473863 17524 leader_election.cc:343] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:00.478276 17450 leader_election.cc:343] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:00.481730 17485 raft_consensus.cc:2468] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:07:00.447839 17627 raft_consensus.cc:2468] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:07:00.486688 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def346af33c747faaf6c9cf0414510f0" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:00.494395 17373 leader_election.cc:304] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:00.495486 17803 raft_consensus.cc:2804] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:00.496047 17803 raft_consensus.cc:697] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:00.497834 17803 consensus_queue.cc:237] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:00.503149 18120 raft_consensus.cc:493] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.504230 18120 raft_consensus.cc:515] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:00.507517 18120 leader_election.cc:290] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:00.514585 17450 leader_election.cc:304] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:00.480242 17524 leader_election.cc:304] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:00.535089 17450 leader_election.cc:304] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:00.532475 17955 raft_consensus.cc:2804] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:00.539346 18107 raft_consensus.cc:2804] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:00.537875 17480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b2d2a20fb474604a1077ce9af500e42" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:00.539469 18037 raft_consensus.cc:2804] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:00.560210 17677 tablet_bootstrap.cc:492] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:07:00.562433 17677 ts_tablet_manager.cc:1403] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.597s	user 0.039s	sys 0.067s
W20260709 13:07:00.537829 17376 leader_election.cc:343] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:00.548740 17955 raft_consensus.cc:493] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.565073 17981 raft_consensus.cc:493] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.558861 18104 raft_consensus.cc:493] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:00.581678 17373 leader_election.cc:343] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:00.572901 17955 raft_consensus.cc:3060] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:00.591106 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 56801.4us
I20260709 13:07:00.592654 17681 tablet_bootstrap.cc:654] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:00.569203 18107 raft_consensus.cc:697] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:00.595953 18107 consensus_queue.cc:237] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:00.574180 18037 raft_consensus.cc:697] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:00.598752 18037 consensus_queue.cc:237] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:00.600288 17969 raft_consensus.cc:493] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.601210 17481 raft_consensus.cc:1275] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:00.600957 17969 raft_consensus.cc:515] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:00.610066 18113 consensus_queue.cc:1048] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:00.612632 17969 leader_election.cc:290] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:00.613346 17629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b2d2a20fb474604a1077ce9af500e42" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:00.614188 17629 raft_consensus.cc:2468] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:07:00.601881 18104 raft_consensus.cc:515] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:00.614269 17407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b2d2a20fb474604a1077ce9af500e42" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:00.573607 17981 raft_consensus.cc:515] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:00.621814 17981 leader_election.cc:290] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:00.623690 17407 raft_consensus.cc:2468] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 0.
I20260709 13:07:00.623940 18104 leader_election.cc:290] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:00.625800 17598 leader_election.cc:304] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:00.626242 17955 raft_consensus.cc:515] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:00.626991 18047 raft_consensus.cc:2804] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:00.627599 18047 raft_consensus.cc:493] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.628139 18047 raft_consensus.cc:3060] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:00.663594 17306 catalog_manager.cc:5697] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:00.641438 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:00.687323 17481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b2d2a20fb474604a1077ce9af500e42" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:00.702118 17678 tablet_bootstrap.cc:492] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:07:00.702844 17678 ts_tablet_manager.cc:1403] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.477s	user 0.025s	sys 0.009s
I20260709 13:07:00.703378 17955 leader_election.cc:290] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:00.705358 17677 raft_consensus.cc:359] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:00.706243 17677 raft_consensus.cc:385] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:00.706529 17677 raft_consensus.cc:740] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.706902 17678 raft_consensus.cc:359] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:00.717234 17678 raft_consensus.cc:385] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:00.717726 17678 raft_consensus.cc:740] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.715946 17677 consensus_queue.cc:260] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:00.721487 17678 consensus_queue.cc:260] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:00.729231 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:00.702481 17553 raft_consensus.cc:3060] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:00.737411 17482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20df005b7408432da9ac4fcf110a20b9" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:00.738540 17482 raft_consensus.cc:2393] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:07:00.747682 17677 ts_tablet_manager.cc:1434] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.184s	user 0.009s	sys 0.001s
W20260709 13:07:00.712711 17598 leader_election.cc:343] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:00.769049 17682 tablet_bootstrap.cc:654] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:00.776185 17598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 47363us
I20260709 13:07:00.749359 18115 consensus_queue.cc:1048] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 13:07:00.779017 17375 leader_election.cc:304] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:00.783044 18102 raft_consensus.cc:493] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.789718 17678 ts_tablet_manager.cc:1434] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.086s	user 0.010s	sys 0.000s
W20260709 13:07:00.806028 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 53851.7us
I20260709 13:07:00.814344 17404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05345b561dfb497c8f1e081b05c1bb5d" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:07:00.819715 17803 raft_consensus.cc:2804] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:00.820199 17803 raft_consensus.cc:493] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.820525 17803 raft_consensus.cc:3060] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:00.821866 17681 tablet_bootstrap.cc:492] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:07:00.822472 17681 ts_tablet_manager.cc:1403] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.521s	user 0.014s	sys 0.015s
I20260709 13:07:00.818759 17629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05345b561dfb497c8f1e081b05c1bb5d" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:00.813328 18102 raft_consensus.cc:515] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:00.827728 17629 raft_consensus.cc:3060] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:00.829382 18102 leader_election.cc:290] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:00.830464 17678 tablet_bootstrap.cc:492] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:07:00.830924 17481 raft_consensus.cc:2468] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 0.
I20260709 13:07:00.825649 17681 raft_consensus.cc:359] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:00.840065 17404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b2d2a20fb474604a1077ce9af500e42" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:00.840457 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b2d2a20fb474604a1077ce9af500e42" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:00.842727 17681 raft_consensus.cc:385] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:00.847482 17681 raft_consensus.cc:740] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:07:00.848953 17681 consensus_queue.cc:260] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:00.853068 17681 ts_tablet_manager.cc:1434] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.003s
I20260709 13:07:00.874024 17681 tablet_bootstrap.cc:492] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:07:00.883927 17407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20df005b7408432da9ac4fcf110a20b9" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:00.871560 17803 raft_consensus.cc:515] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:00.855208 17629 raft_consensus.cc:2468] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:07:00.899878 17803 leader_election.cc:290] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
W20260709 13:07:00.862844 17524 leader_election.cc:343] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:00.882272 17375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 103424us
I20260709 13:07:00.921911 17677 tablet_bootstrap.cc:492] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
W20260709 13:07:00.924183 17596 leader_election.cc:343] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:00.924918 17596 leader_election.cc:304] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:00.835089 18038 raft_consensus.cc:515] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:00.925261 17524 leader_election.cc:304] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:00.936990 17678 tablet_bootstrap.cc:654] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:00.933223 17969 raft_consensus.cc:2749] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:00.935050 17404 raft_consensus.cc:2393] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:07:00.917460 17480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b2d2a20fb474604a1077ce9af500e42" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:00.960008 17633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b2d2a20fb474604a1077ce9af500e42" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:00.957916 17524 leader_election.cc:304] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:00.954384 18105 raft_consensus.cc:2749] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:00.969954 18038 leader_election.cc:290] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:00.945544 18047 raft_consensus.cc:515] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:00.971899 17554 raft_consensus.cc:1240] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Rejecting Update request from peer a9a65d4c22be403682be9bc817650ada for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:00.971902 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:00.974985 17625 raft_consensus.cc:2393] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 1: Already voted for candidate a9a65d4c22be403682be9bc817650ada in this term.
I20260709 13:07:00.976755 17407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b2d2a20fb474604a1077ce9af500e42" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:07:00.980159 18047 leader_election.cc:290] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:00.981308 17681 tablet_bootstrap.cc:654] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:00.984717 17677 tablet_bootstrap.cc:654] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:00.991475 18042 consensus_queue.cc:1048] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:00.991789 17629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:00.997814 18038 raft_consensus.cc:2804] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:00.998567 18038 raft_consensus.cc:697] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:00.999635 18038 consensus_queue.cc:237] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:01.002974 17682 tablet_bootstrap.cc:492] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:07:01.007256 17682 ts_tablet_manager.cc:1403] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.628s	user 0.026s	sys 0.014s
I20260709 13:07:01.017339 17524 leader_election.cc:304] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:01.017539 17303 catalog_manager.cc:5697] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:01.025645 17303 catalog_manager.cc:5697] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:01.033434 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 45724.8us
I20260709 13:07:01.035842 17373 leader_election.cc:304] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:01.044095 17969 consensus_queue.cc:1059] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:01.023675 18104 raft_consensus.cc:2749] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:01.045691 17969 raft_consensus.cc:3055] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:01.054955 18042 raft_consensus.cc:2749] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:01.086007 17450 leader_election.cc:304] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:01.088819 17678 tablet_bootstrap.cc:492] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:07:01.089550 17678 ts_tablet_manager.cc:1403] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.259s	user 0.030s	sys 0.025s
I20260709 13:07:01.094298 17678 raft_consensus.cc:359] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:01.095161 17678 raft_consensus.cc:385] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:01.095480 17678 raft_consensus.cc:740] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.095763 18106 raft_consensus.cc:3060] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:01.097527 17678 consensus_queue.cc:260] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
W20260709 13:07:01.103878 17376 consensus_peers.cc:597] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c -> Peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Couldn't send request to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.103972 17678 ts_tablet_manager.cc:1434] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:07:01.120828 17558 raft_consensus.cc:3060] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:01.074278 17969 raft_consensus.cc:740] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:01.123631 17969 consensus_queue.cc:260] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:01.127029 17969 raft_consensus.cc:3060] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:01.138351 17681 tablet_bootstrap.cc:492] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:07:01.139405 17681 ts_tablet_manager.cc:1403] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.266s	user 0.020s	sys 0.020s
I20260709 13:07:01.140630 17678 tablet_bootstrap.cc:492] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:07:01.143549 17829 consensus_queue.cc:1048] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:01.180843 17682 raft_consensus.cc:359] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:01.181833 17682 raft_consensus.cc:385] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:01.182173 17682 raft_consensus.cc:740] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.188956 17681 raft_consensus.cc:359] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:01.189889 17681 raft_consensus.cc:385] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:01.166961 18047 raft_consensus.cc:493] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.165782 18120 consensus_queue.cc:1048] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260709 13:07:01.183629 17682 consensus_queue.cc:260] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:01.213831 17681 raft_consensus.cc:740] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.231483 18151 consensus_queue.cc:1048] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:01.231635 18047 raft_consensus.cc:515] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:01.248713 17681 consensus_queue.cc:260] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:01.249097 17306 catalog_manager.cc:5697] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:01.256687 17681 ts_tablet_manager.cc:1434] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.117s	user 0.009s	sys 0.000s
I20260709 13:07:01.251159 17682 ts_tablet_manager.cc:1434] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.242s	user 0.009s	sys 0.000s
W20260709 13:07:01.260187 17355 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:01.264732 17677 tablet_bootstrap.cc:492] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:07:01.265473 17677 ts_tablet_manager.cc:1403] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.359s	user 0.015s	sys 0.010s
I20260709 13:07:01.286770 17678 tablet_bootstrap.cc:654] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:01.299136 17481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b2d2a20fb474604a1077ce9af500e42" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:01.307096 17681 tablet_bootstrap.cc:492] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:07:01.309064 18047 leader_election.cc:290] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
W20260709 13:07:01.288259 17598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 51323.9us
I20260709 13:07:01.313189 17682 tablet_bootstrap.cc:492] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:07:01.313890 17375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 209832us
W20260709 13:07:01.330593 17373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 107021us
I20260709 13:07:01.357136 17677 raft_consensus.cc:359] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:01.360069 17677 raft_consensus.cc:385] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:01.360422 17677 raft_consensus.cc:740] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.362604 17677 consensus_queue.cc:260] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:01.365190 17677 ts_tablet_manager.cc:1434] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.099s	user 0.007s	sys 0.001s
I20260709 13:07:01.365993 18104 raft_consensus.cc:493] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.366550 17677 tablet_bootstrap.cc:492] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:07:01.366756 18106 raft_consensus.cc:2749] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:01.366658 18104 raft_consensus.cc:515] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:01.375695 17677 tablet_bootstrap.cc:654] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:01.377226 17681 tablet_bootstrap.cc:654] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:01.383013 17682 tablet_bootstrap.cc:654] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:01.402062 18114 consensus_queue.cc:1048] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:07:01.445065 18041 raft_consensus.cc:493] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.475754 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea055e03bf849bea8fdc851e10a8cbe" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:07:01.487452 17599 leader_election.cc:343] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:01.498324 17677 tablet_bootstrap.cc:492] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:07:01.500824 17677 ts_tablet_manager.cc:1403] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.135s	user 0.018s	sys 0.031s
I20260709 13:07:01.502063 17303 catalog_manager.cc:5697] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:01.446019 18041 raft_consensus.cc:515] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:01.481679 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea055e03bf849bea8fdc851e10a8cbe" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:01.516578 17681 tablet_bootstrap.cc:492] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:07:01.519273 17681 ts_tablet_manager.cc:1403] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.212s	user 0.017s	sys 0.013s
I20260709 13:07:01.530606 18104 leader_election.cc:290] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:01.507779 17481 raft_consensus.cc:2468] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 1.
I20260709 13:07:01.534089 17482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcc480cc6534780aa5a81e4c97e8bef" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:01.535472 18041 leader_election.cc:290] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:01.553412 17596 leader_election.cc:304] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:01.578115 17681 raft_consensus.cc:359] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:01.579152 17681 raft_consensus.cc:385] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:07:01.579275 17373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 247596us
I20260709 13:07:01.579567 17681 raft_consensus.cc:740] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.563235 18047 raft_consensus.cc:2804] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:01.582361 17681 consensus_queue.cc:260] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:01.575477 17969 raft_consensus.cc:493] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.590063 17678 tablet_bootstrap.cc:492] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:07:01.592401 17678 ts_tablet_manager.cc:1403] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.462s	user 0.011s	sys 0.023s
I20260709 13:07:01.593689 17681 ts_tablet_manager.cc:1434] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.074s	user 0.010s	sys 0.000s
W20260709 13:07:01.575335 17598 leader_election.cc:343] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:01.590790 18047 raft_consensus.cc:697] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 1 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:01.619338 17402 raft_consensus.cc:3060] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:01.602470 17598 leader_election.cc:304] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:01.621191 18160 raft_consensus.cc:2749] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:01.621400 17598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 46136.3us
I20260709 13:07:01.624070 18047 consensus_queue.cc:237] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:01.614257 18038 raft_consensus.cc:493] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.625313 17677 raft_consensus.cc:359] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:01.626161 17677 raft_consensus.cc:385] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:01.625566 18038 raft_consensus.cc:515] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:01.639815 18038 leader_election.cc:290] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:01.596086 17969 raft_consensus.cc:515] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:01.644229 17955 raft_consensus.cc:493] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.637367 17682 tablet_bootstrap.cc:492] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:07:01.644975 17955 raft_consensus.cc:515] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:01.645818 17682 ts_tablet_manager.cc:1403] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.360s	user 0.040s	sys 0.010s
I20260709 13:07:01.647477 17955 leader_election.cc:290] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:01.632062 17677 raft_consensus.cc:740] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.648639 17678 raft_consensus.cc:359] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:01.649627 17678 raft_consensus.cc:385] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:01.650039 17678 raft_consensus.cc:740] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.654898 17678 consensus_queue.cc:260] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:01.601467 17626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05345b561dfb497c8f1e081b05c1bb5d" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:07:01.659447 17375 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 46838.7us
I20260709 13:07:01.650514 17677 consensus_queue.cc:260] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:01.644677 17969 leader_election.cc:290] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:01.677259 17408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def346af33c747faaf6c9cf0414510f0" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:01.675580 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfc142508c684e6eb9716cb26e7047b8" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:01.676335 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfc142508c684e6eb9716cb26e7047b8" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:01.681277 17682 raft_consensus.cc:359] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:01.682237 17682 raft_consensus.cc:385] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:01.682528 17682 raft_consensus.cc:740] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.702283 17678 ts_tablet_manager.cc:1434] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.109s	user 0.009s	sys 0.001s
I20260709 13:07:01.711644 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05345b561dfb497c8f1e081b05c1bb5d" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:01.716571 17480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def346af33c747faaf6c9cf0414510f0" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:01.717618 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c295096a121e4940b38125740cea67cb" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:01.718478 17483 raft_consensus.cc:2468] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:07:01.721272 17677 ts_tablet_manager.cc:1434] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.220s	user 0.010s	sys 0.000s
I20260709 13:07:01.702512 17629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c295096a121e4940b38125740cea67cb" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:01.722894 17677 tablet_bootstrap.cc:492] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:07:01.722955 17629 raft_consensus.cc:2468] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:07:01.701772 17626 raft_consensus.cc:2393] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in this term.
I20260709 13:07:01.724047 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcc480cc6534780aa5a81e4c97e8bef" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:07:01.728503 17596 leader_election.cc:343] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:01.716073 17682 consensus_queue.cc:260] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:01.762036 17682 ts_tablet_manager.cc:1434] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.116s	user 0.005s	sys 0.004s
I20260709 13:07:01.796898 17522 leader_election.cc:304] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
W20260709 13:07:01.797518 17599 leader_election.cc:343] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:01.798079 17599 leader_election.cc:304] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:01.798856 18160 raft_consensus.cc:2749] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:01.798930 17555 raft_consensus.cc:3060] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:01.801072 17678 tablet_bootstrap.cc:492] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada: Bootstrap starting.
I20260709 13:07:01.801717 17522 leader_election.cc:304] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:01.802184 17682 tablet_bootstrap.cc:492] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:01.806919 18105 raft_consensus.cc:2804] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:07:01.810953 17373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 59710.5us
I20260709 13:07:01.812289 17524 leader_election.cc:304] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:01.817054 18115 raft_consensus.cc:493] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.817795 18115 raft_consensus.cc:515] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:01.808873 18105 raft_consensus.cc:493] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.821178 18107 raft_consensus.cc:493] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.825052 18105 raft_consensus.cc:3060] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:01.825256 18107 raft_consensus.cc:515] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:01.827735 18107 leader_election.cc:290] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:01.821233 18115 leader_election.cc:290] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:01.824548 17376 leader_election.cc:304] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:01.830682 17681 tablet_bootstrap.cc:492] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
W20260709 13:07:01.808115 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 56512.6us
W20260709 13:07:01.835002 17450 consensus_peers.cc:597] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:01.833266 17629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:01.845593 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b10c75b663e4cbcb3237b59443f77ca" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:01.860857 18115 raft_consensus.cc:2749] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:01.864435 17451 leader_election.cc:343] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:01.838855 18038 raft_consensus.cc:2749] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:01.873867 17303 catalog_manager.cc:5697] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 0 to 1, leader changed from <none> to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 1 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:01.881254 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b10c75b663e4cbcb3237b59443f77ca" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:01.883093 18149 consensus_queue.cc:1048] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.042s
I20260709 13:07:01.884052 18104 consensus_queue.cc:1048] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:01.889981 17482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:01.898249 17410 raft_consensus.cc:3060] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:01.898897 17373 leader_election.cc:304] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:01.911022 18105 raft_consensus.cc:515] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:01.919175 17482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c295096a121e4940b38125740cea67cb" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:01.919976 17482 raft_consensus.cc:3060] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:01.915949 17450 leader_election.cc:343] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:01.922462 17450 leader_election.cc:304] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:01.925051 18153 raft_consensus.cc:2749] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:01.927201 18105 leader_election.cc:290] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:01.939532 17681 tablet_bootstrap.cc:654] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:01.946799 17677 tablet_bootstrap.cc:654] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:01.956920 17678 tablet_bootstrap.cc:654] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:01.954782 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c295096a121e4940b38125740cea67cb" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:01.973706 17678 tablet_bootstrap.cc:492] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada: No bootstrap required, opened a new log
I20260709 13:07:01.974540 17678 ts_tablet_manager.cc:1403] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada: Time spent bootstrapping tablet: real 0.225s	user 0.012s	sys 0.007s
I20260709 13:07:01.976658 17628 raft_consensus.cc:3060] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:01.982524 17682 tablet_bootstrap.cc:654] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:02.001763 17677 tablet_bootstrap.cc:492] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:07:02.002925 17677 ts_tablet_manager.cc:1403] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.280s	user 0.015s	sys 0.008s
I20260709 13:07:02.006819 17677 raft_consensus.cc:359] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:02.014086 17628 raft_consensus.cc:2468] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:07:02.027252 17677 raft_consensus.cc:385] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:02.028266 17677 raft_consensus.cc:740] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.028796 17482 raft_consensus.cc:2468] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:07:02.033591 17981 raft_consensus.cc:493] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.029721 17677 consensus_queue.cc:260] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:02.034353 17981 raft_consensus.cc:515] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:02.034584 18159 consensus_queue.cc:1048] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:07:02.054132 17955 raft_consensus.cc:2749] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:02.057753 17678 raft_consensus.cc:359] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:02.063474 17678 raft_consensus.cc:385] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:02.059919 17981 leader_election.cc:290] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:02.063802 17402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe7eea0c47a4c6998274f4b6d14dfb7" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:02.063891 17678 raft_consensus.cc:740] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.044466 17524 leader_election.cc:304] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:02.068337 17899 raft_consensus.cc:2804] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:02.068943 17899 raft_consensus.cc:697] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
W20260709 13:07:02.069291 17598 leader_election.cc:343] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:02.071543 17681 tablet_bootstrap.cc:492] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:07:02.072162 17681 ts_tablet_manager.cc:1403] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.242s	user 0.019s	sys 0.001s
I20260709 13:07:02.073187 17678 consensus_queue.cc:260] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:02.069926 17899 consensus_queue.cc:237] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:02.074888 17677 ts_tablet_manager.cc:1434] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.000s
I20260709 13:07:02.083784 17678 ts_tablet_manager.cc:1434] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada: Time spent starting tablet: real 0.109s	user 0.007s	sys 0.000s
I20260709 13:07:02.103278 18115 raft_consensus.cc:2749] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:02.104763 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe7eea0c47a4c6998274f4b6d14dfb7" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:02.103431 18114 raft_consensus.cc:493] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.105851 18114 raft_consensus.cc:515] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:02.104897 17677 tablet_bootstrap.cc:492] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:07:02.112526 17626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:02.113516 17626 raft_consensus.cc:2468] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
I20260709 13:07:02.112555 17406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:02.108276 18114 leader_election.cc:290] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
W20260709 13:07:02.128454 17450 leader_election.cc:343] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:02.133937 17599 leader_election.cc:343] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:02.134461 17599 leader_election.cc:304] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:02.134778 17450 leader_election.cc:304] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:02.151235 17681 raft_consensus.cc:359] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:02.152204 17681 raft_consensus.cc:385] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:02.152515 17681 raft_consensus.cc:740] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.153242 17681 consensus_queue.cc:260] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:02.139966 18160 raft_consensus.cc:2749] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:02.155644 17677 tablet_bootstrap.cc:654] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:02.146992 18153 raft_consensus.cc:2804] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:02.161569 18153 raft_consensus.cc:493] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.162192 18153 raft_consensus.cc:3060] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:02.164676 17681 ts_tablet_manager.cc:1434] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.092s	user 0.007s	sys 0.002s
I20260709 13:07:02.155753 17682 tablet_bootstrap.cc:492] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:07:02.165726 17682 ts_tablet_manager.cc:1403] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.364s	user 0.020s	sys 0.016s
I20260709 13:07:02.166116 17681 tablet_bootstrap.cc:492] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:07:02.188927 17682 raft_consensus.cc:359] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:02.190011 17682 raft_consensus.cc:385] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:02.197332 17682 raft_consensus.cc:740] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.199229 17682 consensus_queue.cc:260] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:02.215649 17682 ts_tablet_manager.cc:1434] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.050s	user 0.005s	sys 0.004s
I20260709 13:07:02.237267 17677 tablet_bootstrap.cc:492] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:07:02.238341 17677 ts_tablet_manager.cc:1403] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.155s	user 0.021s	sys 0.004s
I20260709 13:07:02.244153 18156 raft_consensus.cc:493] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.250000 18105 raft_consensus.cc:493] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.257107 18153 raft_consensus.cc:515] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:02.259933 18153 leader_election.cc:290] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:02.263182 18156 raft_consensus.cc:515] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:02.265383 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:02.266537 17628 raft_consensus.cc:3060] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:02.268437 18187 log.cc:927] Time spent T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:07:02.273118 17981 consensus_queue.cc:1048] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:02.279444 18160 raft_consensus.cc:493] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.280037 18160 raft_consensus.cc:515] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:02.280673 18156 leader_election.cc:290] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:02.282954 18160 leader_election.cc:290] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:02.283313 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307acc498b0c4da7b7953e895b049c2a" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:02.285761 17630 raft_consensus.cc:2468] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:07:02.288458 17375 leader_election.cc:304] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:02.288800 17681 tablet_bootstrap.cc:654] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:02.289777 18156 raft_consensus.cc:2804] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:02.290360 18156 raft_consensus.cc:493] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.290828 18156 raft_consensus.cc:3060] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:02.291667 17403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1995e080f6c048fe935afd95f336b109" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
W20260709 13:07:02.294675 17598 leader_election.cc:343] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:02.296770 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:07:02.299216 17682 tablet_bootstrap.cc:492] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:02.302541 17677 raft_consensus.cc:359] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:02.309844 17677 raft_consensus.cc:385] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:02.310417 17677 raft_consensus.cc:740] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.310443 17303 catalog_manager.cc:5697] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:02.305063 17450 leader_election.cc:343] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:02.302738 18105 raft_consensus.cc:515] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:02.318020 18105 leader_election.cc:290] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:02.311808 17677 consensus_queue.cc:260] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:02.323364 17480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1995e080f6c048fe935afd95f336b109" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:02.324270 17480 raft_consensus.cc:2468] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 0.
I20260709 13:07:02.326328 17596 leader_election.cc:304] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:02.327232 18160 raft_consensus.cc:2804] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:07:02.315240 17355 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:02.327677 18160 raft_consensus.cc:493] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.328595 18160 raft_consensus.cc:3060] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:02.336454 17628 raft_consensus.cc:2468] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:07:02.336513 18156 raft_consensus.cc:515] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:02.340989 17677 ts_tablet_manager.cc:1434] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.102s	user 0.008s	sys 0.000s
I20260709 13:07:02.341791 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307acc498b0c4da7b7953e895b049c2a" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:02.376567 17630 raft_consensus.cc:2468] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:07:02.377887 18156 leader_election.cc:290] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:02.377198 17677 tablet_bootstrap.cc:492] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:07:02.385720 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307acc498b0c4da7b7953e895b049c2a" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:02.386813 17555 raft_consensus.cc:2468] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:07:02.386647 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307acc498b0c4da7b7953e895b049c2a" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:02.388418 17630 raft_consensus.cc:3060] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:02.393780 17682 tablet_bootstrap.cc:654] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:02.345618 18160 raft_consensus.cc:515] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:02.350154 17403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307acc498b0c4da7b7953e895b049c2a" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:02.398842 17403 raft_consensus.cc:2393] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:07:02.354290 17681 tablet_bootstrap.cc:492] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:07:02.400179 17681 ts_tablet_manager.cc:1403] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.234s	user 0.008s	sys 0.007s
I20260709 13:07:02.403895 17681 raft_consensus.cc:359] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:02.404667 17681 raft_consensus.cc:385] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:02.404944 17681 raft_consensus.cc:740] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.405650 17681 consensus_queue.cc:260] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:02.408057 17681 ts_tablet_manager.cc:1434] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:07:02.411902 17681 tablet_bootstrap.cc:492] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:07:02.418221 17406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1995e080f6c048fe935afd95f336b109" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
W20260709 13:07:02.419894 17598 leader_election.cc:343] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:02.423667 17677 tablet_bootstrap.cc:654] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:02.424521 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307acc498b0c4da7b7953e895b049c2a" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:02.425293 17556 raft_consensus.cc:3060] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:02.428051 17450 leader_election.cc:304] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:02.429526 17682 tablet_bootstrap.cc:492] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:07:02.430145 17682 ts_tablet_manager.cc:1403] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.131s	user 0.009s	sys 0.005s
I20260709 13:07:02.432392 17829 raft_consensus.cc:2804] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:02.433122 17829 raft_consensus.cc:697] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:02.433975 17829 consensus_queue.cc:237] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:02.442278 18160 leader_election.cc:290] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:02.443714 17480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1995e080f6c048fe935afd95f336b109" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:02.451120 17480 raft_consensus.cc:3060] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:02.454224 17677 tablet_bootstrap.cc:492] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:07:02.455284 17677 ts_tablet_manager.cc:1403] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.078s	user 0.014s	sys 0.000s
I20260709 13:07:02.458673 17677 raft_consensus.cc:359] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:02.459576 17677 raft_consensus.cc:385] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:02.459926 17677 raft_consensus.cc:740] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.460862 17677 consensus_queue.cc:260] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:02.448163 17682 raft_consensus.cc:359] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:02.462284 17524 leader_election.cc:304] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:02.449402 17556 raft_consensus.cc:2468] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:07:02.462509 17682 raft_consensus.cc:385] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:02.463393 18038 raft_consensus.cc:2764] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:07:02.463495 17682 raft_consensus.cc:740] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.448700 17630 raft_consensus.cc:2468] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:07:02.466375 17677 ts_tablet_manager.cc:1434] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:07:02.466706 17303 catalog_manager.cc:5697] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:02.464396 17682 consensus_queue.cc:260] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:02.468619 17677 tablet_bootstrap.cc:492] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e: Bootstrap starting.
I20260709 13:07:02.466777 17375 leader_election.cc:304] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:02.470149 17682 ts_tablet_manager.cc:1434] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.040s	user 0.004s	sys 0.004s
I20260709 13:07:02.471410 18156 raft_consensus.cc:2804] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:02.471953 18156 raft_consensus.cc:697] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:02.472952 18156 consensus_queue.cc:237] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:02.482694 17677 tablet_bootstrap.cc:654] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:02.485644 17681 tablet_bootstrap.cc:654] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:02.482856 17480 raft_consensus.cc:2468] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 1.
I20260709 13:07:02.487368 17596 leader_election.cc:304] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:02.488309 17682 tablet_bootstrap.cc:492] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:02.493579 18160 raft_consensus.cc:2804] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:02.495882 18160 raft_consensus.cc:697] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [term 1 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:02.504141 17682 tablet_bootstrap.cc:654] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:02.497203 18160 consensus_queue.cc:237] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:02.542757 18038 consensus_queue.cc:1048] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:02.580230 17681 tablet_bootstrap.cc:492] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:07:02.580849 17681 ts_tablet_manager.cc:1403] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.172s	user 0.012s	sys 0.002s
I20260709 13:07:02.595243 17306 catalog_manager.cc:5697] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 0 to 1, leader changed from <none> to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 1 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.604972 17304 catalog_manager.cc:5697] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:02.599175 18041 raft_consensus.cc:493] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.585527 17899 consensus_queue.cc:1048] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:07:02.613658 18041 raft_consensus.cc:515] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:02.625926 17480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b10c75b663e4cbcb3237b59443f77ca" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:02.626834 17480 raft_consensus.cc:2468] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:07:02.629113 17373 leader_election.cc:304] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:02.631624 17677 tablet_bootstrap.cc:492] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e: No bootstrap required, opened a new log
I20260709 13:07:02.632261 17677 ts_tablet_manager.cc:1403] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e: Time spent bootstrapping tablet: real 0.164s	user 0.027s	sys 0.029s
I20260709 13:07:02.633090 18156 raft_consensus.cc:2804] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:02.633306 18041 leader_election.cc:290] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:02.630146 18104 raft_consensus.cc:493] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.638784 18156 raft_consensus.cc:493] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.650439 18156 raft_consensus.cc:3060] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:02.646823 18104 raft_consensus.cc:515] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:02.645815 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b10c75b663e4cbcb3237b59443f77ca" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:02.659973 18192 raft_consensus.cc:493] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.660894 18192 raft_consensus.cc:515] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:02.665166 18104 leader_election.cc:290] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:02.668427 17681 raft_consensus.cc:359] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:02.686264 17681 raft_consensus.cc:385] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:02.686736 17681 raft_consensus.cc:740] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.687562 17681 consensus_queue.cc:260] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:02.690205 17681 ts_tablet_manager.cc:1434] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.109s	user 0.007s	sys 0.001s
I20260709 13:07:02.672204 17677 raft_consensus.cc:359] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:02.685297 17682 tablet_bootstrap.cc:492] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:07:02.727707 17682 ts_tablet_manager.cc:1403] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.240s	user 0.038s	sys 0.029s
I20260709 13:07:02.668968 17406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:02.727767 18156 raft_consensus.cc:515] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:02.690909 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea055e03bf849bea8fdc851e10a8cbe" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:02.729192 17630 raft_consensus.cc:2468] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:07:02.695258 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:02.730895 17681 tablet_bootstrap.cc:492] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:07:02.689985 17555 raft_consensus.cc:2468] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:07:02.698755 17403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea055e03bf849bea8fdc851e10a8cbe" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:02.726936 17677 raft_consensus.cc:385] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:02.733242 17677 raft_consensus.cc:740] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.734174 17677 consensus_queue.cc:260] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:02.735507 18156 leader_election.cc:290] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:02.736683 17677 ts_tablet_manager.cc:1434] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e: Time spent starting tablet: real 0.104s	user 0.007s	sys 0.002s
W20260709 13:07:02.737933 17451 leader_election.cc:343] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:02.737957 17524 leader_election.cc:343] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:02.685755 18192 leader_election.cc:290] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:02.741719 17524 leader_election.cc:304] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:02.744457 18104 raft_consensus.cc:2804] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:02.744948 18104 raft_consensus.cc:493] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.745472 18104 raft_consensus.cc:3060] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:02.744592 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b10c75b663e4cbcb3237b59443f77ca" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:02.747093 17558 raft_consensus.cc:3060] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:02.752457 17682 raft_consensus.cc:359] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:02.753237 17682 raft_consensus.cc:385] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:02.753566 17682 raft_consensus.cc:740] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.753998 17681 tablet_bootstrap.cc:654] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:02.755481 18114 raft_consensus.cc:493] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a9a65d4c22be403682be9bc817650ada)
I20260709 13:07:02.756239 17480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b10c75b663e4cbcb3237b59443f77ca" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:02.756102 18114 raft_consensus.cc:515] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:02.756980 17480 raft_consensus.cc:3060] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:02.759496 17450 leader_election.cc:343] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:02.760085 17450 leader_election.cc:304] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:02.754401 17682 consensus_queue.cc:260] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:02.763352 18192 raft_consensus.cc:2749] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:02.760609 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:02.771246 17554 raft_consensus.cc:2393] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 3: Already voted for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in this term.
I20260709 13:07:02.775837 18114 leader_election.cc:290] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:02.783082 17682 ts_tablet_manager.cc:1434] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.002s
I20260709 13:07:02.785826 17558 raft_consensus.cc:2468] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:07:02.787094 17376 leader_election.cc:304] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:07:02.788635 18156 raft_consensus.cc:2804] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:02.789203 18156 raft_consensus.cc:697] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:02.790311 18156 consensus_queue.cc:237] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:02.790449 18104 raft_consensus.cc:515] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:02.806124 17682 tablet_bootstrap.cc:492] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:02.812857 18104 leader_election.cc:290] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:02.793623 17480 raft_consensus.cc:2468] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:07:02.816841 17403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea055e03bf849bea8fdc851e10a8cbe" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:07:02.823486 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea055e03bf849bea8fdc851e10a8cbe" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:02.824568 17628 raft_consensus.cc:3060] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:02.825040 17682 tablet_bootstrap.cc:654] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:02.831964 18114 raft_consensus.cc:493] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.832659 18114 raft_consensus.cc:515] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:02.835517 18114 leader_election.cc:290] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:02.865155 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:02.866674 17630 raft_consensus.cc:2468] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 3.
I20260709 13:07:02.877197 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfc142508c684e6eb9716cb26e7047b8" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:07:02.897753 17598 consensus_peers.cc:597] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:02.917299 17524 leader_election.cc:343] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:02.951273 17306 catalog_manager.cc:5697] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.954747 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfc142508c684e6eb9716cb26e7047b8" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:02.956132 17630 raft_consensus.cc:2468] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 0.
W20260709 13:07:02.962558 17451 leader_election.cc:343] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:02.967069 17403 raft_consensus.cc:3060] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:02.976649 17681 tablet_bootstrap.cc:492] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:07:02.943092 17628 raft_consensus.cc:2468] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:07:02.977375 17681 ts_tablet_manager.cc:1403] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.247s	user 0.061s	sys 0.101s
I20260709 13:07:02.993500 17681 raft_consensus.cc:359] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:02.994318 17681 raft_consensus.cc:385] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:03.008733 18115 consensus_queue.cc:1048] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:03.011861 17682 tablet_bootstrap.cc:492] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:07:03.012511 17682 ts_tablet_manager.cc:1403] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.207s	user 0.061s	sys 0.063s
I20260709 13:07:03.026911 17682 raft_consensus.cc:359] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:03.027962 17682 raft_consensus.cc:385] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:03.029189 17829 consensus_queue.cc:1048] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:07:03.031889 17682 raft_consensus.cc:740] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.029539 17450 leader_election.cc:304] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:03.033087 17682 consensus_queue.cc:260] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:03.033828 17829 raft_consensus.cc:2804] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:03.034233 17829 raft_consensus.cc:493] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting leader election (detected failure of leader a9a65d4c22be403682be9bc817650ada)
I20260709 13:07:03.034588 17829 raft_consensus.cc:3060] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:03.035105 17682 ts_tablet_manager.cc:1434] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.002s
I20260709 13:07:03.036450 17682 tablet_bootstrap.cc:492] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:03.047529 17681 raft_consensus.cc:740] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.048601 17681 consensus_queue.cc:260] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:03.049615 17682 tablet_bootstrap.cc:654] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:03.023347 17524 leader_election.cc:304] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:03.059945 17450 leader_election.cc:304] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:03.060696 17681 ts_tablet_manager.cc:1434] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.070s	user 0.005s	sys 0.002s
I20260709 13:07:03.062114 18192 raft_consensus.cc:2804] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:03.062702 18192 raft_consensus.cc:493] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.062947 18105 raft_consensus.cc:2804] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:03.063105 18192 raft_consensus.cc:3060] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:03.063639 18105 raft_consensus.cc:697] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:03.064653 18105 consensus_queue.cc:237] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:03.062984 17681 tablet_bootstrap.cc:492] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
W20260709 13:07:03.132064 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 72224.3us
I20260709 13:07:03.134714 18159 consensus_queue.cc:1048] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:03.148447 18172 raft_consensus.cc:493] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.149111 18172 raft_consensus.cc:515] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:03.162845 18115 consensus_queue.cc:1048] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:03.187885 17829 raft_consensus.cc:515] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
W20260709 13:07:03.199939 17522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 56212.3us
I20260709 13:07:03.209241 17681 tablet_bootstrap.cc:654] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:03.211834 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:03.218165 17829 leader_election.cc:290] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:03.216020 17630 raft_consensus.cc:2468] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
W20260709 13:07:03.226361 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 68869us
I20260709 13:07:03.211838 18172 leader_election.cc:290] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:03.239354 17375 leader_election.cc:304] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:03.296486 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:03.253214 18041 raft_consensus.cc:2804] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:03.351291 17628 raft_consensus.cc:2468] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 3.
I20260709 13:07:03.368283 17557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:03.372857 17557 raft_consensus.cc:2393] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4df2ca35be5341afa5a621233a4da94e in current term 3: Already voted for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in this term.
I20260709 13:07:03.375008 17682 tablet_bootstrap.cc:492] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:07:03.375794 17682 ts_tablet_manager.cc:1403] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.340s	user 0.081s	sys 0.070s
I20260709 13:07:03.380570 18192 raft_consensus.cc:515] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:03.381727 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:03.387984 17682 raft_consensus.cc:359] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:03.388944 17682 raft_consensus.cc:385] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:03.389250 17682 raft_consensus.cc:740] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.389338 18218 raft_consensus.cc:493] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.390020 18218 raft_consensus.cc:515] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:03.390187 17682 consensus_queue.cc:260] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:03.392674 18218 leader_election.cc:290] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:03.392802 17682 ts_tablet_manager.cc:1434] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:07:03.397800 17682 tablet_bootstrap.cc:492] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:03.435405 18041 raft_consensus.cc:493] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:03.432833 18219 raft_consensus.cc:493] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:03.457247 17355 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:03.452370 18114 consensus_queue.cc:1048] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:07:03.456176 18041 raft_consensus.cc:3060] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:03.473711 18192 leader_election.cc:290] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:03.469055 17402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:03.482321 17682 tablet_bootstrap.cc:654] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:03.494879 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 206536us
W20260709 13:07:03.491868 18229 log.cc:927] Time spent T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:07:03.515620 17681 tablet_bootstrap.cc:492] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:07:03.516403 17681 ts_tablet_manager.cc:1403] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.454s	user 0.023s	sys 0.006s
W20260709 13:07:03.516054 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 56423.7us
I20260709 13:07:03.467149 18219 raft_consensus.cc:515] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:03.523303 17450 leader_election.cc:304] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:03.523589 17306 catalog_manager.cc:5697] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:03.560421 17681 raft_consensus.cc:359] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:03.561750 17681 raft_consensus.cc:385] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:03.573750 17402 raft_consensus.cc:2468] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:07:03.574970 18219 leader_election.cc:290] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:03.532701 18192 raft_consensus.cc:2804] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:03.577764 18192 raft_consensus.cc:697] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 3 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:03.578917 18192 consensus_queue.cc:237] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:03.582731 17681 raft_consensus.cc:740] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
W20260709 13:07:03.585734 17373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 278886us
I20260709 13:07:03.600107 17681 consensus_queue.cc:260] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:03.604425 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfc142508c684e6eb9716cb26e7047b8" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:03.605453 17558 raft_consensus.cc:3060] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:03.608414 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 70338.1us
I20260709 13:07:03.631458 17480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:03.632354 17480 raft_consensus.cc:2468] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 0.
I20260709 13:07:03.633849 17558 raft_consensus.cc:2468] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:07:03.654242 17481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:03.655227 17481 raft_consensus.cc:2468] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 0.
I20260709 13:07:03.607892 18219 raft_consensus.cc:493] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.663444 17522 leader_election.cc:304] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:03.664113 18219 raft_consensus.cc:515] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:03.666091 18104 raft_consensus.cc:2804] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:03.666837 18104 raft_consensus.cc:493] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.667469 18104 raft_consensus.cc:3060] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:03.683136 17624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfc142508c684e6eb9716cb26e7047b8" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:03.685158 17624 raft_consensus.cc:3060] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:03.687135 18219 leader_election.cc:290] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:03.685493 18041 raft_consensus.cc:515] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:03.700170 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea055e03bf849bea8fdc851e10a8cbe" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:03.701210 17630 raft_consensus.cc:2393] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in this term.
I20260709 13:07:03.714753 17451 leader_election.cc:304] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:03.716369 17829 raft_consensus.cc:2804] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:03.717113 17829 raft_consensus.cc:697] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:03.718991 17829 consensus_queue.cc:237] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:03.723968 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:03.724984 17681 ts_tablet_manager.cc:1434] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.208s	user 0.006s	sys 0.006s
I20260709 13:07:03.738083 17373 leader_election.cc:304] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:03.717067 18041 leader_election.cc:290] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:03.743250 17681 tablet_bootstrap.cc:492] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Bootstrap starting.
I20260709 13:07:03.746500 17410 raft_consensus.cc:1240] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Rejecting Update request from peer 4df2ca35be5341afa5a621233a4da94e for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:03.733922 18232 raft_consensus.cc:493] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.747145 18172 consensus_queue.cc:1048] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:07:03.743409 18235 raft_consensus.cc:2804] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:03.758756 17624 raft_consensus.cc:2468] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:07:03.759334 18235 raft_consensus.cc:493] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.759936 18235 raft_consensus.cc:3060] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:03.775399 18232 raft_consensus.cc:515] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:03.801792 18232 leader_election.cc:290] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:03.807222 17481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:03.785600 17553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea055e03bf849bea8fdc851e10a8cbe" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:03.811296 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:07:03.814841 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 70555.3us
I20260709 13:07:03.823648 18104 raft_consensus.cc:515] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:03.827201 17558 raft_consensus.cc:2393] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in this term.
I20260709 13:07:03.829077 18104 leader_election.cc:290] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:03.836524 17682 tablet_bootstrap.cc:492] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:07:03.837251 17682 ts_tablet_manager.cc:1403] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.440s	user 0.023s	sys 0.011s
I20260709 13:07:03.836974 18235 raft_consensus.cc:515] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:03.838656 17681 tablet_bootstrap.cc:654] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:03.822265 18217 consensus_queue.cc:1048] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:07:03.852560 17829 consensus_queue.cc:1059] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:03.860123 18192 raft_consensus.cc:3055] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:03.861829 18192 raft_consensus.cc:740] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:03.862839 18192 consensus_queue.cc:260] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:03.864207 18192 raft_consensus.cc:3060] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:03.864804 17682 raft_consensus.cc:359] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:03.865648 17682 raft_consensus.cc:385] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:03.865933 17682 raft_consensus.cc:740] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.866784 17682 consensus_queue.cc:260] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:03.868537 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:03.872746 17480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:03.845317 18235 leader_election.cc:290] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:03.913381 17482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:03.917155 17682 ts_tablet_manager.cc:1434] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.080s	user 0.016s	sys 0.000s
I20260709 13:07:03.918533 17682 tablet_bootstrap.cc:492] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:07:03.919847 18245 log.cc:927] Time spent T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e: Append to log took a long time: real 0.063s	user 0.002s	sys 0.000s
I20260709 13:07:03.921020 17480 raft_consensus.cc:3060] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:03.928185 18219 raft_consensus.cc:493] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.930292 18219 raft_consensus.cc:515] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:03.934027 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1995e080f6c048fe935afd95f336b109" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:03.937825 17681 tablet_bootstrap.cc:492] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: No bootstrap required, opened a new log
I20260709 13:07:03.938560 17681 ts_tablet_manager.cc:1403] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent bootstrapping tablet: real 0.200s	user 0.018s	sys 0.003s
I20260709 13:07:03.941927 17681 raft_consensus.cc:359] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:03.942893 17681 raft_consensus.cc:385] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:03.943207 17681 raft_consensus.cc:740] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.944044 17681 consensus_queue.cc:260] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:03.945885 17681 ts_tablet_manager.cc:1434] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:07:03.949187 17682 tablet_bootstrap.cc:654] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:03.939436 18219 leader_election.cc:290] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:03.960105 17304 catalog_manager.cc:5697] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 2 to 3, leader changed from a9a65d4c22be403682be9bc817650ada (127.7.89.132) to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 3 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:03.965579 17402 raft_consensus.cc:3060] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:03.972311 17304 catalog_manager.cc:5697] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:03.962494 17485 raft_consensus.cc:2468] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:03.983304 17482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1995e080f6c048fe935afd95f336b109" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:03.995579 18219 consensus_queue.cc:1048] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:03.984009 17682 tablet_bootstrap.cc:492] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c: No bootstrap required, opened a new log
I20260709 13:07:03.997272 17682 ts_tablet_manager.cc:1403] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.079s	user 0.016s	sys 0.005s
I20260709 13:07:03.998849 17480 raft_consensus.cc:2468] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:07:04.001017 17682 raft_consensus.cc:359] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:04.001994 17682 raft_consensus.cc:385] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:04.002292 17682 raft_consensus.cc:740] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.016829 17682 consensus_queue.cc:260] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:04.023805 17402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:07:04.024876 17402 raft_consensus.cc:2393] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:07:04.027305 17376 leader_election.cc:304] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:04.028189 18235 raft_consensus.cc:2749] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:04.031615 17682 ts_tablet_manager.cc:1434] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:07:04.038034 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:04.039049 17625 raft_consensus.cc:2410] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:07:04.051559 17373 leader_election.cc:304] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:04.052640 18235 raft_consensus.cc:2749] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:04.062110 17373 leader_election.cc:304] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: a9a65d4c22be403682be9bc817650ada
I20260709 13:07:04.072736 18235 raft_consensus.cc:2804] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:04.076308 18235 raft_consensus.cc:697] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:04.077440 18235 consensus_queue.cc:237] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:04.074157 17373 leader_election.cc:304] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:04.092602 17307 catalog_manager.cc:5697] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 2 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:04.093816 18219 raft_consensus.cc:2804] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:04.094440 18219 raft_consensus.cc:697] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:04.095397 18219 consensus_queue.cc:237] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:04.110790 17553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:04.111864 17553 raft_consensus.cc:2393] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 1: Already voted for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in this term.
I20260709 13:07:04.135568 17557 raft_consensus.cc:1275] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Refusing update from remote peer 4df2ca35be5341afa5a621233a4da94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:04.143678 17304 catalog_manager.cc:5697] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 0 to 1, leader changed from <none> to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 1 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:04.146421 18211 raft_consensus.cc:493] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.147189 18211 raft_consensus.cc:515] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:04.160818 18211 leader_election.cc:290] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:04.166339 18230 raft_consensus.cc:493] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.173998 17402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe7eea0c47a4c6998274f4b6d14dfb7" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:04.184500 17402 raft_consensus.cc:2468] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 0.
I20260709 13:07:04.184350 18230 raft_consensus.cc:515] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:04.186367 17598 leader_election.cc:304] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:04.159648 17625 raft_consensus.cc:1275] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Refusing update from remote peer 4df2ca35be5341afa5a621233a4da94e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:04.189028 18211 raft_consensus.cc:2804] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:04.189688 18211 raft_consensus.cc:493] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.190107 18211 raft_consensus.cc:3060] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:04.190688 18230 leader_election.cc:290] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:04.195379 18239 consensus_queue.cc:1048] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:04.259774 18217 consensus_queue.cc:1048] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:04.264923 18248 consensus_queue.cc:1048] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:04.265815 18192 consensus_queue.cc:1048] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:04.257107 17524 leader_election.cc:304] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:04.278134 18104 raft_consensus.cc:2749] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:04.300954 17524 leader_election.cc:304] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:04.303923 18232 raft_consensus.cc:2749] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:04.284382 18211 raft_consensus.cc:515] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:04.317183 18211 leader_election.cc:290] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:04.320000 17403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe7eea0c47a4c6998274f4b6d14dfb7" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:07:04.320879 17403 raft_consensus.cc:3060] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:04.324963 17402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe7eea0c47a4c6998274f4b6d14dfb7" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:04.340871 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe7eea0c47a4c6998274f4b6d14dfb7" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:04.341990 17625 raft_consensus.cc:2393] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 1: Already voted for candidate a9a65d4c22be403682be9bc817650ada in this term.
I20260709 13:07:04.349479 17403 raft_consensus.cc:2468] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 1.
I20260709 13:07:04.365403 17551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe7eea0c47a4c6998274f4b6d14dfb7" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:04.351752 17598 leader_election.cc:304] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:04.380785 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe7eea0c47a4c6998274f4b6d14dfb7" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:04.374593 17551 raft_consensus.cc:3060] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:04.377063 18160 raft_consensus.cc:2804] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:04.388499 18160 raft_consensus.cc:697] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 1 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:04.389552 18160 consensus_queue.cc:237] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:04.390872 17524 leader_election.cc:304] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:04.396221 18247 consensus_queue.cc:1048] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:07:04.420588 17410 raft_consensus.cc:3060] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:04.449656 17551 raft_consensus.cc:2468] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 1.
I20260709 13:07:04.450795 18149 raft_consensus.cc:2749] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:04.499627 17480 raft_consensus.cc:1275] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:04.501941 18115 consensus_queue.cc:1048] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:04.508397 17599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 114400us
I20260709 13:07:04.572661 17624 raft_consensus.cc:3060] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:04.647969 17624 raft_consensus.cc:1275] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:04.650161 18235 consensus_queue.cc:1048] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:04.655470 17303 catalog_manager.cc:5697] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 0 to 1, leader changed from <none> to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 1 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:04.681967 18235 consensus_queue.cc:1048] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:04.698231 18231 consensus_queue.cc:1048] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:04.910161 18211 consensus_queue.cc:1048] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:04.930042 18267 consensus_queue.cc:1048] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:05.965070  7526 tablet_server.cc:179] TabletServer@127.7.89.129:0 shutting down...
I20260709 13:07:06.037410  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:06.041695  7526 tablet_replica.cc:333] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.043099  7526 raft_consensus.cc:2243] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.044682  7526 raft_consensus.cc:2272] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.048321  7526 tablet_replica.cc:333] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.049540  7526 raft_consensus.cc:2243] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.050396  7526 raft_consensus.cc:2272] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:06.053409 17524 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
I20260709 13:07:06.054288  7526 tablet_replica.cc:333] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.055711  7526 raft_consensus.cc:2243] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.056564  7526 raft_consensus.cc:2272] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.060137  7526 tablet_replica.cc:333] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.061635  7526 raft_consensus.cc:2243] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.062492  7526 raft_consensus.cc:2272] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.066215  7526 tablet_replica.cc:333] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.067525  7526 raft_consensus.cc:2243] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.068434  7526 raft_consensus.cc:2272] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.072290  7526 tablet_replica.cc:333] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.073755  7526 raft_consensus.cc:2243] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.075271  7526 raft_consensus.cc:2272] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:06.077803 17450 consensus_peers.cc:597] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.080403  7526 tablet_replica.cc:333] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
W20260709 13:07:06.081355 17450 consensus_peers.cc:597] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.081795  7526 raft_consensus.cc:2243] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.083972  7526 raft_consensus.cc:2272] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:06.084767 17524 consensus_peers.cc:597] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.088932  7526 tablet_replica.cc:333] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.090340  7526 raft_consensus.cc:2243] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.091298  7526 raft_consensus.cc:2272] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.097463  7526 tablet_replica.cc:333] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.098758  7526 raft_consensus.cc:2243] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.099602  7526 raft_consensus.cc:2272] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.103390  7526 tablet_replica.cc:333] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.106093  7526 raft_consensus.cc:2243] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.107054  7526 raft_consensus.cc:2272] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.111266  7526 tablet_replica.cc:333] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.112581  7526 raft_consensus.cc:2243] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.113324  7526 raft_consensus.cc:2272] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.116997  7526 tablet_replica.cc:333] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.118142  7526 raft_consensus.cc:2243] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.119377  7526 raft_consensus.cc:2272] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:06.123183 17598 consensus_peers.cc:597] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.123235  7526 tablet_replica.cc:333] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.125546  7526 raft_consensus.cc:2243] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:06.126379 17524 consensus_peers.cc:597] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.126402  7526 raft_consensus.cc:2272] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:06.127486 17524 consensus_peers.cc:597] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.130939  7526 tablet_replica.cc:333] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.131843  7526 raft_consensus.cc:2243] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:06.131875 17450 consensus_peers.cc:597] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.132546  7526 raft_consensus.cc:2272] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.135576  7526 tablet_replica.cc:333] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.136512  7526 raft_consensus.cc:2243] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.137678  7526 raft_consensus.cc:2272] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.140575  7526 tablet_replica.cc:333] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.141549  7526 raft_consensus.cc:2243] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.142146  7526 raft_consensus.cc:2272] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.144912  7526 tablet_replica.cc:333] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
W20260709 13:07:06.145557 17450 consensus_peers.cc:597] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.145852  7526 raft_consensus.cc:2243] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.147131  7526 raft_consensus.cc:2272] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.149724  7526 tablet_replica.cc:333] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.150544  7526 raft_consensus.cc:2243] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.151845  7526 raft_consensus.cc:2272] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.155051  7526 tablet_replica.cc:333] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.156188  7526 raft_consensus.cc:2243] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.156903  7526 raft_consensus.cc:2272] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.160363  7526 tablet_replica.cc:333] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.161449  7526 raft_consensus.cc:2243] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.162744  7526 raft_consensus.cc:2272] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.165807  7526 tablet_replica.cc:333] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.166755  7526 raft_consensus.cc:2243] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.167353  7526 raft_consensus.cc:2272] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.170025  7526 tablet_replica.cc:333] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.170814  7526 raft_consensus.cc:2243] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.171373  7526 raft_consensus.cc:2272] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.173971  7526 tablet_replica.cc:333] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.174830  7526 raft_consensus.cc:2243] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.176018  7526 raft_consensus.cc:2272] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.179376  7526 tablet_replica.cc:333] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
W20260709 13:07:06.179875 17524 consensus_peers.cc:597] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.180264  7526 raft_consensus.cc:2243] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.183398  7526 raft_consensus.cc:2272] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.187057  7526 tablet_replica.cc:333] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.188066  7526 raft_consensus.cc:2243] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.188747  7526 raft_consensus.cc:2272] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:06.189157 17450 consensus_peers.cc:597] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.192121  7526 tablet_replica.cc:333] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
W20260709 13:07:06.192276 17598 consensus_peers.cc:597] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.193315  7526 raft_consensus.cc:2243] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.194442  7526 raft_consensus.cc:2272] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:06.195673 17524 consensus_peers.cc:597] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.197428  7526 tablet_replica.cc:333] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.198294  7526 raft_consensus.cc:2243] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.198885  7526 raft_consensus.cc:2272] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.201735  7526 tablet_replica.cc:333] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.202690  7526 raft_consensus.cc:2243] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.203359  7526 raft_consensus.cc:2272] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.206943  7526 tablet_replica.cc:333] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.207842  7526 raft_consensus.cc:2243] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.208438  7526 raft_consensus.cc:2272] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.211334  7526 tablet_replica.cc:333] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.212235  7526 raft_consensus.cc:2243] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.213483  7526 raft_consensus.cc:2272] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.225616  7526 tablet_replica.cc:333] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.226349  7526 raft_consensus.cc:2243] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.227095  7526 raft_consensus.cc:2272] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.230268  7526 tablet_replica.cc:333] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.231324  7526 raft_consensus.cc:2243] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.233184  7526 raft_consensus.cc:2272] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:06.212237 17524 consensus_peers.cc:597] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.236622  7526 tablet_replica.cc:333] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.237711  7526 raft_consensus.cc:2243] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.238370  7526 raft_consensus.cc:2272] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.242103  7526 tablet_replica.cc:333] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.243204  7526 raft_consensus.cc:2243] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.243876  7526 raft_consensus.cc:2272] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.247219  7526 tablet_replica.cc:333] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.248287  7526 raft_consensus.cc:2243] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.250284  7526 raft_consensus.cc:2272] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:06.251036 17450 consensus_peers.cc:597] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.254088  7526 tablet_replica.cc:333] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.255257  7526 raft_consensus.cc:2243] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.255926  7526 raft_consensus.cc:2272] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.260494  7526 tablet_replica.cc:333] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.261255  7526 raft_consensus.cc:2243] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.261895  7526 raft_consensus.cc:2272] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.264662  7526 tablet_replica.cc:333] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.265343  7526 raft_consensus.cc:2243] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.266266  7526 raft_consensus.cc:2272] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.268580  7526 tablet_replica.cc:333] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.269244  7526 raft_consensus.cc:2243] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.270265  7526 raft_consensus.cc:2272] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.274236  7526 tablet_replica.cc:333] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.275209  7526 raft_consensus.cc:2243] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.276437  7526 raft_consensus.cc:2272] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.279613  7526 tablet_replica.cc:333] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.280263  7526 raft_consensus.cc:2243] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.280771  7526 raft_consensus.cc:2272] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.283316  7526 tablet_replica.cc:333] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.283959  7526 raft_consensus.cc:2243] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.285197  7526 raft_consensus.cc:2272] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.288700  7526 tablet_replica.cc:333] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.289561  7526 raft_consensus.cc:2243] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:06.290904  7526 raft_consensus.cc:2272] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.294327  7526 tablet_replica.cc:333] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
I20260709 13:07:06.295179  7526 raft_consensus.cc:2243] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.295778  7526 raft_consensus.cc:2272] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.299108  7526 tablet_replica.cc:333] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c: stopping tablet replica
W20260709 13:07:06.299767 17598 consensus_peers.cc:597] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:06.301621 17598 consensus_peers.cc:597] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.300010  7526 raft_consensus.cc:2243] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.302642  7526 raft_consensus.cc:2272] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:06.305143 17524 consensus_peers.cc:597] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:06.323897 17450 consensus_peers.cc:597] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:06.344785 17450 consensus_peers.cc:597] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:06.356344 17524 consensus_peers.cc:597] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:06.359277 17450 consensus_peers.cc:597] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:06.360167 17450 consensus_peers.cc:597] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:06.389599 17598 consensus_peers.cc:597] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.403869  7526 tablet_server.cc:196] TabletServer@127.7.89.129:0 shutdown complete.
W20260709 13:07:06.406420 17450 consensus_peers.cc:597] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:06.452095 17450 consensus_peers.cc:597] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:06.503834 17524 consensus_peers.cc:597] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:06.524008 17598 consensus_peers.cc:597] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:06.531951 17450 consensus_peers.cc:597] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:06.554565  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:06.582288 18287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:06.590372 18288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:06.598819 18290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:06.601303  7526 server_base.cc:1061] running on GCE node
I20260709 13:07:06.611491  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:06.611944  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:06.612197  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602426612176 us; error 0 us; skew 500 ppm
I20260709 13:07:06.612869  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:06.619331  7526 webserver.cc:533] Webserver started at http://127.7.89.129:40525/ using document root <none> and password file <none>
I20260709 13:07:06.620175  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:06.620461  7526 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:07:06.625703  7526 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:07:06.649359 18295 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:06.651309  7526 fs_manager.cc:730] Time spent opening block manager: real 0.016s	user 0.004s	sys 0.000s
I20260709 13:07:06.656000  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-0-root
uuid: "337bbc857b9b4e329c817b81bfb0971c"
format_stamp: "Formatted at 2026-07-09 13:06:48 on dist-test-slave-h0rz"
I20260709 13:07:06.657070  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602061986899-7526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:06.689906  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:06.692106  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:06.696113  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:06.782888 18304 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 13:07:07.185281 18281 raft_consensus.cc:493] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.186944 18281 raft_consensus.cc:515] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:07.202546 18285 raft_consensus.cc:493] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.203575 18285 raft_consensus.cc:515] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.204730 18281 leader_election.cc:290] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:07.206156 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:07.210665 18217 raft_consensus.cc:493] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.211609 18217 raft_consensus.cc:515] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.216948 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:07.217299 18217 leader_election.cc:290] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:07.226881 18285 leader_election.cc:290] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
W20260709 13:07:07.218750 17598 leader_election.cc:336] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.229830 17598 leader_election.cc:304] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:07.231033 18281 raft_consensus.cc:2749] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:07.229120 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20df005b7408432da9ac4fcf110a20b9" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:07.239603 18302 raft_consensus.cc:493] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.240564 18302 raft_consensus.cc:515] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.245824 18230 raft_consensus.cc:493] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.246781 18230 raft_consensus.cc:515] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.246346 18302 leader_election.cc:290] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:07.255944 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def346af33c747faaf6c9cf0414510f0" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:07.262133 18230 leader_election.cc:290] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
W20260709 13:07:07.263798 17450 leader_election.cc:336] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.266741 18306 raft_consensus.cc:493] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.267521 18306 raft_consensus.cc:515] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.270572 18306 leader_election.cc:290] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
W20260709 13:07:07.274951 17524 leader_election.cc:336] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.275465 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:07.276217 17551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a74eee34cc514b79b981a6ef6c45d7e2" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:07.276383 17556 raft_consensus.cc:2468] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:07:07.275641 17524 leader_election.cc:304] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:07.277794 18217 raft_consensus.cc:2749] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:07:07.278352 17524 leader_election.cc:336] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.278945 17524 leader_election.cc:304] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:07.279978 18230 raft_consensus.cc:2749] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:07:07.282424 17450 leader_election.cc:336] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.283538 17451 leader_election.cc:304] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:07.284315 18306 raft_consensus.cc:2749] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:07.285277 17451 leader_election.cc:304] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.286056 18306 raft_consensus.cc:2804] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.286755 18306 raft_consensus.cc:493] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
W20260709 13:07:07.288393 17524 leader_election.cc:336] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.288410 18306 raft_consensus.cc:3060] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.289173 17524 leader_election.cc:304] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:07.290254 18217 raft_consensus.cc:2749] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:07.302117 18302 raft_consensus.cc:493] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.302225 18306 raft_consensus.cc:515] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.303011 18302 raft_consensus.cc:515] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:07.305989 18302 leader_election.cc:290] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.308939 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:07.309842 17632 raft_consensus.cc:2468] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:07:07.312069 17450 leader_election.cc:304] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
W20260709 13:07:07.313894 17450 leader_election.cc:336] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.314231 18286 raft_consensus.cc:2804] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:07:07.314688 17450 leader_election.cc:336] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.314868 18286 raft_consensus.cc:493] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.315470 18286 raft_consensus.cc:3060] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.325606 18217 raft_consensus.cc:493] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.326654 18217 raft_consensus.cc:515] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.327883 18306 leader_election.cc:290] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:07.329722 18217 leader_election.cc:290] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.329499 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:07.330420 17556 raft_consensus.cc:3060] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.334709 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307acc498b0c4da7b7953e895b049c2a" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:07.333870 18286 raft_consensus.cc:515] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:07.337625 18286 leader_election.cc:290] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.341374 17629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:07.342247 17629 raft_consensus.cc:3060] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:07.344916 17450 leader_election.cc:336] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.358852 18306 raft_consensus.cc:493] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.341625 17556 raft_consensus.cc:2468] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 2.
I20260709 13:07:07.360167 18306 raft_consensus.cc:515] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
W20260709 13:07:07.360224 17524 leader_election.cc:336] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.362154 17524 leader_election.cc:304] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:07.363080 18217 raft_consensus.cc:2749] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:07.366356 17629 raft_consensus.cc:2468] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 2.
W20260709 13:07:07.368610 17450 leader_election.cc:336] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.369477 18306 leader_election.cc:290] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:07.369673 18217 raft_consensus.cc:493] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.370035 17450 leader_election.cc:304] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.370580 18217 raft_consensus.cc:515] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.371165 18302 raft_consensus.cc:2804] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:07.371794 18302 raft_consensus.cc:697] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:07.372404 17451 leader_election.cc:304] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.373123 18302 consensus_queue.cc:237] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:07.374344 18306 raft_consensus.cc:2804] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:07.375084 18306 raft_consensus.cc:697] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:07.376313 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b10c75b663e4cbcb3237b59443f77ca" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:07.378144 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20df005b7408432da9ac4fcf110a20b9" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:07.378513 18217 leader_election.cc:290] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:07.378940 17556 raft_consensus.cc:2468] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:07:07.380551 18220 raft_consensus.cc:493] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.376315 18306 consensus_queue.cc:237] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.381610 18220 raft_consensus.cc:515] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.384913 18217 raft_consensus.cc:493] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.385777 18217 raft_consensus.cc:515] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.387727 17451 leader_election.cc:304] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.388320 18217 leader_election.cc:290] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.388731 18302 raft_consensus.cc:2804] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.389235 18302 raft_consensus.cc:493] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.389681 18302 raft_consensus.cc:3060] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.394976 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b1eabbf0fc942fabea0c59e6997ad23" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:07.396689 18192 raft_consensus.cc:493] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.397429 18192 raft_consensus.cc:515] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:07.399066 17629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61235f863ca44ed3b6c4715dfd68e223" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:07.399755 18192 leader_election.cc:290] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
W20260709 13:07:07.394305 17598 leader_election.cc:336] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
W20260709 13:07:07.401518 17524 leader_election.cc:336] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.401829 18220 leader_election.cc:290] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.402145 17524 leader_election.cc:304] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:07.403424 18217 raft_consensus.cc:2749] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:07.406177 17599 leader_election.cc:304] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:07.409740 17304 catalog_manager.cc:5697] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 2 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:07.414328 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:07.415648 18302 raft_consensus.cc:515] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.418207 18114 raft_consensus.cc:493] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.418668 18302 leader_election.cc:290] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:07.419126 18114 raft_consensus.cc:515] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.421659 18114 leader_election.cc:290] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:07.422448 18281 raft_consensus.cc:2749] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:07:07.423743 17524 leader_election.cc:336] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.426257 17524 leader_election.cc:304] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:07.434950 18217 raft_consensus.cc:2749] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:07.435809 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20df005b7408432da9ac4fcf110a20b9" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:07.436221 18281 raft_consensus.cc:493] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.436636 17551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7e0518788149c19cf493a65bc598c2" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:07.436965 18281 raft_consensus.cc:515] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.438881 18283 raft_consensus.cc:493] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.439652 18283 raft_consensus.cc:515] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.436692 17556 raft_consensus.cc:3060] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.448395 18281 leader_election.cc:290] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.442540 18283 leader_election.cc:290] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.451025 18302 raft_consensus.cc:493] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
W20260709 13:07:07.451218 17598 leader_election.cc:336] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.451964 18302 raft_consensus.cc:515] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.457990 18302 leader_election.cc:290] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:07.458405 18230 raft_consensus.cc:493] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.459271 18230 raft_consensus.cc:515] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
W20260709 13:07:07.461040 17450 leader_election.cc:336] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.461584 18217 raft_consensus.cc:493] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.461731 17450 leader_election.cc:304] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:07.462656 18302 raft_consensus.cc:2749] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:07.462409 18217 raft_consensus.cc:515] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
W20260709 13:07:07.460793 17598 leader_election.cc:336] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.466373 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:07.468983 18230 leader_election.cc:290] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
W20260709 13:07:07.469136 17450 leader_election.cc:336] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
W20260709 13:07:07.471807 17450 leader_election.cc:336] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.475055 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7e0518788149c19cf493a65bc598c2" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:07.475952 18217 leader_election.cc:290] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:07.475997 17485 raft_consensus.cc:2468] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:07:07.476652 18285 raft_consensus.cc:493] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.477427 18285 raft_consensus.cc:515] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.478029 17522 leader_election.cc:304] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:07.480396 18230 raft_consensus.cc:2804] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.480880 18230 raft_consensus.cc:493] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.481295 18230 raft_consensus.cc:3060] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:07.482522 17450 leader_election.cc:336] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.484544 18285 leader_election.cc:290] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.476873 18217 raft_consensus.cc:493] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.485538 18217 raft_consensus.cc:515] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.488173 18217 leader_election.cc:290] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.491272 17556 raft_consensus.cc:2468] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 2.
I20260709 13:07:07.494129 17451 leader_election.cc:304] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:07.496887 18192 raft_consensus.cc:2749] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:07.496649 18230 raft_consensus.cc:515] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.500020 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34919e11221f465cbcef1151a1c5d4b0" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:07.501674 17551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8bf7f6c34a4edfb28a4914ad4ce49e" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:07.501888 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b1eabbf0fc942fabea0c59e6997ad23" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:07.502678 17632 raft_consensus.cc:2468] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:07:07.504613 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61235f863ca44ed3b6c4715dfd68e223" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:07.505443 17556 raft_consensus.cc:2468] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 1.
I20260709 13:07:07.507373 17599 leader_election.cc:304] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:07.508555 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def346af33c747faaf6c9cf0414510f0" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:07.509368 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7e0518788149c19cf493a65bc598c2" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:07.510080 17485 raft_consensus.cc:3060] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.514837 18283 raft_consensus.cc:2749] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:07.509409 17554 raft_consensus.cc:2468] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:07:07.520263 17451 leader_election.cc:304] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.524909 18192 raft_consensus.cc:2804] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:07.523306 18283 raft_consensus.cc:493] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.525663 18192 raft_consensus.cc:697] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:07.525786 18283 raft_consensus.cc:515] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.528625 18192 consensus_queue.cc:237] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.520337 18230 leader_election.cc:290] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:07.535210 18302 raft_consensus.cc:493] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.536254 18302 raft_consensus.cc:515] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:07.539529 18302 leader_election.cc:290] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
W20260709 13:07:07.540334 17598 leader_election.cc:336] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.532985 18283 leader_election.cc:290] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.542088 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f330683b8d4b0696fad129a8f2ce23" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:07.543203 17485 raft_consensus.cc:2468] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 2.
I20260709 13:07:07.544746 18281 raft_consensus.cc:493] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.545544 18281 raft_consensus.cc:515] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.548322 18281 leader_election.cc:290] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
W20260709 13:07:07.558339 17598 leader_election.cc:336] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.558983 18230 raft_consensus.cc:493] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
W20260709 13:07:07.559489 17450 leader_election.cc:336] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.560112 17450 leader_election.cc:304] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:07.560074 18230 raft_consensus.cc:515] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.565161 18302 raft_consensus.cc:2749] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:07.565361 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a74eee34cc514b79b981a6ef6c45d7e2" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:07.566088 18230 leader_election.cc:290] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:07.568812 17524 leader_election.cc:304] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:07.568959 17522 leader_election.cc:304] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:07.570017 18285 raft_consensus.cc:2804] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.570428 18285 raft_consensus.cc:493] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.570454 18217 raft_consensus.cc:2804] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:07.570983 18285 raft_consensus.cc:3060] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.571447 18217 raft_consensus.cc:697] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:07.572783 18217 consensus_queue.cc:237] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.566282 17483 raft_consensus.cc:2468] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:07:07.581286 17522 leader_election.cc:304] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:07.582836 18230 raft_consensus.cc:2804] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.583308 18230 raft_consensus.cc:493] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.583729 18230 raft_consensus.cc:3060] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.584263 18283 raft_consensus.cc:493] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.586501 17599 leader_election.cc:304] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.590721 18281 raft_consensus.cc:2804] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.589648 18283 raft_consensus.cc:515] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:07:07.591951 17524 leader_election.cc:336] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.591398 18281 raft_consensus.cc:493] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.592643 17524 leader_election.cc:304] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:07.592751 18281 raft_consensus.cc:3060] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:07.593912 17524 leader_election.cc:336] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.593747 18312 raft_consensus.cc:2749] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:07:07.594822 17524 leader_election.cc:336] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.595337 17524 leader_election.cc:304] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:07.597025 18283 leader_election.cc:290] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.597759 18312 raft_consensus.cc:2749] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:07:07.596360 17524 leader_election.cc:336] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
W20260709 13:07:07.605183 17598 leader_election.cc:336] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.605801 18192 raft_consensus.cc:493] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.606492 18192 raft_consensus.cc:515] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.607129 17451 leader_election.cc:304] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.608457 18286 raft_consensus.cc:2804] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.608868 18286 raft_consensus.cc:493] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.608946 18192 leader_election.cc:290] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:07.609336 18286 raft_consensus.cc:3060] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.610654 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e84a923caa9c4755bcb78a35143dfbd9" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:07.612282 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307acc498b0c4da7b7953e895b049c2a" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:07.613005 17558 raft_consensus.cc:2468] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 1.
W20260709 13:07:07.613227 17524 leader_election.cc:336] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.614230 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:07.617753 17599 leader_election.cc:304] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:07.620645 17599 leader_election.cc:304] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.621819 18312 raft_consensus.cc:493] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.622231 18220 raft_consensus.cc:2749] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:07.623138 18220 raft_consensus.cc:2804] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.623662 18220 raft_consensus.cc:493] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.624153 18220 raft_consensus.cc:3060] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.623348 18312 raft_consensus.cc:515] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.609871 18217 raft_consensus.cc:493] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.628932 18312 leader_election.cc:290] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.627039 18217 raft_consensus.cc:515] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.632507 17304 catalog_manager.cc:5697] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:07.635819 17552 raft_consensus.cc:2468] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 1.
I20260709 13:07:07.637305 18217 leader_election.cc:290] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.645227 17599 leader_election.cc:304] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.646049 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:07.647080 17632 raft_consensus.cc:2468] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:07:07.648434 18285 raft_consensus.cc:515] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.649469 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e84a923caa9c4755bcb78a35143dfbd9" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:07.650926 18285 leader_election.cc:290] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.650436 17625 raft_consensus.cc:2468] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:07:07.655438 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b10c75b663e4cbcb3237b59443f77ca" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:07.656682 17552 raft_consensus.cc:2468] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:07:07.657714 18192 raft_consensus.cc:493] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.658531 18192 raft_consensus.cc:515] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.663470 18192 leader_election.cc:290] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:07.666934 18283 raft_consensus.cc:2804] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.667557 18283 raft_consensus.cc:493] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.668082 18283 raft_consensus.cc:3060] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.673235 18286 raft_consensus.cc:515] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.676306 18319 raft_consensus.cc:493] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.678964 18319 raft_consensus.cc:515] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:07:07.688503 17524 leader_election.cc:336] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
W20260709 13:07:07.691226 17598 leader_election.cc:336] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.678124 18220 raft_consensus.cc:515] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.701560 18318 raft_consensus.cc:493] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.696425 18230 raft_consensus.cc:515] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.705766 17451 leader_election.cc:304] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:07.708211 18220 leader_election.cc:290] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.709022 18220 raft_consensus.cc:493] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.710171 18192 raft_consensus.cc:2804] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.711691 18192 raft_consensus.cc:493] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.712095 18192 raft_consensus.cc:3060] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.702417 18318 raft_consensus.cc:515] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:07.714720 18318 leader_election.cc:290] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:07.715404 18319 leader_election.cc:290] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:07.717877 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a74eee34cc514b79b981a6ef6c45d7e2" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:07.718904 17485 raft_consensus.cc:3060] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.725162 18283 raft_consensus.cc:515] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.720211 18281 raft_consensus.cc:515] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.722074 18286 leader_election.cc:290] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:07.722862 18230 leader_election.cc:290] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:07.727283 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:07.728049 17552 raft_consensus.cc:2468] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 1.
I20260709 13:07:07.709719 18220 raft_consensus.cc:515] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:07:07.730254 17598 leader_election.cc:336] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
W20260709 13:07:07.732973 17598 leader_election.cc:336] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.732187 17524 leader_election.cc:304] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:07.735311 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b1eabbf0fc942fabea0c59e6997ad23" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:07.736039 17632 raft_consensus.cc:3060] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.736263 17480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f330683b8d4b0696fad129a8f2ce23" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:07.737015 17480 raft_consensus.cc:2468] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 1.
I20260709 13:07:07.738044 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:07.738797 17483 raft_consensus.cc:2468] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:07.740860 17524 leader_election.cc:304] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:07.736275 17553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def346af33c747faaf6c9cf0414510f0" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
W20260709 13:07:07.741786 17524 leader_election.cc:336] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.743542 17553 raft_consensus.cc:3060] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:07.744081 17524 leader_election.cc:336] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.742753 18230 raft_consensus.cc:2804] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.747264 18230 raft_consensus.cc:493] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.747653 18230 raft_consensus.cc:3060] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.741653 18317 raft_consensus.cc:2804] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.740396 17304 catalog_manager.cc:5697] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 2 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:07.750466 18317 raft_consensus.cc:493] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.751065 18317 raft_consensus.cc:3060] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.751291 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307acc498b0c4da7b7953e895b049c2a" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:07.752076 17552 raft_consensus.cc:3060] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:07.752247 17524 leader_election.cc:336] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.754078 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61235f863ca44ed3b6c4715dfd68e223" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:07.754863 17554 raft_consensus.cc:3060] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:07.757228 17524 leader_election.cc:336] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.761883 17596 leader_election.cc:304] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.764578 17632 raft_consensus.cc:2468] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 2.
I20260709 13:07:07.764973 18192 raft_consensus.cc:515] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.769080 18321 raft_consensus.cc:2804] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.769699 18321 raft_consensus.cc:493] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.769729 17451 leader_election.cc:304] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:07.770236 18321 raft_consensus.cc:3060] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.772598 18220 leader_election.cc:290] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.772876 18303 raft_consensus.cc:2804] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.774856 18303 raft_consensus.cc:493] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.775336 18303 raft_consensus.cc:3060] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.773392 17304 catalog_manager.cc:5697] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 2 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:07.766898 17596 leader_election.cc:304] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.782778 18192 leader_election.cc:290] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:07.784555 18320 raft_consensus.cc:2804] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:07:07.791087 17598 leader_election.cc:336] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.796692 17553 raft_consensus.cc:2468] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 2.
I20260709 13:07:07.784624 18283 leader_election.cc:290] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.797636 17524 leader_election.cc:304] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.798015 18285 raft_consensus.cc:493] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.811101 18285 raft_consensus.cc:515] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.796097 18320 raft_consensus.cc:493] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.813041 18320 raft_consensus.cc:3060] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:07.813756 17554 raft_consensus.cc:2468] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:07.814972 18217 raft_consensus.cc:2804] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:07.815574 18217 raft_consensus.cc:697] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:07.816684 18217 consensus_queue.cc:237] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.817346 17553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:07.818400 18303 raft_consensus.cc:515] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.788533 18317 raft_consensus.cc:515] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.823856 18285 leader_election.cc:290] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.801882 17552 raft_consensus.cc:2468] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:07.789251 18281 leader_election.cc:290] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.804081 17485 raft_consensus.cc:2468] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 2.
W20260709 13:07:07.804332 17598 leader_election.cc:336] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.789255 18321 raft_consensus.cc:515] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:07.819722 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b10c75b663e4cbcb3237b59443f77ca" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:07.824023 17553 raft_consensus.cc:2393] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a9a65d4c22be403682be9bc817650ada in current term 2: Already voted for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in this term.
I20260709 13:07:07.822858 18303 leader_election.cc:290] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:07.808552 18304 ts_tablet_manager.cc:548] Loading tablet metadata (42/45 complete)
I20260709 13:07:07.802558 18230 raft_consensus.cc:515] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.830137 17451 leader_election.cc:304] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:07.846246 18286 raft_consensus.cc:2804] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:07.846848 18286 raft_consensus.cc:697] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:07.830868 17599 leader_election.cc:304] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
W20260709 13:07:07.848299 17598 leader_election.cc:336] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.847929 18286 consensus_queue.cc:237] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.849191 17599 leader_election.cc:304] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.850396 18283 raft_consensus.cc:2804] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:07.850919 18283 raft_consensus.cc:697] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:07.851054 18220 raft_consensus.cc:2804] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:07.851688 18220 raft_consensus.cc:697] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:07.851958 18283 consensus_queue.cc:237] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:07.852676 18220 consensus_queue.cc:237] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:07:07.848160 17450 leader_election.cc:336] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.832422 17522 leader_election.cc:304] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.834604 17556 raft_consensus.cc:3060] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.840387 17629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:07.858721 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e84a923caa9c4755bcb78a35143dfbd9" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:07.858990 17629 raft_consensus.cc:2393] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 2: Already voted for candidate a9a65d4c22be403682be9bc817650ada in this term.
I20260709 13:07:07.859459 17625 raft_consensus.cc:3060] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.841039 17553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:07.861804 17553 raft_consensus.cc:3060] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.835148 18321 leader_election.cc:290] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:07.867892 17599 leader_election.cc:304] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:07.842139 18230 leader_election.cc:290] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.868836 18319 raft_consensus.cc:2749] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:07.869544 18230 raft_consensus.cc:2804] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:07.844506 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34919e11221f465cbcef1151a1c5d4b0" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:07.832885 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8bf7f6c34a4edfb28a4914ad4ce49e" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:07:07.875873 17598 leader_election.cc:336] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.876307 17632 raft_consensus.cc:2468] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 1.
I20260709 13:07:07.875133 18230 raft_consensus.cc:697] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:07.842495 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f330683b8d4b0696fad129a8f2ce23" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:07.825054 18317 leader_election.cc:290] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:07.877537 18320 raft_consensus.cc:515] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:07.878106 17485 raft_consensus.cc:3060] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.877537 17554 raft_consensus.cc:2468] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 1.
I20260709 13:07:07.892750 17524 leader_election.cc:304] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:07.905894 17599 leader_election.cc:304] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.908629 18217 raft_consensus.cc:2804] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.909336 18217 raft_consensus.cc:493] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.909703 17556 raft_consensus.cc:2468] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 2.
I20260709 13:07:07.912883 17553 raft_consensus.cc:2468] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 2.
I20260709 13:07:07.882905 17625 raft_consensus.cc:2468] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 2.
I20260709 13:07:07.909759 18217 raft_consensus.cc:3060] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:07.915688 17524 leader_election.cc:336] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
W20260709 13:07:07.919469 17524 leader_election.cc:336] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:07.919979 17524 leader_election.cc:304] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:07.926083 17485 raft_consensus.cc:2468] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:07.927826 17451 leader_election.cc:304] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:07.927649 17596 leader_election.cc:304] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:07.929036 17451 leader_election.cc:304] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:07.929352 18192 raft_consensus.cc:2804] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:07.930347 18192 raft_consensus.cc:697] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:07.933843 18302 raft_consensus.cc:2804] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:07.934149 18230 consensus_queue.cc:237] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.936748 18192 consensus_queue.cc:237] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.924651 18220 raft_consensus.cc:2804] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:07.931125 18285 raft_consensus.cc:2749] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:07.940608 18220 raft_consensus.cc:493] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:07.944415 17554 raft_consensus.cc:1275] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer 4df2ca35be5341afa5a621233a4da94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:07.931929 18319 raft_consensus.cc:2804] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:07.946995 18319 raft_consensus.cc:697] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:07.948367 18319 consensus_queue.cc:237] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:07.942085 18302 raft_consensus.cc:697] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:07.946285 17304 catalog_manager.cc:5697] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:07.946287 18220 raft_consensus.cc:3060] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.951565 18302 consensus_queue.cc:237] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:07.953390 18320 leader_election.cc:290] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:07.957454 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:07.959789 17485 raft_consensus.cc:3060] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:07.963507 17598 leader_election.cc:336] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
W20260709 13:07:07.973248 17450 leader_election.cc:336] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
W20260709 13:07:07.976408 17450 leader_election.cc:336] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:08.006143 17625 raft_consensus.cc:1275] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Refusing update from remote peer 4df2ca35be5341afa5a621233a4da94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:08.030848 17485 raft_consensus.cc:2468] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 3.
I20260709 13:07:08.032922 17596 leader_election.cc:304] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
W20260709 13:07:07.985533 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 59684.5us
I20260709 13:07:08.034829 18283 raft_consensus.cc:2804] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:08.035672 18283 raft_consensus.cc:697] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 3 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:08.034859 17307 catalog_manager.cc:5697] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 2 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:08.036958 18283 consensus_queue.cc:237] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:08.035175 17303 catalog_manager.cc:5697] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 2 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:08.058502 17524 leader_election.cc:304] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:08.063233 17485 raft_consensus.cc:1275] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:08.064247 18114 consensus_queue.cc:1048] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:08.065966  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:07:08.066504  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 1.332s	user 0.028s	sys 0.024s
I20260709 13:07:08.067106  7526 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
W20260709 13:07:08.067736 17524 leader_election.cc:336] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:08.069243 18230 raft_consensus.cc:2804] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:08.070235 18230 raft_consensus.cc:697] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:08.075377 18322 consensus_queue.cc:1048] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:08.078070 18230 consensus_queue.cc:237] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:07:08.079792 17450 consensus_peers.cc:597] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:08.080683 17450 consensus_peers.cc:597] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:08.081179 17450 leader_election.cc:336] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:08.086253 18328 consensus_queue.cc:1048] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
W20260709 13:07:08.087136 17450 leader_election.cc:336] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:08.089643 18220 raft_consensus.cc:515] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:08.052681 18217 raft_consensus.cc:515] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:07:08.094583 17524 consensus_peers.cc:597] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:08.107059 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8bf7f6c34a4edfb28a4914ad4ce49e" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:08.107935 17632 raft_consensus.cc:3060] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:08.128793 17632 raft_consensus.cc:2468] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 2.
I20260709 13:07:08.094691 18217 leader_election.cc:290] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:08.136835 18304 tablet_bootstrap.cc:492] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:08.149186 18220 leader_election.cc:290] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
W20260709 13:07:08.159616 17598 leader_election.cc:336] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
W20260709 13:07:08.169934 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 63016.8us
W20260709 13:07:08.181990 17450 consensus_peers.cc:597] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:08.175944 17524 leader_election.cc:304] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:08.183725 18230 raft_consensus.cc:2804] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:08.184212 18230 raft_consensus.cc:697] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:08.176445 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34919e11221f465cbcef1151a1c5d4b0" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:08.184990 17552 raft_consensus.cc:3060] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:08.185158 18230 consensus_queue.cc:237] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:08.192852 17558 raft_consensus.cc:1275] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer 4df2ca35be5341afa5a621233a4da94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:08.193300 17307 catalog_manager.cc:5697] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 2 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:08.195173 17307 catalog_manager.cc:5697] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 2 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:08.236828 17552 raft_consensus.cc:2468] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
W20260709 13:07:08.285480 17524 leader_election.cc:336] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:08.295529 17599 leader_election.cc:304] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:08.316342 18286 consensus_queue.cc:1048] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:08.289532 17303 catalog_manager.cc:5697] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 2 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:08.319506 17303 catalog_manager.cc:5697] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 2 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:08.321449 17303 catalog_manager.cc:5697] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 2 to 3, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 3 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:08.303061 18319 raft_consensus.cc:2804] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:08.413851 17304 catalog_manager.cc:5697] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:08.428464 17451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 87367.3us
W20260709 13:07:08.440444 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 54998.9us
I20260709 13:07:08.438947 17304 catalog_manager.cc:5697] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:08.483428 18337 log.cc:927] Time spent T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:07:08.506680 17552 raft_consensus.cc:1275] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:08.510018 18347 consensus_queue.cc:1048] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:08.510666 18340 log.cc:927] Time spent T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317: Append to log took a long time: real 0.184s	user 0.002s	sys 0.007s
W20260709 13:07:08.555517 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 92017.9us
I20260709 13:07:08.567822 17626 raft_consensus.cc:1275] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:08.571832 17598 consensus_peers.cc:597] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:08.572558 17598 consensus_peers.cc:597] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:08.584292 18319 raft_consensus.cc:697] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:08.608300 17483 raft_consensus.cc:1275] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:08.617152 17598 consensus_peers.cc:597] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:08.631408 17598 consensus_peers.cc:597] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:08.632972 18217 consensus_queue.cc:1048] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:08.658511 17450 consensus_peers.cc:597] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:08.661249 17629 raft_consensus.cc:1275] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:08.664283 18230 consensus_queue.cc:1048] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:08.674345 17624 raft_consensus.cc:1275] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:08.686069 17524 consensus_peers.cc:597] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:08.687988 17552 raft_consensus.cc:1275] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:08.691612 18319 consensus_queue.cc:237] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:07:08.705118 17524 consensus_peers.cc:597] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:08.711606 17524 consensus_peers.cc:597] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:08.751698 17598 consensus_peers.cc:597] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:08.766206 17552 raft_consensus.cc:1275] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer 4df2ca35be5341afa5a621233a4da94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:08.776060 17450 consensus_peers.cc:597] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:08.777204 17450 consensus_peers.cc:597] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:08.785492 17552 raft_consensus.cc:1275] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer 4df2ca35be5341afa5a621233a4da94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:08.786571 18304 tablet_bootstrap.cc:492] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:08.787904 18304 tablet_bootstrap.cc:492] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
W20260709 13:07:08.787953 17598 consensus_peers.cc:597] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:08.788740 17598 consensus_peers.cc:597] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:08.792893 17481 raft_consensus.cc:1275] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:08.793919 17482 raft_consensus.cc:1275] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:08.796895 18304 ts_tablet_manager.cc:1403] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.664s	user 0.030s	sys 0.026s
I20260709 13:07:08.798172 18320 consensus_queue.cc:1048] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:08.800602 18320 consensus_queue.cc:1048] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:08.803817 18352 consensus_queue.cc:1048] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:08.810803 18304 raft_consensus.cc:359] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:08.811815 18304 raft_consensus.cc:740] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:08.815939 18322 consensus_queue.cc:1048] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:08.831000 18304 consensus_queue.cc:260] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:08.838569 18230 consensus_queue.cc:1048] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:08.842145 18304 ts_tablet_manager.cc:1434] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:07:08.843675 18304 tablet_bootstrap.cc:492] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:08.846437 18354 consensus_queue.cc:1048] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:08.849329 17450 consensus_peers.cc:597] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:08.852371 17304 catalog_manager.cc:5697] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 2 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:08.853540 17450 consensus_peers.cc:597] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:08.857697 17450 consensus_peers.cc:597] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:08.875218 18192 consensus_queue.cc:1048] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:08.881201 17524 consensus_peers.cc:597] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:08.883129 17524 consensus_peers.cc:597] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:08.890956 17524 consensus_peers.cc:597] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:08.893916 17524 consensus_peers.cc:597] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:08.923777 17554 raft_consensus.cc:1275] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer 4df2ca35be5341afa5a621233a4da94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:08.925374 17598 consensus_peers.cc:597] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:08.928437 17524 consensus_peers.cc:597] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:08.929136 17524 consensus_peers.cc:597] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:08.943173 18286 consensus_queue.cc:1048] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:08.949502 17524 consensus_peers.cc:597] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:08.953752 17524 consensus_peers.cc:597] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:08.960796 17303 catalog_manager.cc:5697] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 1 to 2, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:08.963596 17450 consensus_peers.cc:597] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:08.983022 18304 tablet_bootstrap.cc:492] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:08.984231 18304 tablet_bootstrap.cc:492] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:08.985080 18304 ts_tablet_manager.cc:1403] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.142s	user 0.011s	sys 0.014s
I20260709 13:07:09.007336 18304 raft_consensus.cc:359] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:09.008361 18304 raft_consensus.cc:740] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
W20260709 13:07:09.009133 17450 consensus_peers.cc:597] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:09.009914 17450 consensus_peers.cc:597] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:09.019156 18304 consensus_queue.cc:260] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:09.043012 18304 ts_tablet_manager.cc:1434] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.001s
I20260709 13:07:09.044070 18304 tablet_bootstrap.cc:492] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:07:09.049649 17450 consensus_peers.cc:597] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:09.056489 17524 consensus_peers.cc:597] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:09.072304 17598 consensus_peers.cc:597] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:09.082398  7526 ts_tablet_manager.cc:600] Registering tablets (15/45 complete)
W20260709 13:07:09.100286 17524 consensus_peers.cc:597] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:09.102950 17450 consensus_peers.cc:597] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:09.103861 17450 consensus_peers.cc:597] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:09.115728 17450 consensus_peers.cc:597] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:09.116336 17450 consensus_peers.cc:597] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:09.137553 17450 consensus_peers.cc:597] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:09.141433 17598 consensus_peers.cc:597] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:09.142160 17598 consensus_peers.cc:597] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:09.216096 18304 tablet_bootstrap.cc:492] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:09.217388 18304 tablet_bootstrap.cc:492] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:09.218171 18304 ts_tablet_manager.cc:1403] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.174s	user 0.021s	sys 0.040s
I20260709 13:07:09.221747 18304 raft_consensus.cc:359] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:09.222836 18304 raft_consensus.cc:740] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:09.223663 18304 consensus_queue.cc:260] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:09.232059 18304 ts_tablet_manager.cc:1434] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:07:09.233111 18304 tablet_bootstrap.cc:492] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:09.245088 17556 raft_consensus.cc:1275] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:09.246923 18281 consensus_queue.cc:1048] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:09.250654 17450 consensus_peers.cc:597] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:09.294364 18304 tablet_bootstrap.cc:492] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:09.295498 18304 tablet_bootstrap.cc:492] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:09.296252 18304 ts_tablet_manager.cc:1403] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.063s	user 0.022s	sys 0.001s
I20260709 13:07:09.304023 18304 raft_consensus.cc:359] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:09.304913 18304 raft_consensus.cc:740] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:09.305884 18304 consensus_queue.cc:260] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:09.308621 18304 ts_tablet_manager.cc:1434] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:07:09.309630 18304 tablet_bootstrap.cc:492] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:09.440542  7526 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:07:09.441260  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 1.374s	user 0.386s	sys 0.018s
W20260709 13:07:09.524286 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 44287.3us
I20260709 13:07:09.524709 18304 tablet_bootstrap.cc:492] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:09.525818 18304 tablet_bootstrap.cc:492] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:09.526703 18304 ts_tablet_manager.cc:1403] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.217s	user 0.070s	sys 0.106s
I20260709 13:07:09.530083 18304 raft_consensus.cc:359] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:09.533304 18304 raft_consensus.cc:740] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:09.534698 18304 consensus_queue.cc:260] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:09.538379 18304 ts_tablet_manager.cc:1434] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:07:09.539415 18304 tablet_bootstrap.cc:492] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:09.567492 18304 tablet_bootstrap.cc:492] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:09.568739 18304 tablet_bootstrap.cc:492] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:09.569554 18304 ts_tablet_manager.cc:1403] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.006s
I20260709 13:07:09.572924 18304 raft_consensus.cc:359] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:09.575897 18304 raft_consensus.cc:740] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:09.576884 18304 consensus_queue.cc:260] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:09.579629 18304 ts_tablet_manager.cc:1434] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:07:09.580880 18304 tablet_bootstrap.cc:492] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:09.631639 18304 tablet_bootstrap.cc:492] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:09.633755 18304 tablet_bootstrap.cc:492] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:09.634649 18304 ts_tablet_manager.cc:1403] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.054s	user 0.026s	sys 0.010s
I20260709 13:07:09.637967 18304 raft_consensus.cc:359] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:09.638813 18304 raft_consensus.cc:740] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:09.655419 18304 consensus_queue.cc:260] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:09.657923 18304 ts_tablet_manager.cc:1434] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 13:07:09.661521 18304 tablet_bootstrap.cc:492] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:09.678776 18217 raft_consensus.cc:493] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.679733 18217 raft_consensus.cc:515] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:09.682592 18217 leader_election.cc:290] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:09.740542 17624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:09.741683 17624 raft_consensus.cc:2468] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 2.
I20260709 13:07:09.753443 17524 leader_election.cc:304] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:09.754837 18230 raft_consensus.cc:2804] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:09.755265 18230 raft_consensus.cc:493] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.755602 18230 raft_consensus.cc:3060] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:09.766539 17524 leader_election.cc:336] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:09.779991 18304 tablet_bootstrap.cc:492] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:09.782205 18230 raft_consensus.cc:515] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:09.803813 18304 tablet_bootstrap.cc:492] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:09.805251 18304 ts_tablet_manager.cc:1403] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.144s	user 0.027s	sys 0.033s
I20260709 13:07:09.814352 18230 leader_election.cc:290] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:09.823323 17624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:09.824234 17624 raft_consensus.cc:3060] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:09.824488 18304 raft_consensus.cc:359] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:09.825656 18304 raft_consensus.cc:740] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:09.828748 18304 consensus_queue.cc:260] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:09.832201 18304 ts_tablet_manager.cc:1434] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.003s
I20260709 13:07:09.833328 18304 tablet_bootstrap.cc:492] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:07:09.838416 17524 leader_election.cc:336] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:09.852586 17624 raft_consensus.cc:2468] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 3.
I20260709 13:07:09.855615 17524 leader_election.cc:304] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:09.856577 18230 raft_consensus.cc:2804] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:09.859427 18230 raft_consensus.cc:697] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:09.864312 18230 consensus_queue.cc:237] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:09.881563 18304 tablet_bootstrap.cc:492] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:09.883314 18304 tablet_bootstrap.cc:492] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:09.884260 18304 ts_tablet_manager.cc:1403] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.014s
I20260709 13:07:09.904217 18304 raft_consensus.cc:359] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:09.905269 18304 raft_consensus.cc:740] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:09.906397 18304 consensus_queue.cc:260] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:09.929622 17304 catalog_manager.cc:5697] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 1 to 3, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:09.987443 18304 ts_tablet_manager.cc:1434] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.086s	user 0.003s	sys 0.004s
I20260709 13:07:09.993954 18304 tablet_bootstrap.cc:492] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:10.070554 18304 tablet_bootstrap.cc:492] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:10.074952 18304 tablet_bootstrap.cc:492] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:10.079942 18304 ts_tablet_manager.cc:1403] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.086s	user 0.017s	sys 0.004s
I20260709 13:07:10.113248 18304 raft_consensus.cc:359] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:10.126845 18304 raft_consensus.cc:740] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:10.135178 18304 consensus_queue.cc:260] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:10.145345 18369 raft_consensus.cc:493] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.146054 18369 raft_consensus.cc:515] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:10.159058 18369 leader_election.cc:290] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:10.174942 18304 ts_tablet_manager.cc:1434] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.089s	user 0.006s	sys 0.003s
I20260709 13:07:10.176343 18304 tablet_bootstrap.cc:492] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:10.250162 17626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:10.254796 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:10.264087 18296 leader_election.cc:304] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:10.265802 18369 raft_consensus.cc:2749] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:10.328558 18304 tablet_bootstrap.cc:492] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:10.330062 18304 tablet_bootstrap.cc:492] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:10.331090 18304 ts_tablet_manager.cc:1403] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.155s	user 0.024s	sys 0.006s
I20260709 13:07:10.335117 18304 raft_consensus.cc:359] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:10.338217 18304 raft_consensus.cc:740] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:10.339360 18304 consensus_queue.cc:260] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:10.342139 18304 ts_tablet_manager.cc:1434] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:07:10.345752 18304 tablet_bootstrap.cc:492] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:10.359997 18369 raft_consensus.cc:493] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.360741 18369 raft_consensus.cc:515] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:10.369470 17626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61235f863ca44ed3b6c4715dfd68e223" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:10.379940 18369 leader_election.cc:290] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:10.405748 18304 tablet_bootstrap.cc:492] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:10.409257 18304 tablet_bootstrap.cc:492] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:10.410274 18304 ts_tablet_manager.cc:1403] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.003s
I20260709 13:07:10.414140 18304 raft_consensus.cc:359] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:10.415089 18304 raft_consensus.cc:740] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:10.417099 18304 consensus_queue.cc:260] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
W20260709 13:07:10.417701 17524 consensus_peers.cc:597] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:10.426836 18304 ts_tablet_manager.cc:1434] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:07:10.428459 18304 tablet_bootstrap.cc:492] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:10.449793 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61235f863ca44ed3b6c4715dfd68e223" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:10.452272 18299 leader_election.cc:304] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:10.454069 18369 raft_consensus.cc:2749] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:10.464881 18304 tablet_bootstrap.cc:492] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:07:10.468216 17524 consensus_peers.cc:597] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:10.482287 18304 tablet_bootstrap.cc:492] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:10.484354 18304 ts_tablet_manager.cc:1403] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.056s	user 0.023s	sys 0.000s
I20260709 13:07:10.487105 17626 raft_consensus.cc:1275] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:10.492343 18304 raft_consensus.cc:359] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:10.496567 18230 consensus_queue.cc:1048] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:10.494161 18304 raft_consensus.cc:740] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:10.502070 18304 consensus_queue.cc:260] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:10.508652 18304 ts_tablet_manager.cc:1434] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:07:10.510028 18304 tablet_bootstrap.cc:492] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:07:10.603029 17450 consensus_peers.cc:597] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:10.620203 18304 tablet_bootstrap.cc:492] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:10.621953 18304 tablet_bootstrap.cc:492] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:10.623152 18304 ts_tablet_manager.cc:1403] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.114s	user 0.031s	sys 0.026s
I20260709 13:07:10.634572 18304 raft_consensus.cc:359] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:10.636320 18304 raft_consensus.cc:740] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:10.637193 18304 consensus_queue.cc:260] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:10.639955 18304 ts_tablet_manager.cc:1434] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.005s
I20260709 13:07:10.640961 18304 tablet_bootstrap.cc:492] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:10.673557  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.129:34943
I20260709 13:07:10.673627 18462 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.129:34943 every 8 connection(s)
I20260709 13:07:10.688131 18304 tablet_bootstrap.cc:492] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:10.689448 18304 tablet_bootstrap.cc:492] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:10.690339 18304 ts_tablet_manager.cc:1403] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.011s
I20260709 13:07:10.701411 18304 raft_consensus.cc:359] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:10.702591 18304 raft_consensus.cc:740] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:10.703729 18304 consensus_queue.cc:260] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:10.708595 18304 ts_tablet_manager.cc:1434] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 13:07:10.709693 18304 tablet_bootstrap.cc:492] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:10.735260 18369 raft_consensus.cc:493] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.736035 18369 raft_consensus.cc:515] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:10.754526 18369 leader_election.cc:290] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:10.759969 18463 heartbeater.cc:344] Connected to a master server at 127.7.89.190:45519
I20260709 13:07:10.760726 18463 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:10.761778 18463 heartbeater.cc:507] Master 127.7.89.190:45519 requested a full tablet report, sending...
I20260709 13:07:10.772789 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:10.970182 18369 raft_consensus.cc:493] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.970894 18369 raft_consensus.cc:515] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:10.973451 18369 leader_election.cc:290] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:10.998780 18417 raft_consensus.cc:3060] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:11.009057 17598 consensus_peers.cc:597] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:11.023797 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:11.024318 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b10c75b663e4cbcb3237b59443f77ca" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:11.025377 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b10c75b663e4cbcb3237b59443f77ca" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:11.024346 18369 raft_consensus.cc:493] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.032406 18369 raft_consensus.cc:515] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:07:11.034726 17450 consensus_peers.cc:597] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:11.035611 18422 raft_consensus.cc:3060] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:11.036557 17450 consensus_peers.cc:597] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:11.037621 18369 leader_election.cc:290] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:11.038167 18369 raft_consensus.cc:493] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.040372 18420 raft_consensus.cc:3060] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.043987 18426 raft_consensus.cc:3060] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.063720 18304 tablet_bootstrap.cc:492] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:07:11.066785 17598 consensus_peers.cc:597] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:11.080374 18298 leader_election.cc:304] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:11.084570 17304 ts_manager.cc:194] Re-registered known tserver with Master: 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:11.091276 18424 raft_consensus.cc:3060] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:11.098510 18298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 61294.8us
I20260709 13:07:11.107278 18304 tablet_bootstrap.cc:492] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:11.108521 18304 ts_tablet_manager.cc:1403] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.399s	user 0.052s	sys 0.038s
I20260709 13:07:11.038605 18369 raft_consensus.cc:515] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:11.112160 18304 raft_consensus.cc:359] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:11.114662 18304 raft_consensus.cc:740] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:11.119668 18488 raft_consensus.cc:2749] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:11.123911 18369 leader_election.cc:290] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:11.138759 18304 consensus_queue.cc:260] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:11.155143 18304 ts_tablet_manager.cc:1434] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 13:07:11.156298 18304 tablet_bootstrap.cc:492] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:11.168411 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032dc48c2ae44160841c456abd340e40" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:11.169890 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7e0518788149c19cf493a65bc598c2" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:11.189080 18296 leader_election.cc:304] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
W20260709 13:07:11.195577 17598 consensus_peers.cc:597] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:11.198465 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032dc48c2ae44160841c456abd340e40" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:11.203285 18298 leader_election.cc:304] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:11.205564 18369 raft_consensus.cc:2749] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:11.220687 18423 raft_consensus.cc:3060] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:11.221320 17450 consensus_peers.cc:597] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:11.222198 17450 consensus_peers.cc:597] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:11.223392 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7e0518788149c19cf493a65bc598c2" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:11.248955 18296 leader_election.cc:304] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:11.262784 18369 raft_consensus.cc:2749] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:07:11.250100 17450 consensus_peers.cc:597] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:11.253283 18486 raft_consensus.cc:2749] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:11.265828 18304 tablet_bootstrap.cc:492] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:11.270134 17304 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54062
I20260709 13:07:11.275821 18422 raft_consensus.cc:3060] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.277299 18463 heartbeater.cc:499] Master 127.7.89.190:45519 was elected leader, sending a full tablet report...
I20260709 13:07:11.283488 18304 tablet_bootstrap.cc:492] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:11.286130 18304 ts_tablet_manager.cc:1403] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.130s	user 0.025s	sys 0.006s
W20260709 13:07:11.292778 17450 consensus_peers.cc:597] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:11.295038 18425 raft_consensus.cc:3060] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.307520 18304 raft_consensus.cc:359] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:11.312889 18304 raft_consensus.cc:740] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:11.314000 18304 consensus_queue.cc:260] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:07:11.312317 17524 consensus_peers.cc:597] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:11.319720 18417 raft_consensus.cc:3060] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.322041 18304 ts_tablet_manager.cc:1434] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 13:07:11.323185 18304 tablet_bootstrap.cc:492] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:07:11.372059 17524 consensus_peers.cc:597] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:11.373659 17598 consensus_peers.cc:597] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:11.375394 17524 consensus_peers.cc:597] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:11.395781 18304 tablet_bootstrap.cc:492] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:11.397141 18304 tablet_bootstrap.cc:492] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:11.398720 18304 ts_tablet_manager.cc:1403] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.076s	user 0.023s	sys 0.009s
W20260709 13:07:11.402462 17450 consensus_peers.cc:597] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:11.406019 18304 raft_consensus.cc:359] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:11.408999 18304 raft_consensus.cc:740] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:11.411080 18304 consensus_queue.cc:260] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:07:11.415580 17598 consensus_peers.cc:597] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:11.416515 17598 consensus_peers.cc:597] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:11.417904 18304 ts_tablet_manager.cc:1434] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:07:11.419741 18304 tablet_bootstrap.cc:492] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:07:11.459821 17524 consensus_peers.cc:597] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:11.460837 17524 consensus_peers.cc:597] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:11.461550 17524 consensus_peers.cc:597] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:11.462543 17524 consensus_peers.cc:597] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:11.465526 17524 consensus_peers.cc:597] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:11.466295 17524 consensus_peers.cc:597] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:11.476717 18304 tablet_bootstrap.cc:492] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:11.478344 18304 tablet_bootstrap.cc:492] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:11.485819 18304 ts_tablet_manager.cc:1403] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.066s	user 0.024s	sys 0.004s
I20260709 13:07:11.495934 18304 raft_consensus.cc:359] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:11.496879 18304 raft_consensus.cc:740] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:11.497730 18304 consensus_queue.cc:260] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:11.499353 18304 ts_tablet_manager.cc:1434] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:07:11.500305 18304 tablet_bootstrap.cc:492] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:11.529781 18304 tablet_bootstrap.cc:492] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:11.531190 18304 tablet_bootstrap.cc:492] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:11.532218 18304 ts_tablet_manager.cc:1403] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.032s	user 0.020s	sys 0.008s
I20260709 13:07:11.536092 18304 raft_consensus.cc:359] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:11.537294 18304 raft_consensus.cc:740] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:11.539637 18304 consensus_queue.cc:260] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:11.542949 18304 ts_tablet_manager.cc:1434] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
W20260709 13:07:11.543358 17450 consensus_peers.cc:597] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:11.546252 18304 tablet_bootstrap.cc:492] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:07:11.573377 17598 consensus_peers.cc:597] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:11.593390 18304 tablet_bootstrap.cc:492] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:11.594884 18304 tablet_bootstrap.cc:492] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:11.595832 18304 ts_tablet_manager.cc:1403] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.050s	user 0.028s	sys 0.007s
I20260709 13:07:11.606739 18304 raft_consensus.cc:359] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:11.607671 18304 raft_consensus.cc:740] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:11.608599 18304 consensus_queue.cc:260] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:11.611291 18304 ts_tablet_manager.cc:1434] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:07:11.612300 18304 tablet_bootstrap.cc:492] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:11.760389 18304 tablet_bootstrap.cc:492] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:11.761516 18304 tablet_bootstrap.cc:492] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:11.762267 18304 ts_tablet_manager.cc:1403] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.150s	user 0.031s	sys 0.015s
I20260709 13:07:11.794235 18304 raft_consensus.cc:359] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:07:11.795498 17654 debug-util.cc:398] Leaking SignalData structure 0x7b08003bd6e0 after lost signal to thread 7529
W20260709 13:07:11.797000 17654 debug-util.cc:398] Leaking SignalData structure 0x7b080015bfc0 after lost signal to thread 17342
W20260709 13:07:11.798688 17654 debug-util.cc:398] Leaking SignalData structure 0x7b080023c220 after lost signal to thread 17509
W20260709 13:07:11.800211 17654 debug-util.cc:398] Leaking SignalData structure 0x7b0800036960 after lost signal to thread 17583
I20260709 13:07:11.801126 18304 raft_consensus.cc:740] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
W20260709 13:07:11.801692 17654 debug-util.cc:398] Leaking SignalData structure 0x7b08000aac20 after lost signal to thread 17657
I20260709 13:07:11.809329 18304 consensus_queue.cc:260] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:11.812105 18304 ts_tablet_manager.cc:1434] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.050s	user 0.009s	sys 0.002s
I20260709 13:07:11.814246 18304 tablet_bootstrap.cc:492] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:11.845944 18426 raft_consensus.cc:3060] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.850926 18420 raft_consensus.cc:3060] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:11.880894 17450 consensus_peers.cc:597] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:11.893133 18304 tablet_bootstrap.cc:492] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:11.894495 18304 tablet_bootstrap.cc:492] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:11.895401 18304 ts_tablet_manager.cc:1403] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.082s	user 0.032s	sys 0.027s
I20260709 13:07:11.898767 18304 raft_consensus.cc:359] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:11.899643 18304 raft_consensus.cc:740] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:11.900640 18304 consensus_queue.cc:260] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:11.925120 18304 ts_tablet_manager.cc:1434] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.005s
I20260709 13:07:11.927802 18304 tablet_bootstrap.cc:492] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:11.931231 18420 raft_consensus.cc:3060] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:07:12.002025 18304 tablet_bootstrap.cc:492] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:12.003279 18304 tablet_bootstrap.cc:492] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:12.004153 18304 ts_tablet_manager.cc:1403] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.077s	user 0.020s	sys 0.017s
I20260709 13:07:12.009598 18304 raft_consensus.cc:359] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:12.010555 18304 raft_consensus.cc:740] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.011530 18304 consensus_queue.cc:260] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:12.014359 18304 ts_tablet_manager.cc:1434] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:07:12.016386 18304 tablet_bootstrap.cc:492] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:12.090438 18304 tablet_bootstrap.cc:492] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:12.092002 18304 tablet_bootstrap.cc:492] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:12.093005 18304 ts_tablet_manager.cc:1403] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.077s	user 0.023s	sys 0.030s
I20260709 13:07:12.096041 18304 raft_consensus.cc:359] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:12.096938 18304 raft_consensus.cc:740] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.097939 18304 consensus_queue.cc:260] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:12.100662 18304 ts_tablet_manager.cc:1434] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:12.122716 18304 tablet_bootstrap.cc:492] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:12.167452 18304 tablet_bootstrap.cc:492] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:12.168756 18304 tablet_bootstrap.cc:492] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:12.169595 18304 ts_tablet_manager.cc:1403] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.047s	user 0.020s	sys 0.017s
I20260709 13:07:12.173372 18304 raft_consensus.cc:359] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:12.174244 18304 raft_consensus.cc:740] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.175235 18304 consensus_queue.cc:260] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:12.177960 18304 ts_tablet_manager.cc:1434] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 13:07:12.193123 18304 tablet_bootstrap.cc:492] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:07:12.384174 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 63685.1us
I20260709 13:07:12.650655 18304 tablet_bootstrap.cc:492] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:07:12.651034 17451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 71820.6us
I20260709 13:07:12.651978 18304 tablet_bootstrap.cc:492] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:12.653800 18304 ts_tablet_manager.cc:1403] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.461s	user 0.124s	sys 0.112s
W20260709 13:07:12.679576 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 83241.9us
I20260709 13:07:12.685238 18304 raft_consensus.cc:359] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:12.686230 18304 raft_consensus.cc:740] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.689478 18304 consensus_queue.cc:260] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:12.692680 18304 ts_tablet_manager.cc:1434] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:07:12.697371 18304 tablet_bootstrap.cc:492] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:07:12.702816 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 52068us
I20260709 13:07:12.810521 18304 tablet_bootstrap.cc:492] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:12.814138 18304 tablet_bootstrap.cc:492] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:12.815088 18304 ts_tablet_manager.cc:1403] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.118s	user 0.038s	sys 0.020s
I20260709 13:07:12.820519 18304 raft_consensus.cc:359] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:12.821302 18304 raft_consensus.cc:740] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.821997 18304 consensus_queue.cc:260] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:12.824409 18304 ts_tablet_manager.cc:1434] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:07:12.825724 18304 tablet_bootstrap.cc:492] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:12.875398 18304 tablet_bootstrap.cc:492] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:12.876895 18304 tablet_bootstrap.cc:492] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:12.877931 18304 ts_tablet_manager.cc:1403] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.052s	user 0.023s	sys 0.003s
I20260709 13:07:12.881328 18304 raft_consensus.cc:359] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:12.882263 18304 raft_consensus.cc:740] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.883594 18304 consensus_queue.cc:260] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:12.886030 18304 ts_tablet_manager.cc:1434] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:07:12.887130 18304 tablet_bootstrap.cc:492] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:12.918391 18304 tablet_bootstrap.cc:492] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:12.919884 18304 tablet_bootstrap.cc:492] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:12.920794 18304 ts_tablet_manager.cc:1403] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.009s
I20260709 13:07:12.924000 18304 raft_consensus.cc:359] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:12.924883 18304 raft_consensus.cc:740] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.925833 18304 consensus_queue.cc:260] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:12.928491 18304 ts_tablet_manager.cc:1434] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:12.929450 18304 tablet_bootstrap.cc:492] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:12.967242 18304 tablet_bootstrap.cc:492] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:12.968669 18304 tablet_bootstrap.cc:492] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:12.969615 18304 ts_tablet_manager.cc:1403] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.002s
I20260709 13:07:12.972892 18304 raft_consensus.cc:359] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:12.973846 18304 raft_consensus.cc:740] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.978065 18304 consensus_queue.cc:260] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:12.980363 18304 ts_tablet_manager.cc:1434] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:07:12.986518 18304 tablet_bootstrap.cc:492] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:13.013538 18304 tablet_bootstrap.cc:492] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:13.014981 18304 tablet_bootstrap.cc:492] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:13.015923 18304 ts_tablet_manager.cc:1403] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.030s	user 0.022s	sys 0.003s
I20260709 13:07:13.019066 18304 raft_consensus.cc:359] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:13.020018 18304 raft_consensus.cc:740] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:13.020996 18304 consensus_queue.cc:260] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:13.023849 18304 ts_tablet_manager.cc:1434] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:07:13.024966 18304 tablet_bootstrap.cc:492] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:13.050521 18304 tablet_bootstrap.cc:492] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:13.052017 18304 tablet_bootstrap.cc:492] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:13.052927 18304 ts_tablet_manager.cc:1403] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.028s	user 0.022s	sys 0.003s
I20260709 13:07:13.060895 18304 raft_consensus.cc:359] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:13.062000 18304 raft_consensus.cc:740] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:13.063019 18304 consensus_queue.cc:260] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:13.065635 18304 ts_tablet_manager.cc:1434] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.001s
I20260709 13:07:13.066659 18304 tablet_bootstrap.cc:492] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:13.090900 18304 tablet_bootstrap.cc:492] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:13.092212 18304 tablet_bootstrap.cc:492] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:13.093518 18304 ts_tablet_manager.cc:1403] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.027s	user 0.021s	sys 0.004s
I20260709 13:07:13.099460 18304 raft_consensus.cc:359] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:13.100558 18304 raft_consensus.cc:740] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:13.101532 18304 consensus_queue.cc:260] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:13.104331 18304 ts_tablet_manager.cc:1434] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 13:07:13.105400 18304 tablet_bootstrap.cc:492] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:13.130721 18304 tablet_bootstrap.cc:492] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:13.132365 18304 tablet_bootstrap.cc:492] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:13.133293 18304 ts_tablet_manager.cc:1403] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.008s
I20260709 13:07:13.136355 18304 raft_consensus.cc:359] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:13.137511 18304 raft_consensus.cc:740] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:13.138463 18304 consensus_queue.cc:260] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:13.141270 18304 ts_tablet_manager.cc:1434] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:07:13.142417 18304 tablet_bootstrap.cc:492] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:13.167886 18304 tablet_bootstrap.cc:492] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:13.169324 18304 tablet_bootstrap.cc:492] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:13.170287 18304 ts_tablet_manager.cc:1403] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.012s
I20260709 13:07:13.173676 18304 raft_consensus.cc:359] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:13.174675 18304 raft_consensus.cc:740] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:13.175853 18304 consensus_queue.cc:260] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:13.178522 18304 ts_tablet_manager.cc:1434] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:07:13.179638 18304 tablet_bootstrap.cc:492] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:13.267802 18426 raft_consensus.cc:3060] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.285205 18420 raft_consensus.cc:3060] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.294724 18304 tablet_bootstrap.cc:492] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:13.296104 18304 tablet_bootstrap.cc:492] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:13.296985 18304 ts_tablet_manager.cc:1403] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.118s	user 0.021s	sys 0.004s
I20260709 13:07:13.316855 18424 raft_consensus.cc:3060] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.328742 18304 raft_consensus.cc:359] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:13.329732 18304 raft_consensus.cc:740] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:13.330801 18304 consensus_queue.cc:260] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:13.333195 18304 ts_tablet_manager.cc:1434] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 13:07:13.334362 18304 tablet_bootstrap.cc:492] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:07:13.385319 17598 consensus_peers.cc:597] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:13.575114 18304 tablet_bootstrap.cc:492] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:13.578161 18304 tablet_bootstrap.cc:492] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:13.596935 18304 ts_tablet_manager.cc:1403] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.263s	user 0.021s	sys 0.026s
I20260709 13:07:13.600138 18304 raft_consensus.cc:359] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:13.600937 18304 raft_consensus.cc:740] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:13.618716 18304 consensus_queue.cc:260] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:13.624276 18304 ts_tablet_manager.cc:1434] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:07:13.626380 18304 tablet_bootstrap.cc:492] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:07:13.762432 17451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 45155.6us
W20260709 13:07:13.777460 17524 consensus_peers.cc:597] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:13.784458 18304 tablet_bootstrap.cc:492] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:13.792955 18304 tablet_bootstrap.cc:492] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:13.793910 18304 ts_tablet_manager.cc:1403] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.168s	user 0.022s	sys 0.000s
I20260709 13:07:13.802706 18304 raft_consensus.cc:359] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:13.803552 18304 raft_consensus.cc:740] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:13.804459 18304 consensus_queue.cc:260] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:13.813808 18304 ts_tablet_manager.cc:1434] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:07:13.815431 18304 tablet_bootstrap.cc:492] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:07:13.817719 17450 consensus_peers.cc:597] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:13.854338 18419 raft_consensus.cc:3060] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:13.855230 17598 consensus_peers.cc:597] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:13.896934 17598 consensus_peers.cc:597] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:13.990252 18304 tablet_bootstrap.cc:492] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:13.991755 18304 tablet_bootstrap.cc:492] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:13.992676 18304 ts_tablet_manager.cc:1403] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.178s	user 0.026s	sys 0.016s
I20260709 13:07:13.996068 18304 raft_consensus.cc:359] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:13.998605 18304 raft_consensus.cc:740] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:14.000656 18304 consensus_queue.cc:260] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:14.024392 18304 ts_tablet_manager.cc:1434] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:07:14.036599 18304 tablet_bootstrap.cc:492] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:14.091531 18304 tablet_bootstrap.cc:492] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:14.095659 18304 tablet_bootstrap.cc:492] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:14.096990 18304 ts_tablet_manager.cc:1403] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.061s	user 0.026s	sys 0.001s
I20260709 13:07:14.099850 18304 raft_consensus.cc:359] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:14.100665 18304 raft_consensus.cc:740] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:14.101447 18304 consensus_queue.cc:260] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:07:14.102324 17524 consensus_peers.cc:597] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:14.107745 18304 ts_tablet_manager.cc:1434] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:07:14.109223 18304 tablet_bootstrap.cc:492] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:14.147794 18304 tablet_bootstrap.cc:492] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:14.149056 18304 tablet_bootstrap.cc:492] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:14.150039 18304 ts_tablet_manager.cc:1403] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.041s	user 0.022s	sys 0.003s
I20260709 13:07:14.153331 18304 raft_consensus.cc:359] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:14.154939 18304 raft_consensus.cc:740] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:14.155988 18304 consensus_queue.cc:260] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:14.160784 18304 ts_tablet_manager.cc:1434] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.003s
I20260709 13:07:14.163525 18304 tablet_bootstrap.cc:492] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
W20260709 13:07:14.181979 17450 consensus_peers.cc:597] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:14.300801 18304 tablet_bootstrap.cc:492] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:14.302124 18304 tablet_bootstrap.cc:492] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:14.303120 18304 ts_tablet_manager.cc:1403] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.140s	user 0.041s	sys 0.028s
I20260709 13:07:14.327044 18304 raft_consensus.cc:359] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:14.328011 18304 raft_consensus.cc:740] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:14.328886 18304 consensus_queue.cc:260] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:14.331938 18304 ts_tablet_manager.cc:1434] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.001s
I20260709 13:07:14.334160 18304 tablet_bootstrap.cc:492] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:14.500087 18304 tablet_bootstrap.cc:492] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:14.501434 18304 tablet_bootstrap.cc:492] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:14.502357 18304 ts_tablet_manager.cc:1403] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.168s	user 0.024s	sys 0.004s
I20260709 13:07:14.532609 18304 raft_consensus.cc:359] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:14.533488 18304 raft_consensus.cc:740] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:14.534346 18304 consensus_queue.cc:260] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:14.552244 18424 raft_consensus.cc:3060] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.537245 18304 ts_tablet_manager.cc:1434] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.000s
I20260709 13:07:14.555837 18304 tablet_bootstrap.cc:492] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap starting.
I20260709 13:07:14.589767 18304 tablet_bootstrap.cc:492] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:14.590942 18304 tablet_bootstrap.cc:492] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c: Bootstrap complete.
I20260709 13:07:14.648715 18304 ts_tablet_manager.cc:1403] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c: Time spent bootstrapping tablet: real 0.093s	user 0.037s	sys 0.047s
W20260709 13:07:14.650640 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 57806.6us
I20260709 13:07:14.652468 18304 raft_consensus.cc:359] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:14.653589 18304 raft_consensus.cc:740] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:14.654580 18304 consensus_queue.cc:260] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:07:14.655571 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 70150.7us
I20260709 13:07:14.657500 18304 ts_tablet_manager.cc:1434] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
W20260709 13:07:14.900038 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 120565us
W20260709 13:07:15.266594 17451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 114145us
W20260709 13:07:19.092720 17451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 65301us
W20260709 13:07:19.455683 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 55715.2us
I20260709 13:07:20.729945  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:21.094211 17483 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29"
dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53832
I20260709 13:07:21.095486 17483 raft_consensus.cc:606] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:21.112108 17483 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6633559a9d2643a29622caab848e5442"
dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53832
I20260709 13:07:21.114728 17483 raft_consensus.cc:606] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:21.187199 17553 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13"
dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53546
I20260709 13:07:21.192077 17553 raft_consensus.cc:606] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:21.202353 17553 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b"
dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53546
I20260709 13:07:21.203672 17553 raft_consensus.cc:606] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:21.214169 17553 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53"
dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53546
I20260709 13:07:21.215416 17553 raft_consensus.cc:606] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:21.232650  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:21.235239  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:21.334416 18577 raft_consensus.cc:993] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e: : Instructing follower 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:21.335143 18577 raft_consensus.cc:1081] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Signalling peer 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:21.337416 18426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29"
dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:40288
I20260709 13:07:21.338147 18426 raft_consensus.cc:493] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:21.338666 18426 raft_consensus.cc:3060] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.349351 18577 raft_consensus.cc:993] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e: : Instructing follower 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:21.350188 18534 raft_consensus.cc:1081] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Signalling peer 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:21.353276 18419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6633559a9d2643a29622caab848e5442"
dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:40288
I20260709 13:07:21.353916 18419 raft_consensus.cc:493] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:21.354260 18419 raft_consensus.cc:3060] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.357776 18426 raft_consensus.cc:515] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:21.369735 17624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:21.372289 17624 raft_consensus.cc:3060] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.372390 18419 raft_consensus.cc:515] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:21.376013 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6633559a9d2643a29622caab848e5442" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:21.376384 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6633559a9d2643a29622caab848e5442" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:21.376698 17632 raft_consensus.cc:3060] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.377053 17484 raft_consensus.cc:3055] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:21.377378 17484 raft_consensus.cc:740] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:21.378507 17484 consensus_queue.cc:260] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:21.379809 17484 raft_consensus.cc:3060] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.405371 17632 raft_consensus.cc:2468] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:21.406414 18419 leader_election.cc:290] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:21.407449 18298 leader_election.cc:304] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:21.410579 17484 raft_consensus.cc:2468] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:21.413996 18583 raft_consensus.cc:2804] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:21.430126 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:21.430941 17484 raft_consensus.cc:3055] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:21.431254 17484 raft_consensus.cc:740] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:21.437917 18426 leader_election.cc:290] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:21.432144 17484 consensus_queue.cc:260] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:21.440032 17484 raft_consensus.cc:3060] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.457062 18583 raft_consensus.cc:697] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:21.458586 18583 consensus_queue.cc:237] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:21.464424 17484 raft_consensus.cc:2468] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:21.467828 18296 leader_election.cc:304] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:21.469040 18584 raft_consensus.cc:2804] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:21.493129 18584 raft_consensus.cc:697] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:21.494737 18584 consensus_queue.cc:237] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:21.511937 17624 raft_consensus.cc:2468] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
W20260709 13:07:21.625591 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 91534.1us
I20260709 13:07:21.661469 18579 raft_consensus.cc:993] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: : Instructing follower 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:21.662257 18579 raft_consensus.cc:1081] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Signalling peer 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:21.681082 17304 catalog_manager.cc:5697] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 2 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:21.747319 18419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53"
dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:40314
I20260709 13:07:21.748132 18419 raft_consensus.cc:493] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:21.748811 18419 raft_consensus.cc:3060] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.783394 18419 raft_consensus.cc:515] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:21.786279 18419 leader_election.cc:290] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:21.786516 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:21.787884 18579 raft_consensus.cc:993] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: : Instructing follower 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:21.788312 17483 raft_consensus.cc:3060] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.788367 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:21.788586 18579 raft_consensus.cc:1081] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Signalling peer 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:21.789280 17558 raft_consensus.cc:3055] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:21.789661 17558 raft_consensus.cc:740] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:21.792287 17558 consensus_queue.cc:260] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:21.793962 17558 raft_consensus.cc:3060] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.799021 18591 raft_consensus.cc:993] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: : Instructing follower 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:21.807440 17483 raft_consensus.cc:2468] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:21.809584 18579 raft_consensus.cc:1081] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Signalling peer 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:21.811546 18296 leader_election.cc:304] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:21.812996 18583 raft_consensus.cc:2804] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:21.813711 18583 raft_consensus.cc:697] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:21.816655 18583 consensus_queue.cc:237] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:21.825069 18426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13"
dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:40314
I20260709 13:07:21.825964 18426 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:21.826382 18426 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:21.835307 18419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b"
dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:40314
I20260709 13:07:21.836370 18419 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:21.836887 18419 raft_consensus.cc:3060] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.840368 17558 raft_consensus.cc:2468] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:21.847059 18426 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:21.856940 18426 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 4 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:21.859009 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:21.859848 17632 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:21.865012 18419 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:21.867493 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:21.868789 17558 raft_consensus.cc:3055] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:21.869688 17558 raft_consensus.cc:740] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:21.870677 17624 raft_consensus.cc:1275] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:21.870962 18419 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:21.872900 18584 consensus_queue.cc:1048] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:21.875574 17624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:21.876411 17624 raft_consensus.cc:3060] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.881896 17632 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 4.
I20260709 13:07:21.886123 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:21.887034 17552 raft_consensus.cc:3055] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:21.887452 17552 raft_consensus.cc:740] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:21.887773 18298 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:21.888927 18584 raft_consensus.cc:2804] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:21.888783 17552 consensus_queue.cc:260] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:21.884450 17558 consensus_queue.cc:260] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:21.889863 18584 raft_consensus.cc:697] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 4 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:21.892256 17552 raft_consensus.cc:3060] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.892625 18584 consensus_queue.cc:237] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:21.897814 17558 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:21.914386 17624 raft_consensus.cc:2468] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:21.915839 18298 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:21.917259 18583 raft_consensus.cc:2804] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:21.917833 18583 raft_consensus.cc:697] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:21.918913 18583 consensus_queue.cc:237] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:21.945389 17307 catalog_manager.cc:5697] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 2 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:21.949117 17307 catalog_manager.cc:5697] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 1 to 2, leader changed from 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 2 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:21.952318 17307 catalog_manager.cc:5697] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 3 to 4, leader changed from 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 4 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:21.963326 17552 raft_consensus.cc:2468] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:21.964283 17630 raft_consensus.cc:1275] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:21.976790 17558 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 4.
I20260709 13:07:21.983326 18601 consensus_queue.cc:1048] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:21.990937 17483 raft_consensus.cc:1275] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:21.992789 18601 consensus_queue.cc:1048] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:07:22.002954 17307 catalog_manager.cc:5697] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 1 to 2, leader changed from 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 2 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:22.086009 17483 raft_consensus.cc:1275] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:22.092878 18584 consensus_queue.cc:1048] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:22.236613  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:22.294832  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:22.311386 17483 raft_consensus.cc:1275] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:22.313372 18584 consensus_queue.cc:1048] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:22.356192 17552 raft_consensus.cc:1275] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:22.360311 18584 consensus_queue.cc:1048] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:22.369431 17558 raft_consensus.cc:1275] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:22.371472 18584 consensus_queue.cc:1048] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:22.374573 17630 raft_consensus.cc:1275] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:22.381992 18601 consensus_queue.cc:1048] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:22.408411 17483 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c"
dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53832
I20260709 13:07:22.409190 17483 raft_consensus.cc:606] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:22.414824 17484 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1"
dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53832
I20260709 13:07:22.415524 17484 raft_consensus.cc:606] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:22.422833 17484 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20df005b7408432da9ac4fcf110a20b9"
dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53832
I20260709 13:07:22.423547 17484 raft_consensus.cc:606] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:22.435045 17624 raft_consensus.cc:1275] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:22.435425 17483 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50"
dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53832
I20260709 13:07:22.436456 17483 raft_consensus.cc:606] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:22.443020 18583 consensus_queue.cc:1048] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:22.453486 17554 raft_consensus.cc:1275] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:22.455441 18583 consensus_queue.cc:1048] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:23.367031 18626 raft_consensus.cc:493] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.367825 18626 raft_consensus.cc:515] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:23.370766 18626 leader_election.cc:290] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
W20260709 13:07:23.384500 17525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.018s	sys 0.007s
W20260709 13:07:23.385138 17525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.018s	sys 0.007s
I20260709 13:07:23.412824 18628 raft_consensus.cc:493] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.413623 18628 raft_consensus.cc:515] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:23.416785 18628 leader_election.cc:290] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:23.443367 18584 raft_consensus.cc:493] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9a65d4c22be403682be9bc817650ada)
I20260709 13:07:23.444140 18584 raft_consensus.cc:515] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:23.447594 18584 leader_election.cc:290] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:23.449608 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1995e080f6c048fe935afd95f336b109" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:07:23.459744 17522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.972s	user 0.000s	sys 0.042s
W20260709 13:07:23.460374 17522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.974s	user 0.000s	sys 0.042s
I20260709 13:07:23.463588 18595 raft_consensus.cc:493] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e3a92cfda7b4a9c8ddf43c0c9f67317)
I20260709 13:07:23.464620 18595 raft_consensus.cc:515] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:23.471437 18624 raft_consensus.cc:493] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.472325 18624 raft_consensus.cc:515] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:23.474926 18624 leader_election.cc:290] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:23.485555 18627 raft_consensus.cc:493] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e3a92cfda7b4a9c8ddf43c0c9f67317)
I20260709 13:07:23.486333 18627 raft_consensus.cc:515] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:23.509326 18584 raft_consensus.cc:493] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a9a65d4c22be403682be9bc817650ada)
I20260709 13:07:23.510062 18584 raft_consensus.cc:515] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:23.511421 18627 leader_election.cc:290] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:23.512575 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:23.514979 18595 leader_election.cc:290] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:23.515944 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626dcd42c0f74fccb9c2a42ee7d443ee" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:23.518410 17632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49584 (request call id 15) took 1133 ms. Trace:
I20260709 13:07:23.520151 18422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.512603 18584 leader_election.cc:290] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:23.532401 17551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41552 (request call id 1003) took 1118 ms. Trace:
I20260709 13:07:23.527470 18413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800f4d4505654dc58f390fcc9f0d90db" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.533041 17551 rpcz_store.cc:276] 0709 13:07:22.413636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.414736 (+  1100us) service_pool.cc:224] Handling call
0709 13:07:23.531467 (+1116731us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b54000411a8. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32e7422aa1 00007f32e749debc 00007f32e74d308c 00007f32e74bd93b 00007f32e74bd3d2 00007f32e0a34bd4 00007f32e0a34b62 00007f32e0a34a91 00007f32e0a34a0c 00007f32e0a33cc2 00007f32e083448c
0709 13:07:23.531629 (+   162us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:23.531844 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:23.531875 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:23.531931 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:07:23.531968 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.532035 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.532158 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29830016}
I20260709 13:07:23.540198 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55f8c5bd47d4a31a9562ab2b75ab56f" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:23.549220 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800f4d4505654dc58f390fcc9f0d90db" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:23.551801 17522 leader_election.cc:304] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:23.553083 18537 raft_consensus.cc:2749] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:23.553934 17522 leader_election.cc:304] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:23.557225 18627 raft_consensus.cc:493] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.557943 18627 raft_consensus.cc:515] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:23.559082 18583 raft_consensus.cc:493] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.559757 18583 raft_consensus.cc:515] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:23.560954 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05345b561dfb497c8f1e081b05c1bb5d" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:23.562094 18583 leader_election.cc:290] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:23.562635 18635 raft_consensus.cc:493] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.560477 18627 leader_election.cc:290] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:23.563248 18635 raft_consensus.cc:515] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:23.565704 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:23.567413 18634 raft_consensus.cc:493] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:23.568142 18634 raft_consensus.cc:515] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:23.570778 18634 leader_election.cc:290] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:23.571952 18635 leader_election.cc:290] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:23.573693 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:23.577584 18635 raft_consensus.cc:493] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e3a92cfda7b4a9c8ddf43c0c9f67317)
I20260709 13:07:23.578338 18635 raft_consensus.cc:515] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:23.580698 18635 leader_election.cc:290] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:23.581724 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626dcd42c0f74fccb9c2a42ee7d443ee" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:23.582639 17485 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dcb6246353144b0a8b9648ac089a1da6"
dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
mode: GRACEFUL
new_leader_uuid: "a9a65d4c22be403682be9bc817650ada"
 from {username='slave'} at 127.0.0.1:53832
I20260709 13:07:23.583321 17485 raft_consensus.cc:606] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Received request to transfer leadership to TS a9a65d4c22be403682be9bc817650ada
I20260709 13:07:23.585834  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:23.588071 18584 raft_consensus.cc:493] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.588860 18584 raft_consensus.cc:515] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:23.588124  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:23.592144 18584 leader_election.cc:290] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:23.592404 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8bf7f6c34a4edfb28a4914ad4ce49e" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:23.594676 18626 raft_consensus.cc:493] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a9a65d4c22be403682be9bc817650ada)
I20260709 13:07:23.519014 17632 rpcz_store.cc:276] 0709 13:07:22.384994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.385814 (+   820us) service_pool.cc:224] Handling call
0709 13:07:22.386182 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:22.398124 (+ 11942us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.398173 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:22.406995 (+  8822us) log.cc:844] Serialized 26 byte log entry
0709 13:07:22.527810 (+120815us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:07:22.527879 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.553702 (+ 25823us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:23.518118 (+964416us) raft_consensus.cc:1685] finished
0709 13:07:23.518142 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.518314 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12272,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":139,"thread_start_us":119989,"threads_started":1,"wal-append.queue_time_us":253426}
I20260709 13:07:23.595481 18626 raft_consensus.cc:515] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:23.596705 17450 leader_election.cc:304] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:23.597684 18632 raft_consensus.cc:2749] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:23.598246 18635 raft_consensus.cc:493] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.599004 18635 raft_consensus.cc:515] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:23.601172 18635 leader_election.cc:290] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:23.602756 18633 raft_consensus.cc:993] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e: : Instructing follower a9a65d4c22be403682be9bc817650ada to start an election
I20260709 13:07:23.603183 18624 raft_consensus.cc:2749] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:23.603425 18633 raft_consensus.cc:1081] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Signalling peer a9a65d4c22be403682be9bc817650ada to start an election
W20260709 13:07:23.604477 17522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 50633.9us
I20260709 13:07:23.605419 18624 raft_consensus.cc:493] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.605453 17632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dcb6246353144b0a8b9648ac089a1da6"
dest_uuid: "a9a65d4c22be403682be9bc817650ada"
 from {username='slave'} at 127.0.0.1:53188
I20260709 13:07:23.606493 17632 raft_consensus.cc:493] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:23.598410 18626 leader_election.cc:290] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:23.607044 17632 raft_consensus.cc:3060] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:23.606109 18624 raft_consensus.cc:515] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:23.610670 18624 leader_election.cc:290] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:23.616519 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b10c75b663e4cbcb3237b59443f77ca" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:23.635108 18426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40314 (request call id 347) took 1172 ms. Trace:
I20260709 13:07:23.635589 18426 rpcz_store.cc:276] 0709 13:07:22.463092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.463274 (+   182us) service_pool.cc:224] Handling call
0709 13:07:23.634183 (+1170909us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b5800411698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:07:23.634415 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:23.634689 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:23.634714 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:23.634759 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:07:23.634793 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.634854 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.634984 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6214400}
I20260709 13:07:23.643535 18591 raft_consensus.cc:493] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9a65d4c22be403682be9bc817650ada)
I20260709 13:07:23.644400 18591 raft_consensus.cc:515] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:23.647425 18413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b10c75b663e4cbcb3237b59443f77ca" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.652001 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1995e080f6c048fe935afd95f336b109" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:23.652400 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:23.655041 17626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307acc498b0c4da7b7953e895b049c2a" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:23.657212 18417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05345b561dfb497c8f1e081b05c1bb5d" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.658089 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22993c01d4634a3a92c5a2903aa07ffe" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:23.664410 18419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40302 (request call id 257) took 1202 ms. Trace:
I20260709 13:07:23.646093 18426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61235f863ca44ed3b6c4715dfd68e223" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
W20260709 13:07:23.647018 17558 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.160s	user 0.007s	sys 0.019s
I20260709 13:07:23.664948 18419 rpcz_store.cc:276] 0709 13:07:22.461965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.462150 (+   185us) service_pool.cc:224] Handling call
0709 13:07:23.662349 (+1200199us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b5800411698. stack: 00007f32df3da343 00007f32df3da240 00007f32ded691d4 00007f32e8aa7304 00007f32e8aa7299 00007f32df294db1 00007f32df3d4b71 00007f32df3d4a22 00007f32e82edc09 00007f32e7447ee0 00007f32e75110ab 00007f32e7511131 00007f32e74d2fc0 00007f32e74bd929 00007f32e74bd3d2 00007f32e0a34bd4
0709 13:07:23.662568 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:23.664045 (+  1477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:23.664081 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:23.664126 (+    45us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:07:23.664164 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.664231 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.664334 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8421632}
I20260709 13:07:23.652002 18591 leader_election.cc:290] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
W20260709 13:07:23.672499 17558 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.186s	user 0.007s	sys 0.020s
I20260709 13:07:23.672937 18413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.673606 17558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56294 (request call id 10) took 1298 ms. Trace:
I20260709 13:07:23.671670 17632 raft_consensus.cc:515] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:23.672539 18634 raft_consensus.cc:493] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.678045 18634 raft_consensus.cc:515] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:23.679764 18591 raft_consensus.cc:493] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.680559 18591 raft_consensus.cc:515] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:23.681890 18634 leader_election.cc:290] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:23.682654 18624 raft_consensus.cc:493] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.683441 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e559727ff02540749beb944b741d471f" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:23.683795 17632 leader_election.cc:290] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:23.684202 18298 leader_election.cc:304] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:23.686010 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307acc498b0c4da7b7953e895b049c2a" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:23.686391 17551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:23.686800 18413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8bf7f6c34a4edfb28a4914ad4ce49e" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.686010 18417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.674155 17558 rpcz_store.cc:276] 0709 13:07:22.375163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.382635 (+  7472us) service_pool.cc:224] Handling call
0709 13:07:22.382995 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:22.383442 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.383470 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:22.383999 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:07:22.400463 (+ 16464us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:07:22.479190 (+ 78727us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.673106 (+1193916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.673221 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:23.673291 (+    70us) raft_consensus.cc:1685] finished
0709 13:07:23.673309 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.673525 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5658,"prepare.queue_time_us":78755,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":180303,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":237,"thread_start_us":1275741,"threads_started":3,"wal-append.queue_time_us":17928,"wal-append.run_cpu_time_us":3098,"wal-append.run_wall_time_us":1101682}
I20260709 13:07:23.683441 18624 raft_consensus.cc:515] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
W20260709 13:07:23.688807 17624 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.229s	user 0.043s	sys 0.037s
W20260709 13:07:23.689378 17624 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.230s	user 0.043s	sys 0.037s
I20260709 13:07:23.692103 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:23.692368 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb6246353144b0a8b9648ac089a1da6" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:23.693290 17558 raft_consensus.cc:3060] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:23.708637 18426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.728653 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:23.708396 18584 raft_consensus.cc:2749] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:23.730265 18624 leader_election.cc:290] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:23.710685 18634 raft_consensus.cc:493] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e3a92cfda7b4a9c8ddf43c0c9f67317)
I20260709 13:07:23.740170 17524 leader_election.cc:304] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:23.742511 17558 raft_consensus.cc:2468] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:23.724164 18626 raft_consensus.cc:493] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.737833 18630 raft_consensus.cc:493] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a9a65d4c22be403682be9bc817650ada)
I20260709 13:07:23.761169 17599 leader_election.cc:304] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:23.762099 18638 raft_consensus.cc:493] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e3a92cfda7b4a9c8ddf43c0c9f67317)
I20260709 13:07:23.763824 18298 leader_election.cc:304] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:23.764503 18626 raft_consensus.cc:515] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:23.746135 18591 leader_election.cc:290] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
W20260709 13:07:23.779342 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 52108.5us
W20260709 13:07:23.766734 17552 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.336s	user 0.002s	sys 0.053s
I20260709 13:07:23.765839 18636 raft_consensus.cc:493] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9a65d4c22be403682be9bc817650ada)
I20260709 13:07:23.781755 17624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49584 (request call id 17) took 1334 ms. Trace:
I20260709 13:07:23.757295 18624 raft_consensus.cc:2749] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:07:23.785977 17552 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.355s	user 0.003s	sys 0.053s
I20260709 13:07:23.788941 18426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.796914 17524 leader_election.cc:304] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:23.798394 18591 raft_consensus.cc:2749] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:23.801819 17552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56294 (request call id 11) took 1371 ms. Trace:
I20260709 13:07:23.782411 17624 rpcz_store.cc:276] 0709 13:07:22.447541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.453760 (+  6219us) service_pool.cc:224] Handling call
0709 13:07:22.454250 (+   490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:22.458108 (+  3858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.458142 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:22.458738 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 13:07:23.689830 (+1231092us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:07:23.751644 (+ 61814us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.781352 (+ 29708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.781449 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:23.781501 (+    52us) raft_consensus.cc:1685] finished
0709 13:07:23.781517 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.781684 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4070,"prepare.queue_time_us":29557,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":4967,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":13527,"spinlock_wait_cycles":55168,"thread_start_us":1314994,"threads_started":3,"wal-append.queue_time_us":201729,"wal-append.run_cpu_time_us":1985,"wal-append.run_wall_time_us":1002354}
I20260709 13:07:23.802289 17552 rpcz_store.cc:276] 0709 13:07:22.429989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.430160 (+   171us) service_pool.cc:224] Handling call
0709 13:07:22.430431 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:22.430647 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:07:23.788169 (+1357522us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.801400 (+ 13231us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:23.801479 (+    79us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:07:23.801531 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.801606 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.801743 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":48904,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"thread_start_us":1369786,"threads_started":2,"wal-append.queue_time_us":12449}
I20260709 13:07:23.744187 18634 raft_consensus.cc:515] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:23.808458 18299 leader_election.cc:304] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:23.779130 18638 raft_consensus.cc:515] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:07:23.809649 18298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 45958.3us
I20260709 13:07:23.810062 18413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe7eea0c47a4c6998274f4b6d14dfb7" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.812888 18640 raft_consensus.cc:2749] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:23.783852 18632 raft_consensus.cc:493] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a9a65d4c22be403682be9bc817650ada)
I20260709 13:07:23.817929 18644 raft_consensus.cc:493] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e3a92cfda7b4a9c8ddf43c0c9f67317)
I20260709 13:07:23.839942 18644 raft_consensus.cc:515] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:23.840734 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5414b9916b148bba0932c91f3770f57" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:23.816567 18628 raft_consensus.cc:493] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e3a92cfda7b4a9c8ddf43c0c9f67317)
I20260709 13:07:23.796737 18635 raft_consensus.cc:2804] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:23.842355 18628 raft_consensus.cc:515] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:23.831071 18638 leader_election.cc:290] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:23.787320 18636 raft_consensus.cc:515] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:23.835525 18632 raft_consensus.cc:515] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:23.837198 18626 leader_election.cc:290] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:23.838502 18631 raft_consensus.cc:493] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.838980 18584 raft_consensus.cc:2749] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:23.770385 18630 raft_consensus.cc:515] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:23.850160 18426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050a17ead1c04486b472a25b0bb179ca" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.848896 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c295096a121e4940b38125740cea67cb" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:07:23.853314 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 69168.3us
I20260709 13:07:23.848230 18631 raft_consensus.cc:515] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:23.855681 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050a17ead1c04486b472a25b0bb179ca" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:23.860029 18426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def346af33c747faaf6c9cf0414510f0" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.865242 17624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61235f863ca44ed3b6c4715dfd68e223" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:23.843405 18635 raft_consensus.cc:697] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:23.868423 18631 leader_election.cc:290] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:23.868325 18635 consensus_queue.cc:237] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:23.870553 18630 leader_election.cc:290] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:23.845134 18634 leader_election.cc:290] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:23.842558 18644 leader_election.cc:290] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:23.845783 18628 leader_election.cc:290] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:23.879361 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def346af33c747faaf6c9cf0414510f0" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:23.848090 18640 raft_consensus.cc:493] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
W20260709 13:07:23.843310 18299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 76928.9us
I20260709 13:07:23.897207 18299 leader_election.cc:304] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:23.887465 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36168ca382614c3f9061a116e5d12df5" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:23.899593 17627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe7eea0c47a4c6998274f4b6d14dfb7" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:23.888283 18631 raft_consensus.cc:493] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:23.901842 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:23.902040 18640 raft_consensus.cc:515] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:23.847431 18636 leader_election.cc:290] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
W20260709 13:07:23.885394 17598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 138389us
I20260709 13:07:23.847592 18632 leader_election.cc:290] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:23.922237 17624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f330683b8d4b0696fad129a8f2ce23" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:23.924798 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:23.925699 18417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f330683b8d4b0696fad129a8f2ce23" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.927166 17598 leader_election.cc:304] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:23.928325 17524 leader_election.cc:304] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:23.931083 18628 raft_consensus.cc:2749] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:23.932559 17524 leader_election.cc:304] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:23.933622 17522 leader_election.cc:304] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:23.932745 17598 leader_election.cc:304] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:23.934662 18650 raft_consensus.cc:493] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9a65d4c22be403682be9bc817650ada)
I20260709 13:07:23.935321 18650 raft_consensus.cc:515] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:23.935672 17551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9a47a9cc56480fada4e8084a44682a" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:23.937788 18650 leader_election.cc:290] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:23.938791 18631 raft_consensus.cc:515] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:23.941221 18631 leader_election.cc:290] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:23.941045 18601 raft_consensus.cc:2749] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:23.934705 18627 raft_consensus.cc:2749] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:23.944675 18638 raft_consensus.cc:493] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.945405 18638 raft_consensus.cc:515] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:23.946151 18640 leader_election.cc:290] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:23.947489 18413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36168ca382614c3f9061a116e5d12df5" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.948695 18638 leader_election.cc:290] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:23.957796 18419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac16fb87c954fd9b4cef5850ffc16e9" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.960187 18638 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:23.960902 18638 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:23.963547 18595 raft_consensus.cc:993] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e: : Instructing follower 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:23.963750 18579 raft_consensus.cc:2749] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:23.964289 18595 raft_consensus.cc:1081] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Signalling peer 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:23.965673 18579 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:23.966051 18423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5414b9916b148bba0932c91f3770f57" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.966801 18537 raft_consensus.cc:2749] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:23.967259 18638 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:23.968539 17522 leader_election.cc:304] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:23.971052 18537 raft_consensus.cc:2749] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:23.970486 18591 raft_consensus.cc:2749] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:23.972411 17524 leader_election.cc:304] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:23.975466 17551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032dc48c2ae44160841c456abd340e40" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:23.974864 18624 raft_consensus.cc:2749] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:23.985090 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:23.987588 17552 raft_consensus.cc:2468] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:07:23.992165 18426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.995301 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4b5ba400ad420f89c16a97b71e3b70" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:23.995407 18299 leader_election.cc:304] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:07:23.995299 18420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:23.998533 18640 raft_consensus.cc:2804] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:23.999032 18640 raft_consensus.cc:493] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:23.999509 18640 raft_consensus.cc:3060] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.001086 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:24.002378 17450 leader_election.cc:304] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:24.003461 18636 raft_consensus.cc:493] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9a65d4c22be403682be9bc817650ada)
I20260709 13:07:24.004024 18636 raft_consensus.cc:515] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:23.966329 18579 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:24.007663 17598 leader_election.cc:304] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:24.007941 17450 leader_election.cc:304] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:24.019464 17559 raft_consensus.cc:2468] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
W20260709 13:07:24.006155 18298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 156981us
W20260709 13:07:24.052992 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 45109.7us
W20260709 13:07:24.064096 17599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 53362.7us
I20260709 13:07:24.046171 18638 raft_consensus.cc:493] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:24.068815 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:24.067791 17450 leader_election.cc:304] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:24.069635 17484 raft_consensus.cc:2468] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 2.
I20260709 13:07:24.076422 17522 leader_election.cc:304] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:24.079398 18651 raft_consensus.cc:2749] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:24.080044 18629 raft_consensus.cc:2749] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:24.081459 18651 raft_consensus.cc:2749] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.081914 18644 raft_consensus.cc:493] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:24.082634 18644 raft_consensus.cc:515] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:24.083112 18590 raft_consensus.cc:2749] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.085918 18644 leader_election.cc:290] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:24.087754 18647 raft_consensus.cc:493] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:24.088526 18647 raft_consensus.cc:515] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:24.089763 18624 raft_consensus.cc:2804] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:24.090310 18624 raft_consensus.cc:493] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:24.090744 18624 raft_consensus.cc:3060] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:24.091122 18579 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:24.078728 18636 leader_election.cc:290] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:24.096235 18640 raft_consensus.cc:515] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:24.077312 18638 raft_consensus.cc:515] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:24.098994 18640 leader_election.cc:290] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:24.099495 18638 leader_election.cc:290] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:24.100634 17556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:24.101419 17556 raft_consensus.cc:3060] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.119660 17556 raft_consensus.cc:2468] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:24.101169 18658 raft_consensus.cc:493] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9a65d4c22be403682be9bc817650ada)
I20260709 13:07:24.125168 18569 raft_consensus.cc:493] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9a65d4c22be403682be9bc817650ada)
I20260709 13:07:24.126366 18647 leader_election.cc:290] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:24.143406 18569 raft_consensus.cc:515] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:24.145231 18299 leader_election.cc:304] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:07:24.147184 18584 raft_consensus.cc:2804] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:24.135100 18658 raft_consensus.cc:515] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:24.148051 18584 raft_consensus.cc:697] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:24.149852 18584 consensus_queue.cc:237] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:24.153002 18658 leader_election.cc:290] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:24.153687 18624 raft_consensus.cc:515] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:24.177846 17552 raft_consensus.cc:1240] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Rejecting Update request from peer 337bbc857b9b4e329c817b81bfb0971c for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:24.179270 18569 leader_election.cc:290] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
W20260709 13:07:24.183851 17599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 118052us
W20260709 13:07:24.188596 17654 debug-util.cc:398] Leaking SignalData structure 0x7b0800054ce0 after lost signal to thread 7529
W20260709 13:07:24.190297 17654 debug-util.cc:398] Leaking SignalData structure 0x7b08000a66c0 after lost signal to thread 17342
W20260709 13:07:24.192437 17654 debug-util.cc:398] Leaking SignalData structure 0x7b080015a300 after lost signal to thread 17509
I20260709 13:07:24.193972 18663 raft_consensus.cc:493] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9a65d4c22be403682be9bc817650ada)
I20260709 13:07:24.194975 17624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032dc48c2ae44160841c456abd340e40" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:24.196669 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:24.194943 18663 raft_consensus.cc:515] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:24.197450 17625 raft_consensus.cc:2468] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 1.
I20260709 13:07:24.199709 18663 leader_election.cc:290] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:24.203694 18624 leader_election.cc:290] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
W20260709 13:07:24.205137 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 184027us
I20260709 13:07:24.205024 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:24.206147 17485 raft_consensus.cc:3060] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:24.211728 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:24.212736 17625 raft_consensus.cc:2468] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 2.
I20260709 13:07:24.213116 18417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:24.217919 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:24.219398 18633 raft_consensus.cc:993] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e: : Instructing follower 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:24.220242 18633 raft_consensus.cc:1081] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Signalling peer 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:24.221908 18650 raft_consensus.cc:993] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e: : Instructing follower 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:24.222187 18417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa3738962c3242b5aa69f93b778bea53" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:07:24.222715 18650 raft_consensus.cc:1081] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Signalling peer 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:24.225121 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bc57cc906f4179a8f6ff86c42b0247" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
W20260709 13:07:24.232717 17654 debug-util.cc:398] Leaking SignalData structure 0x7b080014ef40 after lost signal to thread 17583
I20260709 13:07:24.235183 17524 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:24.242985 18537 raft_consensus.cc:2804] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:24.243582 18537 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:24.244000 18537 raft_consensus.cc:3060] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:24.246886 18298 leader_election.cc:304] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:24.247933 18640 raft_consensus.cc:2749] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.257576 18640 consensus_queue.cc:1059] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:24.258504 18601 raft_consensus.cc:3055] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:24.258956 18601 raft_consensus.cc:740] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:24.259809 17599 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:24.260373 18298 leader_election.cc:304] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:24.260797 18635 raft_consensus.cc:2804] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:24.261353 18635 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:24.261874 18635 raft_consensus.cc:3060] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:24.262167 18655 raft_consensus.cc:2804] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:24.262651 18655 raft_consensus.cc:493] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:24.263044 18655 raft_consensus.cc:3060] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.259856 18601 consensus_queue.cc:260] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:24.269683 18601 raft_consensus.cc:3060] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:24.278658 17654 debug-util.cc:398] Leaking SignalData structure 0x7b08000d3860 after lost signal to thread 17657
I20260709 13:07:24.282289 17481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85fb92208674e2ba20983a453290a8b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:24.283209 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:24.282410 17482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:07:24.288631 17654 debug-util.cc:398] Leaking SignalData structure 0x7b080019ebc0 after lost signal to thread 18462
I20260709 13:07:24.296334 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:24.286520 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:24.300191 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e559727ff02540749beb944b741d471f" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:24.304476 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:24.298557 17485 raft_consensus.cc:2468] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 3.
I20260709 13:07:24.308348 17482 raft_consensus.cc:1240] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Rejecting Update request from peer 337bbc857b9b4e329c817b81bfb0971c for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:24.309118 17522 leader_election.cc:304] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:24.310127 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:24.310693 18662 raft_consensus.cc:2804] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:24.311354 18662 raft_consensus.cc:697] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
W20260709 13:07:24.312608 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 245814us
I20260709 13:07:24.312512 18537 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:24.315418 18662 consensus_queue.cc:237] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:24.317224 17625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1995e080f6c048fe935afd95f336b109" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:24.317529 18537 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:24.318667 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:24.319061 18417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:07:24.320721 17524 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:24.321516 18417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1995e080f6c048fe935afd95f336b109" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:24.322182 18655 raft_consensus.cc:515] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:24.324916 18625 raft_consensus.cc:2749] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:24.326076 18417 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c"
dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:40288
W20260709 13:07:24.326777 17598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 280753us
I20260709 13:07:24.328871 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcb6246353144b0a8b9648ac089a1da6" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:24.329607 17484 raft_consensus.cc:3055] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:24.330022 17484 raft_consensus.cc:740] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:24.332316 17624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fb377eea3d41e68cb4f46212e6e857" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:24.333617 18420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fb377eea3d41e68cb4f46212e6e857" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:24.335239 18417 raft_consensus.cc:493] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:24.335817 18417 raft_consensus.cc:3060] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.336484 17482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c295096a121e4940b38125740cea67cb" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:24.339882 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ccac61a2bab40708cfe17c17ae1de65" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:24.340937 17482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9a47a9cc56480fada4e8084a44682a" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:24.342537 18423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ccac61a2bab40708cfe17c17ae1de65" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:24.343685 17482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4b5ba400ad420f89c16a97b71e3b70" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:24.344565 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bc57cc906f4179a8f6ff86c42b0247" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:07:24.345487 18670 log.cc:927] Time spent T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:07:24.346468 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81d2393d5664464837df0b9d2a0d25b" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:24.348647 17624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b2d2a20fb474604a1077ce9af500e42" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:24.351327 17551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81d2393d5664464837df0b9d2a0d25b" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:24.352332 18420 raft_consensus.cc:1240] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Rejecting Update request from peer 4df2ca35be5341afa5a621233a4da94e for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:24.352213 18423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b2d2a20fb474604a1077ce9af500e42" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:24.360569 18655 leader_election.cc:290] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:24.353506 17554 raft_consensus.cc:1240] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Rejecting Update request from peer 4df2ca35be5341afa5a621233a4da94e for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:24.353849 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:24.363678 18423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:24.350346 18417 raft_consensus.cc:515] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:24.365873 18419 raft_consensus.cc:1240] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Rejecting Update request from peer 4df2ca35be5341afa5a621233a4da94e for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:24.354203 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:24.354808 18426 raft_consensus.cc:1240] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Rejecting Update request from peer 4df2ca35be5341afa5a621233a4da94e for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:24.372478 17596 leader_election.cc:304] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:24.374104 18668 raft_consensus.cc:2749] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.342258 17484 consensus_queue.cc:260] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:24.377491 17484 raft_consensus.cc:3060] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.383993 18417 leader_election.cc:290] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:24.386848 17596 leader_election.cc:304] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:24.379575 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:24.388871 18628 raft_consensus.cc:2749] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:24.388968 17554 raft_consensus.cc:3060] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.391675 17596 leader_election.cc:304] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:24.393291 18628 raft_consensus.cc:2749] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:24.378850 17304 catalog_manager.cc:5697] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 2 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.395028 17596 leader_election.cc:304] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:24.395843 18628 raft_consensus.cc:2749] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.397436 17596 leader_election.cc:304] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:24.403050 18426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50"
dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:40288
I20260709 13:07:24.403849 18426 raft_consensus.cc:493] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:24.404255 18426 raft_consensus.cc:3060] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:24.404904 18635 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:24.407384 18635 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:24.408142 18628 raft_consensus.cc:2749] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.408175 18417 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1"
dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:40288
I20260709 13:07:24.409688 18417 raft_consensus.cc:493] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:24.410182 18417 raft_consensus.cc:3060] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.409857 18633 consensus_queue.cc:1059] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:24.412011 18666 raft_consensus.cc:3055] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:24.412483 18666 raft_consensus.cc:740] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:24.413551 18666 consensus_queue.cc:260] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:24.417457 18426 raft_consensus.cc:515] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:24.419323 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:24.419574 17482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:24.420315 17482 raft_consensus.cc:3055] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:24.420694 17482 raft_consensus.cc:740] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:24.421653 17482 consensus_queue.cc:260] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:24.422960 17482 raft_consensus.cc:3060] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.426134 17596 leader_election.cc:304] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:24.428918 18666 raft_consensus.cc:3060] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.429183 18423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:07:24.433462 18426 leader_election.cc:290] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 3 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:24.440421 18628 raft_consensus.cc:2749] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.444204 18296 leader_election.cc:304] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c; no voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:24.444994 17551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:24.445878 17551 raft_consensus.cc:3060] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:24.446241 18601 raft_consensus.cc:2749] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.450588 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:24.451448 17485 raft_consensus.cc:3055] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:24.451810 17485 raft_consensus.cc:740] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:24.453958 17485 consensus_queue.cc:260] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:24.454991 17485 raft_consensus.cc:3060] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:24.462656 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:24.472924 17558 raft_consensus.cc:1240] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Rejecting Update request from peer 337bbc857b9b4e329c817b81bfb0971c for earlier term 2. Current term is 3. Ops: []
W20260709 13:07:24.473613 17596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 55473.1us
I20260709 13:07:24.474515 17596 leader_election.cc:304] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:24.475026 18601 consensus_queue.cc:1059] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:24.475394 18628 raft_consensus.cc:2749] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.476204 18655 raft_consensus.cc:3055] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:24.476786 18655 raft_consensus.cc:740] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:24.476899 17596 leader_election.cc:304] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:24.477357 18669 consensus_queue.cc:1059] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:24.477746 18628 raft_consensus.cc:2749] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.477847 18655 consensus_queue.cc:260] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:24.479228 17596 leader_election.cc:304] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:24.479781 18655 raft_consensus.cc:3060] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:24.480026 18628 raft_consensus.cc:2749] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.478729 18633 raft_consensus.cc:3055] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:24.483888 18633 raft_consensus.cc:740] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:24.485159 18633 consensus_queue.cc:260] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:24.486712 18633 raft_consensus.cc:3060] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.486678 17596 leader_election.cc:304] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:24.488420 18628 raft_consensus.cc:2749] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.488932 17596 leader_election.cc:304] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:24.451363 18417 raft_consensus.cc:515] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:24.492233 17484 raft_consensus.cc:2468] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:24.493072 18628 raft_consensus.cc:2749] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.469403 17554 raft_consensus.cc:2468] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:24.495724 18299 leader_election.cc:304] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:07:24.497117 18601 raft_consensus.cc:2804] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:24.497654 18601 raft_consensus.cc:697] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:24.498729 18601 consensus_queue.cc:237] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:24.501885 17485 raft_consensus.cc:2468] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 3.
I20260709 13:07:24.503494 18296 leader_election.cc:304] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:24.504477 18640 raft_consensus.cc:2804] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:24.505043 18640 raft_consensus.cc:697] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 3 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:24.506017 18640 consensus_queue.cc:237] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:24.507404 17450 leader_election.cc:304] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:24.508714 17624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:24.509409 17624 raft_consensus.cc:3060] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.509460 18650 raft_consensus.cc:2749] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.511293 18417 leader_election.cc:290] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:24.512826 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:24.513635 17485 raft_consensus.cc:3055] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:24.513976 17485 raft_consensus.cc:740] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:24.515381 17483 raft_consensus.cc:1275] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.515173 17485 consensus_queue.cc:260] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:24.519912 17624 raft_consensus.cc:2468] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:24.521481 18298 leader_election.cc:304] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:24.522863 18645 raft_consensus.cc:2804] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:24.523415 18645 raft_consensus.cc:697] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:24.524405 18645 consensus_queue.cc:237] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:24.521556 17558 raft_consensus.cc:1240] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Rejecting Update request from peer 4df2ca35be5341afa5a621233a4da94e for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:24.529673 17551 raft_consensus.cc:2468] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 3.
I20260709 13:07:24.530052 17485 raft_consensus.cc:3060] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.531056 17599 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:24.551159 17552 raft_consensus.cc:1240] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Rejecting Update request from peer 4df2ca35be5341afa5a621233a4da94e for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:24.532465 17482 raft_consensus.cc:2468] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:24.533010 18668 consensus_queue.cc:1048] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:24.540702 17450 leader_election.cc:304] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:24.547376 17558 raft_consensus.cc:1275] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.559218 18666 raft_consensus.cc:2749] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.570124 18640 consensus_queue.cc:1048] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:24.576642 17450 leader_election.cc:304] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:24.577719 18666 raft_consensus.cc:2749] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.581903 17450 leader_election.cc:304] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:24.583052 18633 raft_consensus.cc:2749] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.583710 17450 leader_election.cc:304] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
W20260709 13:07:24.589174 17448 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.594800 18633 raft_consensus.cc:2749] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.606537  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:24.619926 17479 raft_consensus.cc:1275] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.620141 18627 raft_consensus.cc:2749] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:07:24.621307 17599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 90356us
I20260709 13:07:24.621979 18645 consensus_queue.cc:1048] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:24.626514 17596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 80599us
I20260709 13:07:24.628911 17555 raft_consensus.cc:1275] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.639106 17485 raft_consensus.cc:2468] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:24.646826 17307 catalog_manager.cc:5697] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 2 to 3, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:24.648053 17306 catalog_manager.cc:5697] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 2 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:24.671279 18665 consensus_queue.cc:1048] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:24.698742 17303 catalog_manager.cc:5697] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 2 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.709447 17485 raft_consensus.cc:1275] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:24.713945 17303 catalog_manager.cc:5697] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 2 to 3, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 3 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.716349 18662 consensus_queue.cc:1048] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:24.718472 17303 catalog_manager.cc:5697] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 2 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.791612 18417 raft_consensus.cc:1275] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:24.804675  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:24.848135 18537 consensus_queue.cc:1048] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:24.887099 17485 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01c85566c0a845438507d71c6936d507"
dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53832
I20260709 13:07:24.888000 17485 raft_consensus.cc:606] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:24.894917 17485 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4"
dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53832
I20260709 13:07:24.895689 17485 raft_consensus.cc:606] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:24.912981 17485 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3"
dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53832
I20260709 13:07:24.913712 17485 raft_consensus.cc:606] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:24.934417 17624 raft_consensus.cc:1275] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.936290 18645 consensus_queue.cc:1048] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:24.992484 17558 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c295096a121e4940b38125740cea67cb"
dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
mode: GRACEFUL
new_leader_uuid: "a9a65d4c22be403682be9bc817650ada"
 from {username='slave'} at 127.0.0.1:53546
I20260709 13:07:24.993221 17558 raft_consensus.cc:606] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Received request to transfer leadership to TS a9a65d4c22be403682be9bc817650ada
I20260709 13:07:24.999631 18666 raft_consensus.cc:993] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e: : Instructing follower 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:25.000344 18666 raft_consensus.cc:1081] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Signalling peer 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:25.004671 18426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4"
dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:40288
I20260709 13:07:25.005539 18426 raft_consensus.cc:493] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:25.006042 18426 raft_consensus.cc:3060] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.010417 17479 raft_consensus.cc:1275] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:25.010972 17482 raft_consensus.cc:1275] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:25.012584 18645 consensus_queue.cc:1048] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:25.017755 18645 consensus_queue.cc:1048] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:25.019691 17558 raft_consensus.cc:1275] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:25.023466 17484 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb"
dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
mode: GRACEFUL
new_leader_uuid: "a9a65d4c22be403682be9bc817650ada"
 from {username='slave'} at 127.0.0.1:53832
I20260709 13:07:25.024168 17484 raft_consensus.cc:606] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Received request to transfer leadership to TS a9a65d4c22be403682be9bc817650ada
I20260709 13:07:25.025624  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:25.026401 18645 consensus_queue.cc:1048] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:25.028133  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:25.034760 18426 raft_consensus.cc:515] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:25.043479 17552 raft_consensus.cc:1275] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:25.045081 18655 consensus_queue.cc:1048] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:25.051234 18426 leader_election.cc:290] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:25.053623 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:25.054785 17484 raft_consensus.cc:3055] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:25.055238 17484 raft_consensus.cc:740] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:25.056370 17484 consensus_queue.cc:260] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:25.058523 17484 raft_consensus.cc:3060] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.086704 17484 raft_consensus.cc:2468] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:25.092191 18296 leader_election.cc:304] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:25.097959 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:25.104209 17632 raft_consensus.cc:3060] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.109236 18656 raft_consensus.cc:993] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e: : Instructing follower a9a65d4c22be403682be9bc817650ada to start an election
I20260709 13:07:25.109884 18656 raft_consensus.cc:1081] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Signalling peer a9a65d4c22be403682be9bc817650ada to start an election
I20260709 13:07:25.136281 17630 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb"
dest_uuid: "a9a65d4c22be403682be9bc817650ada"
 from {username='slave'} at 127.0.0.1:53188
I20260709 13:07:25.137007 17484 raft_consensus.cc:1275] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:25.137193 17630 raft_consensus.cc:493] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:25.137864 17630 raft_consensus.cc:3060] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.138778 18601 consensus_queue.cc:1048] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:25.142496 18631 raft_consensus.cc:993] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317: : Instructing follower a9a65d4c22be403682be9bc817650ada to start an election
I20260709 13:07:25.103461 18645 raft_consensus.cc:2804] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:25.146279 18631 raft_consensus.cc:1081] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Signalling peer a9a65d4c22be403682be9bc817650ada to start an election
I20260709 13:07:25.158839 17632 raft_consensus.cc:2468] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:25.164314 17632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c295096a121e4940b38125740cea67cb"
dest_uuid: "a9a65d4c22be403682be9bc817650ada"
 from {username='slave'} at 127.0.0.1:53212
I20260709 13:07:25.169979 17632 raft_consensus.cc:493] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:25.171139 17632 raft_consensus.cc:3060] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.173149 17630 raft_consensus.cc:515] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:25.154959 18645 raft_consensus.cc:697] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:25.198228 17630 leader_election.cc:290] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:25.202512 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:25.203451 17552 raft_consensus.cc:3060] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.208724 17482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:25.209641 17482 raft_consensus.cc:3055] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:25.210052 17482 raft_consensus.cc:740] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:25.211059 17482 consensus_queue.cc:260] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:25.212728 17482 raft_consensus.cc:3060] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.218004 17632 raft_consensus.cc:515] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:25.196847 18645 consensus_queue.cc:237] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:25.224929 17552 raft_consensus.cc:2468] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:25.231454 17599 leader_election.cc:304] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:25.232546 18635 raft_consensus.cc:2804] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:25.233094 18635 raft_consensus.cc:697] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:25.234072 18635 consensus_queue.cc:237] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:25.244683 17632 leader_election.cc:290] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:25.248044 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c295096a121e4940b38125740cea67cb" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:25.248901 17483 raft_consensus.cc:3060] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.262429 17483 raft_consensus.cc:2468] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:25.264838 17596 leader_election.cc:304] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:25.269311 18632 raft_consensus.cc:993] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e: : Instructing follower 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:25.269809 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c295096a121e4940b38125740cea67cb" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:25.270499 17558 raft_consensus.cc:3055] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:25.270725 18656 raft_consensus.cc:1081] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Signalling peer 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:25.270961 17558 raft_consensus.cc:740] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:25.274101 18665 raft_consensus.cc:2804] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:25.274819 18665 raft_consensus.cc:697] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:25.272107 17558 consensus_queue.cc:260] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:25.275846 18665 consensus_queue.cc:237] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:25.281912 18417 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3"
dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:40288
I20260709 13:07:25.282871 18417 raft_consensus.cc:493] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:25.283442 18417 raft_consensus.cc:3060] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.282946 17558 raft_consensus.cc:3060] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.301120 17482 raft_consensus.cc:2468] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:25.329252 17306 catalog_manager.cc:5697] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 2 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:25.311715 17307 catalog_manager.cc:5697] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 2 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:25.340114 17558 raft_consensus.cc:2468] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:25.357344 18417 raft_consensus.cc:515] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
W20260709 13:07:25.364436 17599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 86697.1us
I20260709 13:07:25.364926 18666 raft_consensus.cc:993] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e: : Instructing follower 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:25.365494 18666 raft_consensus.cc:1081] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Signalling peer 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:25.367926 18419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "01c85566c0a845438507d71c6936d507"
dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:40288
I20260709 13:07:25.368592 18419 raft_consensus.cc:493] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:25.368983 18419 raft_consensus.cc:3060] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.382261 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:25.383173 17558 raft_consensus.cc:3060] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.384778 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:25.385535 17484 raft_consensus.cc:3055] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:25.385875 17484 raft_consensus.cc:740] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:25.386149 18417 leader_election.cc:290] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:25.388862 17484 consensus_queue.cc:260] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:25.391484 17484 raft_consensus.cc:3060] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.398394 17558 raft_consensus.cc:2468] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:25.399889 18299 leader_election.cc:304] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:07:25.400970 18645 raft_consensus.cc:2804] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:25.401520 18645 raft_consensus.cc:697] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:25.402565 18645 consensus_queue.cc:237] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:25.509418 18419 raft_consensus.cc:515] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:25.519882 18419 leader_election.cc:290] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:25.519618 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c85566c0a845438507d71c6936d507" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:25.521211 17482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c85566c0a845438507d71c6936d507" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:25.522197 17632 raft_consensus.cc:3060] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.522233 17482 raft_consensus.cc:3055] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:25.524654 17482 raft_consensus.cc:740] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:25.525731 17482 consensus_queue.cc:260] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:25.527374 17482 raft_consensus.cc:3060] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:25.530638 17484 raft_consensus.cc:2468] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:25.541862 17632 raft_consensus.cc:2468] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:25.547829 18298 leader_election.cc:304] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:25.544955 17482 raft_consensus.cc:2468] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:25.551007 18645 raft_consensus.cc:2804] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:25.551888 18645 raft_consensus.cc:697] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:25.553237 18645 consensus_queue.cc:237] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:25.616477 18669 raft_consensus.cc:493] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.617079 18669 raft_consensus.cc:515] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:25.620852 18669 leader_election.cc:290] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:25.621531 17303 catalog_manager.cc:5697] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 1 to 2, leader changed from 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 2 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:25.623644 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:25.623528 17304 catalog_manager.cc:5697] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 2 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:25.624482 18419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:25.633482 17304 catalog_manager.cc:5697] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 2 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:25.634836 18419 raft_consensus.cc:2468] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 2.
I20260709 13:07:25.637380 17450 leader_election.cc:304] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: a9a65d4c22be403682be9bc817650ada
I20260709 13:07:25.638957 18656 raft_consensus.cc:2804] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:25.639412 18656 raft_consensus.cc:493] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.639765 18656 raft_consensus.cc:3060] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:25.647603 17482 raft_consensus.cc:1275] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:25.652040 18700 consensus_queue.cc:1048] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:25.748590 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 79473.9us
I20260709 13:07:25.756552 18656 raft_consensus.cc:515] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:25.786320 17558 raft_consensus.cc:1275] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:25.787873 18656 leader_election.cc:290] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:25.789739 17632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:25.848438 18426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:07:25.849185 18426 raft_consensus.cc:3060] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:25.886556 18426 raft_consensus.cc:2468] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 3.
I20260709 13:07:25.890905 17484 raft_consensus.cc:1275] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:25.894134 17450 leader_election.cc:304] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: a9a65d4c22be403682be9bc817650ada
I20260709 13:07:25.955572 17482 raft_consensus.cc:1275] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:25.910426 18712 consensus_queue.cc:1048] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
W20260709 13:07:25.965029 17599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 74750.6us
W20260709 13:07:25.952075 18299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.131:40851, user_credentials={real_user=slave}} blocked reactor thread for 55201.1us
I20260709 13:07:25.972056 18665 consensus_queue.cc:1048] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:07:26.007290 17558 raft_consensus.cc:1275] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:26.007849 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 113819us
I20260709 13:07:25.963672 18713 consensus_queue.cc:1048] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:07:25.974599 18721 raft_consensus.cc:2804] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:26.031396  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:26.043416 17630 raft_consensus.cc:1275] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:26.046267 18712 consensus_queue.cc:1048] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:26.053499 17558 raft_consensus.cc:1275] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:26.042598 18721 raft_consensus.cc:697] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 3 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:26.104681 18721 consensus_queue.cc:237] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:26.122215 18715 consensus_queue.cc:1048] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:26.175243 18296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 116695us
I20260709 13:07:26.175577 17484 raft_consensus.cc:1275] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:26.149603 18298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 97642.1us
W20260709 13:07:26.173522 17598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 133381us
I20260709 13:07:26.206753 18726 consensus_queue.cc:1048] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:26.219103  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:07:26.285698 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 105099us
W20260709 13:07:26.285755 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 99954.3us
I20260709 13:07:26.284142 18712 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.326792 18731 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.329836 18712 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
W20260709 13:07:26.397003 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 53258.8us
I20260709 13:07:26.357568 18731 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:26.425292 18731 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:26.400611 18713 consensus_queue.cc:1048] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:07:26.485215 17632 raft_consensus.cc:1275] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:26.455596 18665 raft_consensus.cc:493] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4df2ca35be5341afa5a621233a4da94e)
I20260709 13:07:26.518594 18712 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:26.522723 18645 consensus_queue.cc:1048] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:26.524224 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:26.525077 17558 raft_consensus.cc:2468] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 3.
I20260709 13:07:26.501658 18665 raft_consensus.cc:515] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:07:26.538216 17598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 140727us
I20260709 13:07:26.556077 17485 raft_consensus.cc:1275] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:26.563246 18423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:26.564158 18423 raft_consensus.cc:2468] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 3.
W20260709 13:07:26.567698 18732 log.cc:927] Time spent T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e: Append to log took a long time: real 0.223s	user 0.003s	sys 0.000s
I20260709 13:07:26.555248 18299 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:07:26.572353 18696 raft_consensus.cc:2804] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:26.572827 18696 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.573200 18696 raft_consensus.cc:3060] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:26.574522 18751 log.cc:927] Time spent T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada: Append to log took a long time: real 0.117s	user 0.000s	sys 0.002s
I20260709 13:07:26.597339 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:26.604180 18712 consensus_queue.cc:1048] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:26.607996 17482 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2226d75e22544facaa146d281f852f27"
dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53832
I20260709 13:07:26.608861 17482 raft_consensus.cc:606] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:26.615749 18423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f14faf54714af3abeff5693ace0e8c" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c" is_pre_election: true
I20260709 13:07:26.617558 18423 raft_consensus.cc:2368] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a9a65d4c22be403682be9bc817650ada for earlier term 2. Current term is 3.
I20260709 13:07:26.633008 17482 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfc142508c684e6eb9716cb26e7047b8"
dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
mode: GRACEFUL
new_leader_uuid: "a9a65d4c22be403682be9bc817650ada"
 from {username='slave'} at 127.0.0.1:53832
I20260709 13:07:26.633930 17482 raft_consensus.cc:606] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Received request to transfer leadership to TS a9a65d4c22be403682be9bc817650ada
I20260709 13:07:26.638967 18669 raft_consensus.cc:993] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e: : Instructing follower 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:26.639485 18669 raft_consensus.cc:1081] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Signalling peer 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:26.642747 17485 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e559727ff02540749beb944b741d471f"
dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
mode: GRACEFUL
new_leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
 from {username='slave'} at 127.0.0.1:53832
I20260709 13:07:26.644186 17485 raft_consensus.cc:606] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Received request to transfer leadership to TS 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:26.643455 18426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2226d75e22544facaa146d281f852f27"
dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:40288
I20260709 13:07:26.645186 18426 raft_consensus.cc:493] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:26.646263 18426 raft_consensus.cc:3060] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:26.648473 17598 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:26.650224  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:26.654021  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:26.567754 18665 leader_election.cc:290] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:26.667332 18714 raft_consensus.cc:2804] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:26.667979 18714 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.668313 18714 raft_consensus.cc:3060] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:26.682266 17598 leader_election.cc:400] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Vote denied by peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943) with higher term. Message: Invalid argument: T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a9a65d4c22be403682be9bc817650ada for earlier term 2. Current term is 3.
I20260709 13:07:26.682499 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:26.589586 18696 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:26.703533 17630 raft_consensus.cc:3060] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:07:26.708806 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:26.709605 17558 raft_consensus.cc:3060] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:26.712189 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:26.716746 18420 raft_consensus.cc:1275] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Refusing update from remote peer 4df2ca35be5341afa5a621233a4da94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:26.719321 18669 consensus_queue.cc:1048] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:26.720311 17551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:26.682780 17598 leader_election.cc:403] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:26.679713 18426 raft_consensus.cc:515] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:26.732990 18714 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:26.723268 18426 leader_election.cc:290] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:26.735146 17631 raft_consensus.cc:2393] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 337bbc857b9b4e329c817b81bfb0971c in current term 4: Already voted for candidate a9a65d4c22be403682be9bc817650ada in this term.
I20260709 13:07:26.736280 18714 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 4 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:26.741479 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:26.742307 17483 raft_consensus.cc:3055] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:26.742964 17483 raft_consensus.cc:740] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:26.743795 17483 consensus_queue.cc:260] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:26.745406 17483 raft_consensus.cc:3060] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:26.750778 18696 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 4 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:26.755491 17630 raft_consensus.cc:1275] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Refusing update from remote peer 4df2ca35be5341afa5a621233a4da94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:26.756232 18665 raft_consensus.cc:2749] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943) with higher term. Message: Invalid argument: T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a9a65d4c22be403682be9bc817650ada for earlier term 2. Current term is 3.
I20260709 13:07:26.782521 17558 raft_consensus.cc:2468] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 4.
I20260709 13:07:26.782918 18632 consensus_queue.cc:1048] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:26.784403 18299 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: a9a65d4c22be403682be9bc817650ada
I20260709 13:07:26.785861 18696 raft_consensus.cc:2804] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:26.788071 18696 raft_consensus.cc:697] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 4 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:26.789987 18696 consensus_queue.cc:237] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:26.805130 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:26.806175 17630 raft_consensus.cc:3060] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:26.811856 18426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
I20260709 13:07:26.817956 17304 catalog_manager.cc:5697] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:26.835158 18708 raft_consensus.cc:993] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e: : Instructing follower 3e3a92cfda7b4a9c8ddf43c0c9f67317 to start an election
I20260709 13:07:26.836417 18708 raft_consensus.cc:1081] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Signalling peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 to start an election
I20260709 13:07:26.843283 17483 raft_consensus.cc:2468] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:26.850669 18296 leader_election.cc:304] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:26.852195 18696 raft_consensus.cc:2804] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:26.853009 18696 raft_consensus.cc:697] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:26.854081 18696 consensus_queue.cc:237] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:26.861140 17551 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e559727ff02540749beb944b741d471f"
dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
 from {username='slave'} at 127.0.0.1:41552
I20260709 13:07:26.861704 17630 raft_consensus.cc:2468] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:26.862154 17551 raft_consensus.cc:493] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:26.862788 17551 raft_consensus.cc:3060] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:26.869731 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:26.870823 17558 raft_consensus.cc:2393] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a9a65d4c22be403682be9bc817650ada in current term 4: Already voted for candidate 337bbc857b9b4e329c817b81bfb0971c in this term.
I20260709 13:07:26.874760 17551 raft_consensus.cc:515] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:26.876564 17599 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:26.877482 17551 leader_election.cc:290] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:26.877789 17483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e559727ff02540749beb944b741d471f" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:26.878477 17483 raft_consensus.cc:3055] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:26.878830 17483 raft_consensus.cc:740] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:26.879639 17483 consensus_queue.cc:260] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:26.880684 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e559727ff02540749beb944b741d471f" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:26.880963 17483 raft_consensus.cc:3060] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:26.881407 17630 raft_consensus.cc:3060] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:26.889319 18714 raft_consensus.cc:2749] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:26.891003 17558 raft_consensus.cc:1240] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Rejecting Update request from peer 4df2ca35be5341afa5a621233a4da94e for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:26.891726 17630 raft_consensus.cc:2468] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 2.
I20260709 13:07:27.223676 17483 raft_consensus.cc:2468] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 2.
I20260709 13:07:27.225179 17522 leader_election.cc:304] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:27.226176 18743 raft_consensus.cc:2804] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:27.226750 18743 raft_consensus.cc:697] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:27.227670 18743 consensus_queue.cc:237] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:27.246536 17628 raft_consensus.cc:1275] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:27.256656 17551 raft_consensus.cc:1275] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:27.267666 18729 consensus_queue.cc:1048] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:27.277550 18730 consensus_queue.cc:1048] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:27.275350 17479 raft_consensus.cc:1275] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:27.336546 17628 raft_consensus.cc:1275] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:27.339704 18713 consensus_queue.cc:1048] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:27.341535 18713 consensus_queue.cc:1048] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:27.350333 18708 raft_consensus.cc:993] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e: : Instructing follower a9a65d4c22be403682be9bc817650ada to start an election
I20260709 13:07:27.359753 18708 raft_consensus.cc:1081] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Signalling peer a9a65d4c22be403682be9bc817650ada to start an election
I20260709 13:07:27.371727 17624 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dfc142508c684e6eb9716cb26e7047b8"
dest_uuid: "a9a65d4c22be403682be9bc817650ada"
 from {username='slave'} at 127.0.0.1:53188
I20260709 13:07:27.372563 17624 raft_consensus.cc:493] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:27.373072 17624 raft_consensus.cc:3060] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:27.435858 17450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 65019.4us
I20260709 13:07:27.496640 17624 raft_consensus.cc:515] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:27.501480 17624 leader_election.cc:290] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:27.502377 17630 raft_consensus.cc:1240] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Rejecting Update request from peer 4df2ca35be5341afa5a621233a4da94e for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:27.505417 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfc142508c684e6eb9716cb26e7047b8" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:27.507203 17479 raft_consensus.cc:3055] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:27.507804 17479 raft_consensus.cc:740] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:27.507598 18708 consensus_queue.cc:1059] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:27.510867 17479 consensus_queue.cc:260] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:27.547557 17479 raft_consensus.cc:3060] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.597273 17559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfc142508c684e6eb9716cb26e7047b8" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:27.606669 17559 raft_consensus.cc:3060] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.612074 17304 catalog_manager.cc:5697] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 2 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } }
I20260709 13:07:27.646179 17484 raft_consensus.cc:1275] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:27.648788 18631 consensus_queue.cc:1048] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:27.656185 17479 raft_consensus.cc:2468] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:27.658578  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:27.690336 17596 leader_election.cc:304] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:27.695293 18756 raft_consensus.cc:2804] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:27.700114 17559 raft_consensus.cc:2468] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:27.735920 18756 raft_consensus.cc:697] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:27.737831 18756 consensus_queue.cc:237] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:27.797271 17303 catalog_manager.cc:5697] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:27.799991 17304 catalog_manager.cc:5697] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } }
I20260709 13:07:27.812076 17628 raft_consensus.cc:1275] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:27.851426 17304 catalog_manager.cc:5697] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 1 to 2, leader changed from 4df2ca35be5341afa5a621233a4da94e (127.7.89.130) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 2 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:27.864702 18631 consensus_queue.cc:1048] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:27.886493  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:27.948246 17559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4"
dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53546
I20260709 13:07:27.948953 17559 raft_consensus.cc:606] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:27.980515 17628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb"
dest_uuid: "a9a65d4c22be403682be9bc817650ada"
mode: GRACEFUL
new_leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
 from {username='slave'} at 127.0.0.1:43858
I20260709 13:07:27.981213 17628 raft_consensus.cc:606] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Received request to transfer leadership to TS 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:27.983235  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:27.984975  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:28.267755 17485 raft_consensus.cc:1275] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:28.269536 18789 consensus_queue.cc:1048] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:28.276355 18757 raft_consensus.cc:993] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: : Instructing follower 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:28.277071 18757 raft_consensus.cc:1081] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Signalling peer 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:28.284940 18419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4"
dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:40314
I20260709 13:07:28.285732 18419 raft_consensus.cc:493] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:28.286136 18419 raft_consensus.cc:3060] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:28.300171 18459 debug-util.cc:398] Leaking SignalData structure 0x7b080016d080 after lost signal to thread 7529
W20260709 13:07:28.302119 18459 debug-util.cc:398] Leaking SignalData structure 0x7b08000ba300 after lost signal to thread 17342
W20260709 13:07:28.307278 18459 debug-util.cc:398] Leaking SignalData structure 0x7b08001a8240 after lost signal to thread 17509
W20260709 13:07:28.309381 18459 debug-util.cc:398] Leaking SignalData structure 0x7b08000d8f00 after lost signal to thread 17583
W20260709 13:07:28.324548 18459 debug-util.cc:398] Leaking SignalData structure 0x7b080011d960 after lost signal to thread 17657
W20260709 13:07:28.326679 18459 debug-util.cc:398] Leaking SignalData structure 0x7b08003c3020 after lost signal to thread 18462
I20260709 13:07:28.338992 17558 raft_consensus.cc:1275] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:28.343448 18419 raft_consensus.cc:515] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:28.348084 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:28.348801 17558 raft_consensus.cc:3055] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:28.349113 17558 raft_consensus.cc:740] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:28.351162 18419 leader_election.cc:290] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 4 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:28.351738 17624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:28.352451 17624 raft_consensus.cc:3060] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:28.350010 17558 consensus_queue.cc:260] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:28.354918 17558 raft_consensus.cc:3060] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:28.365190 18756 consensus_queue.cc:1048] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:28.377447 17624 raft_consensus.cc:2468] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 4.
I20260709 13:07:28.399245 18298 leader_election.cc:304] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:28.400627 18739 raft_consensus.cc:2804] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:28.401289 18739 raft_consensus.cc:697] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 4 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:28.402566 18739 consensus_queue.cc:237] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:28.411146 17558 raft_consensus.cc:2468] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 4.
I20260709 13:07:28.438947 17303 catalog_manager.cc:5697] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 3 to 4, leader changed from 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 4 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:28.538378 18789 raft_consensus.cc:993] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada: : Instructing follower 3e3a92cfda7b4a9c8ddf43c0c9f67317 to start an election
I20260709 13:07:28.539052 18789 raft_consensus.cc:1081] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Signalling peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 to start an election
I20260709 13:07:28.560370 17551 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb"
dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
 from {username='slave'} at 127.0.0.1:41544
I20260709 13:07:28.561232 17551 raft_consensus.cc:493] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:28.561674 17551 raft_consensus.cc:3060] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.609099 17551 raft_consensus.cc:515] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:28.612955 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:28.613768 17485 raft_consensus.cc:3060] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.625005 17485 raft_consensus.cc:2468] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 3.
I20260709 13:07:28.636122 17551 leader_election.cc:290] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: Requested vote from peers 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:28.638506 17522 leader_election.cc:304] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:28.643311 18793 raft_consensus.cc:2804] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:28.644026 18793 raft_consensus.cc:697] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:28.645119 17624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eef6e41b27ca4b229606bc1df6b023fb" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:28.645244 18793 consensus_queue.cc:237] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:28.646235 17624 raft_consensus.cc:3055] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:28.647073 17624 raft_consensus.cc:740] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:28.648279 17624 consensus_queue.cc:260] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } }
I20260709 13:07:28.650214 17624 raft_consensus.cc:3060] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.687759 17624 raft_consensus.cc:2468] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 3.
I20260709 13:07:28.704236 17303 catalog_manager.cc:5697] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 2 to 3, leader changed from a9a65d4c22be403682be9bc817650ada (127.7.89.132) to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:28.946715 17551 raft_consensus.cc:1275] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:28.948939 18739 consensus_queue.cc:1048] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:28.986110  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:28.990846 17630 raft_consensus.cc:1275] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:28.993325 18730 consensus_queue.cc:1048] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:29.091220 17485 raft_consensus.cc:1275] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:29.093218 18743 consensus_queue.cc:1048] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:29.101929 17630 raft_consensus.cc:1275] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:29.105685 18743 consensus_queue.cc:1048] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:29.122469  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:29.230034 17558 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f"
dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
mode: GRACEFUL
new_leader_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:53546
I20260709 13:07:29.230820 17558 raft_consensus.cc:606] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Received request to transfer leadership to TS 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:29.233997  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:07:29.236522  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:07:29.249188 17596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 72220.3us
W20260709 13:07:29.651500 18298 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.132:40637, user_credentials={real_user=slave}} blocked reactor thread for 48977.8us
I20260709 13:07:29.685350 18793 raft_consensus.cc:993] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: : Instructing follower 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:29.685981 18793 raft_consensus.cc:1081] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Signalling peer 337bbc857b9b4e329c817b81bfb0971c to start an election
I20260709 13:07:29.689734 18419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f"
dest_uuid: "337bbc857b9b4e329c817b81bfb0971c"
 from {username='slave'} at 127.0.0.1:40314
I20260709 13:07:29.690495 18419 raft_consensus.cc:493] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:29.690948 18419 raft_consensus.cc:3060] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:29.702993 18419 raft_consensus.cc:515] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:29.707899 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:29.708783 17558 raft_consensus.cc:3055] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:29.709589 18419 leader_election.cc:290] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637)
I20260709 13:07:29.709167 17558 raft_consensus.cc:740] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:29.712774 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f" candidate_uuid: "337bbc857b9b4e329c817b81bfb0971c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:29.713625 17628 raft_consensus.cc:3060] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:29.711941 17558 consensus_queue.cc:260] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:29.717172 17558 raft_consensus.cc:3060] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:29.743398 17558 raft_consensus.cc:2468] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:29.745438 18299 leader_election.cc:304] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 
I20260709 13:07:29.746644 18823 raft_consensus.cc:2804] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:29.747226 18823 raft_consensus.cc:697] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Becoming Leader. State: Replica: 337bbc857b9b4e329c817b81bfb0971c, State: Running, Role: LEADER
I20260709 13:07:29.749082 18823 consensus_queue.cc:237] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:29.753353 17628 raft_consensus.cc:2468] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337bbc857b9b4e329c817b81bfb0971c in term 2.
I20260709 13:07:29.849602 17304 catalog_manager.cc:5697] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c reported cstate change: term changed from 1 to 2, leader changed from 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131) to 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129). New cstate: current_term: 2 leader_uuid: "337bbc857b9b4e329c817b81bfb0971c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.237733  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:30.291540 17551 raft_consensus.cc:1275] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:30.297706 18823 consensus_queue.cc:1048] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:30.323733  7526 tablet_server.cc:179] TabletServer@127.7.89.129:34943 shutting down...
W20260709 13:07:30.327010 17598 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 381 similar messages]
I20260709 13:07:30.367189 17628 raft_consensus.cc:1275] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Refusing update from remote peer 337bbc857b9b4e329c817b81bfb0971c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:30.381888 18824 consensus_queue.cc:1048] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
W20260709 13:07:30.506628 18296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 66355.7us
W20260709 13:07:30.537088 17598 consensus_peers.cc:597] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.543028 17450 consensus_peers.cc:597] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.538215 17598 consensus_peers.cc:597] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.559371 17598 consensus_peers.cc:597] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.570971 17598 consensus_peers.cc:597] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.575536 17450 consensus_peers.cc:597] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.575654 17598 consensus_peers.cc:597] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.583716 17598 consensus_peers.cc:597] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.581388 17450 consensus_peers.cc:597] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.589838 17450 consensus_peers.cc:597] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.633313 17450 consensus_peers.cc:597] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.634900 17450 consensus_peers.cc:597] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.644495 17598 consensus_peers.cc:597] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.679791 17450 consensus_peers.cc:597] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.680433 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 73460.6us
I20260709 13:07:30.726833  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:07:30.728948 17450 consensus_peers.cc:597] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:30.730844  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.731841  7526 raft_consensus.cc:2243] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.732349  7526 raft_consensus.cc:2272] T 36168ca382614c3f9061a116e5d12df5 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.738586  7526 tablet_replica.cc:333] stopping tablet replica
W20260709 13:07:30.733239 17450 consensus_peers.cc:597] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:30.739995  7526 raft_consensus.cc:2243] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.741869  7526 raft_consensus.cc:2272] T 800f4d4505654dc58f390fcc9f0d90db P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.748817  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.750044  7526 raft_consensus.cc:2243] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.750808  7526 raft_consensus.cc:2272] T 2d047496a466464ba1777cf63c5c84b0 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.754755  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.755815  7526 raft_consensus.cc:2243] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.756443  7526 raft_consensus.cc:2272] T eea055e03bf849bea8fdc851e10a8cbe P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.761970  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.762974  7526 raft_consensus.cc:2243] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.763695  7526 raft_consensus.cc:2272] T a74eee34cc514b79b981a6ef6c45d7e2 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.767239  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.768235  7526 raft_consensus.cc:2243] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:30.769620  7526 raft_consensus.cc:2272] T 242531b3d6ad4ea4bdd979f393b59f29 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.773161  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.774152  7526 raft_consensus.cc:2243] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:30.774338 17524 consensus_peers.cc:597] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:30.774888  7526 raft_consensus.cc:2272] T fa3738962c3242b5aa69f93b778bea53 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:30.775544 17524 consensus_peers.cc:597] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.776232 17524 consensus_peers.cc:597] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.776873 17524 consensus_peers.cc:597] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:30.778875  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.780232  7526 raft_consensus.cc:2243] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.780821  7526 raft_consensus.cc:2272] T 05345b561dfb497c8f1e081b05c1bb5d P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.784442  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.785574  7526 raft_consensus.cc:2243] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.786193  7526 raft_consensus.cc:2272] T 1995e080f6c048fe935afd95f336b109 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.791322  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.792543  7526 raft_consensus.cc:2243] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.793177  7526 raft_consensus.cc:2272] T a5414b9916b148bba0932c91f3770f57 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:30.794759 17598 consensus_peers.cc:597] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:30.798671  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.800071  7526 raft_consensus.cc:2243] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.800863  7526 raft_consensus.cc:2272] T b5fb377eea3d41e68cb4f46212e6e857 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.808419  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.809391  7526 raft_consensus.cc:2243] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.810366  7526 raft_consensus.cc:2272] T a7f14faf54714af3abeff5693ace0e8c P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.814883  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.815901  7526 raft_consensus.cc:2243] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:07:30.816700 17598 consensus_peers.cc:597] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:30.817309  7526 raft_consensus.cc:2272] T 01c85566c0a845438507d71c6936d507 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:30.818900 17598 consensus_peers.cc:597] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:30.821412  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.822430  7526 raft_consensus.cc:2243] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.823120  7526 raft_consensus.cc:2272] T 20df005b7408432da9ac4fcf110a20b9 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.826601  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.828382  7526 raft_consensus.cc:2243] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.828966  7526 raft_consensus.cc:2272] T 0b2d2a20fb474604a1077ce9af500e42 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.832607  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.836158  7526 raft_consensus.cc:2243] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.836869  7526 raft_consensus.cc:2272] T 1b10c75b663e4cbcb3237b59443f77ca P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:30.841097 17524 consensus_peers.cc:597] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:30.846527  7526 tablet_replica.cc:333] stopping tablet replica
W20260709 13:07:30.849918 17524 consensus_peers.cc:597] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.852787 17524 consensus_peers.cc:597] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.853817 17524 consensus_peers.cc:597] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:30.851002  7526 raft_consensus.cc:2243] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.856758  7526 raft_consensus.cc:2272] T 5b1eabbf0fc942fabea0c59e6997ad23 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.861027  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.862021  7526 raft_consensus.cc:2243] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:30.863204  7526 raft_consensus.cc:2272] T 445a4c9a8b0441f0ad2725785483a5d4 P 337bbc857b9b4e329c817b81bfb0971c [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:30.864742 17598 consensus_peers.cc:597] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:30.867178  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.874007  7526 raft_consensus.cc:2243] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:30.876123  7526 raft_consensus.cc:2272] T 944f8419f8c94888bb53565c46d6f69b P 337bbc857b9b4e329c817b81bfb0971c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.880975  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.882117  7526 raft_consensus.cc:2243] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:30.883507  7526 raft_consensus.cc:2272] T 2226d75e22544facaa146d281f852f27 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.887850  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.889000  7526 raft_consensus.cc:2243] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.889811  7526 raft_consensus.cc:2272] T 22993c01d4634a3a92c5a2903aa07ffe P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.895815  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.897058  7526 raft_consensus.cc:2243] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.899653  7526 raft_consensus.cc:2272] T def346af33c747faaf6c9cf0414510f0 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.903726  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.911242  7526 raft_consensus.cc:2243] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.912016  7526 raft_consensus.cc:2272] T 37b734b40edd440fad746ea61f858c1b P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.940081  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.941097  7526 raft_consensus.cc:2243] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:30.946753  7526 raft_consensus.cc:2272] T 8259e46a7776417a80d1be42d4419a13 P 337bbc857b9b4e329c817b81bfb0971c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.950824  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.951850  7526 raft_consensus.cc:2243] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.952565  7526 raft_consensus.cc:2272] T 62f330683b8d4b0696fad129a8f2ce23 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:30.952759 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 76250.5us
W20260709 13:07:30.953896 17524 consensus_peers.cc:597] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:30.957998  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.959564  7526 raft_consensus.cc:2243] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:30.961506  7526 raft_consensus.cc:2272] T b1f46bc9df3a44dd927d40f3778a5ec3 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.965220  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.966284  7526 raft_consensus.cc:2243] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:30.973029  7526 raft_consensus.cc:2272] T bd407ea24c014a8690ac37978d4b0b50 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.977041  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.978137  7526 raft_consensus.cc:2243] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.984622  7526 raft_consensus.cc:2272] T 34919e11221f465cbcef1151a1c5d4b0 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.989251  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.990326  7526 raft_consensus.cc:2243] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.991130  7526 raft_consensus.cc:2272] T cac16fb87c954fd9b4cef5850ffc16e9 P 337bbc857b9b4e329c817b81bfb0971c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.996444  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:30.997584  7526 raft_consensus.cc:2243] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:31.003985  7526 raft_consensus.cc:2272] T 14ead71306714d3a9784b47ddaa7588c P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.008023  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.009179  7526 raft_consensus.cc:2243] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.026070  7526 raft_consensus.cc:2272] T 2ccac61a2bab40708cfe17c17ae1de65 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.030272  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.031435  7526 raft_consensus.cc:2243] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:31.032855  7526 raft_consensus.cc:2272] T 1e00eadc811946b48dfdf947398a84c1 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.037559  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.038594  7526 raft_consensus.cc:2243] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.039381  7526 raft_consensus.cc:2272] T 307acc498b0c4da7b7953e895b049c2a P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.042904  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.043887  7526 raft_consensus.cc:2243] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:31.045221  7526 raft_consensus.cc:2272] T 8b8583086b304aa484d68b0a8175ceb4 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.048985  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.050033  7526 raft_consensus.cc:2243] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:07:31.057250 17524 consensus_peers.cc:597] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:31.058995  7526 raft_consensus.cc:2272] T 6633559a9d2643a29622caab848e5442 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.063509  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.065099  7526 raft_consensus.cc:2243] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.066057  7526 raft_consensus.cc:2272] T 032dc48c2ae44160841c456abd340e40 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.070379  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.071604  7526 raft_consensus.cc:2243] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.072297  7526 raft_consensus.cc:2272] T 050a17ead1c04486b472a25b0bb179ca P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.075856  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.076891  7526 raft_consensus.cc:2243] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.077680  7526 raft_consensus.cc:2272] T 61235f863ca44ed3b6c4715dfd68e223 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.081084  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.083388  7526 raft_consensus.cc:2243] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.084221  7526 raft_consensus.cc:2272] T f55f8c5bd47d4a31a9562ab2b75ab56f P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.107861  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.116292  7526 raft_consensus.cc:2243] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.117084  7526 raft_consensus.cc:2272] T e84a923caa9c4755bcb78a35143dfbd9 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:31.118575 17524 consensus_peers.cc:597] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:31.123539  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.124749  7526 raft_consensus.cc:2243] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.125530  7526 raft_consensus.cc:2272] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.153076  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.154431  7526 raft_consensus.cc:2243] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.155158  7526 raft_consensus.cc:2272] T abe7eea0c47a4c6998274f4b6d14dfb7 P 337bbc857b9b4e329c817b81bfb0971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.182593  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.183498  7526 raft_consensus.cc:2243] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.184240  7526 raft_consensus.cc:2272] T 0f7e0518788149c19cf493a65bc598c2 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.189392  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.190204  7526 raft_consensus.cc:2243] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:31.191375  7526 raft_consensus.cc:2272] T 0b795b9f53f64ad387b207e8da82ff6f P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.202945  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:31.203733  7526 raft_consensus.cc:2243] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:31.204912  7526 raft_consensus.cc:2272] T 6170cc45e2ea48598650c74f9c06b644 P 337bbc857b9b4e329c817b81bfb0971c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:31.350754 17598 connection.cc:769] client connection to 127.7.89.129:34943 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260709 13:07:31.809859 18839 raft_consensus.cc:493] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:31.810689 18839 raft_consensus.cc:515] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:31.812790 18839 leader_election.cc:290] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:31.813825 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c85566c0a845438507d71c6936d507" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:07:31.817363 17450 leader_election.cc:336] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:31.817905 17450 leader_election.cc:304] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:31.818816 18822 raft_consensus.cc:2749] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:32.001947 18793 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.002547 18793 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.005076 18793 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.006796 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:07:32.010121 17524 leader_election.cc:336] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.010715 17524 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:32.011471 18793 raft_consensus.cc:2749] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:32.042698 18822 raft_consensus.cc:493] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.043329 18822 raft_consensus.cc:515] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.045686 18822 leader_election.cc:290] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.048224 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:07:32.052886 17450 leader_election.cc:336] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.053530 17450 leader_election.cc:304] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:32.056625 18822 raft_consensus.cc:2749] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:32.122238 18822 raft_consensus.cc:493] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.122900 18822 raft_consensus.cc:515] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.125382 18822 leader_election.cc:290] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.126880 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:07:32.136245 17450 leader_election.cc:336] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.136900 17450 leader_election.cc:304] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:32.138230 18822 raft_consensus.cc:2749] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:32.154824 18842 raft_consensus.cc:493] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.155769 18842 raft_consensus.cc:515] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.155808 18858 raft_consensus.cc:493] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.157931 18858 raft_consensus.cc:515] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.159302 18847 raft_consensus.cc:493] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.159984 18847 raft_consensus.cc:515] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.162765 18793 raft_consensus.cc:493] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.163446 18793 raft_consensus.cc:515] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:32.163702 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:32.164433 17479 raft_consensus.cc:2468] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:32.165926 18793 leader_election.cc:290] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.166365 17596 leader_election.cc:304] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:32.168949 18847 leader_election.cc:290] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.169701 18847 raft_consensus.cc:2804] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:32.170140 18847 raft_consensus.cc:493] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.171353 18847 raft_consensus.cc:3060] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:32.175850 17598 leader_election.cc:336] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
W20260709 13:07:32.176698 17598 leader_election.cc:336] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.172013 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:32.177943 18793 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.178589 18793 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.179729 18858 leader_election.cc:290] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.180546 18858 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.181120 18858 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.183648 18793 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.170885 18842 leader_election.cc:290] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.191761 18858 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
W20260709 13:07:32.193481 17598 leader_election.cc:336] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.199992 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:32.203037 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c85566c0a845438507d71c6936d507" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:32.203591 18861 raft_consensus.cc:493] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.203981 17484 raft_consensus.cc:2468] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:32.204277 18861 raft_consensus.cc:515] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
W20260709 13:07:32.208149 17524 leader_election.cc:336] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.210980 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:32.211637 17631 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 4.
W20260709 13:07:32.208266 17598 leader_election.cc:336] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.209021 18861 leader_election.cc:290] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.209445 17599 leader_election.cc:304] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:32.209792 18847 raft_consensus.cc:515] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.213002 18859 raft_consensus.cc:2749] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:32.215255 17596 leader_election.cc:304] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.210129 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:32.215684 18839 raft_consensus.cc:493] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.216387 18839 raft_consensus.cc:515] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.218992 18859 raft_consensus.cc:2804] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:32.219744 18859 raft_consensus.cc:493] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.218912 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b8583086b304aa484d68b0a8175ceb4" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:32.220150 18859 raft_consensus.cc:3060] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.220162 18839 leader_election.cc:290] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.220602 17484 raft_consensus.cc:3060] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.214558 18847 leader_election.cc:290] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.223063 18847 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.223582 18847 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.224292 18839 raft_consensus.cc:493] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.224951 18839 raft_consensus.cc:515] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:32.227978 18839 leader_election.cc:290] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:32.228503 17524 leader_election.cc:304] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:32.231474 18861 raft_consensus.cc:2749] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:32.234836 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
W20260709 13:07:32.235636 17598 leader_election.cc:336] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.236958 18860 raft_consensus.cc:493] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.237298 18847 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.240514 18859 raft_consensus.cc:515] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:07:32.236377 17524 leader_election.cc:336] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.244612 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c85566c0a845438507d71c6936d507" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:32.245391 17479 raft_consensus.cc:3060] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.257984 17479 raft_consensus.cc:2468] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 3.
I20260709 13:07:32.259075 18860 raft_consensus.cc:515] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.263064 17485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:07:32.263761 17598 leader_election.cc:336] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.263825 17485 raft_consensus.cc:2468] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:32.265925 17596 leader_election.cc:304] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:32.267287 18860 leader_election.cc:290] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.269582 18860 raft_consensus.cc:2804] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:32.270216 18860 raft_consensus.cc:493] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.270792 18860 raft_consensus.cc:3060] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.275156 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:32.276204 17554 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 4.
I20260709 13:07:32.276206 18859 leader_election.cc:290] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.267985 18862 raft_consensus.cc:493] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.268714 17596 leader_election.cc:304] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:32.278376 18862 raft_consensus.cc:515] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.279044 18859 raft_consensus.cc:2804] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:32.279942 18859 raft_consensus.cc:697] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 3 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:32.281414 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:32.281196 18859 consensus_queue.cc:237] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.282213 17554 raft_consensus.cc:2468] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 4.
W20260709 13:07:32.285295 17598 leader_election.cc:336] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.287528 18862 leader_election.cc:290] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.292336 18860 raft_consensus.cc:515] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.294251 17599 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.295233 18847 raft_consensus.cc:2804] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:32.295768 18847 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.296308 18847 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.298986 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:32.299669 17599 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.300663 18858 raft_consensus.cc:2804] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:32.300861 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242531b3d6ad4ea4bdd979f393b59f29" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:32.301302 18858 raft_consensus.cc:493] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.301781 17479 raft_consensus.cc:3060] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.301936 18858 raft_consensus.cc:3060] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.306605 17484 raft_consensus.cc:2468] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 3.
I20260709 13:07:32.307229 18793 raft_consensus.cc:493] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.307876 18793 raft_consensus.cc:515] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.308187 17596 leader_election.cc:304] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.311163 18842 raft_consensus.cc:2804] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:32.311978 18842 raft_consensus.cc:697] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 3 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:32.313004 18842 consensus_queue.cc:237] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:07:32.314499 17450 leader_election.cc:336] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.315301 17450 leader_election.cc:304] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
W20260709 13:07:32.323501 17450 leader_election.cc:336] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.332696 18793 leader_election.cc:290] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
W20260709 13:07:32.333108 17450 leader_election.cc:336] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.333618 17450 leader_election.cc:304] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
W20260709 13:07:32.334450 17598 leader_election.cc:336] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.337991 18862 raft_consensus.cc:2749] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:32.346580 18860 leader_election.cc:290] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.350057 17479 raft_consensus.cc:2468] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 3.
I20260709 13:07:32.351097 18839 raft_consensus.cc:2749] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:07:32.353921 17596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:36599, user_credentials={real_user=slave}} blocked reactor thread for 45866.8us
I20260709 13:07:32.354290 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:32.354653 18865 raft_consensus.cc:493] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.356112 17524 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.357690 18861 raft_consensus.cc:2804] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:32.355432 18865 raft_consensus.cc:515] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:32.358243 18861 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.358814 17554 raft_consensus.cc:2468] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 3.
I20260709 13:07:32.358819 18861 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.361574 18847 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.365550 18847 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
W20260709 13:07:32.369899  7526 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 18459)
I20260709 13:07:32.372598 17596 leader_election.cc:304] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:32.370554 18793 raft_consensus.cc:493] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.371232 17451 leader_election.cc:304] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.375924 18793 raft_consensus.cc:515] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.376823 18839 raft_consensus.cc:2804] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:32.377386 18839 raft_consensus.cc:493] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.377849 18839 raft_consensus.cc:3060] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:32.378527 18793 leader_election.cc:290] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
W20260709 13:07:32.371170 17524 leader_election.cc:336] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.379634 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:32.379876 17524 leader_election.cc:304] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:32.380668 18793 raft_consensus.cc:2749] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:32.378981 18865 leader_election.cc:290] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
W20260709 13:07:32.383913 17598 leader_election.cc:336] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.385694 18858 raft_consensus.cc:515] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.388708 18858 leader_election.cc:290] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.388281 17303 catalog_manager.cc:5697] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 2 to 3, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 3 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.389873 18839 raft_consensus.cc:515] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:32.391135 17479 raft_consensus.cc:2393] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 4: Already voted for candidate 4df2ca35be5341afa5a621233a4da94e in this term.
I20260709 13:07:32.394846 18842 raft_consensus.cc:2804] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:32.395552 18842 raft_consensus.cc:697] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 3 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:32.396078 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:32.398468 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "944f8419f8c94888bb53565c46d6f69b" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:32.399092 18847 raft_consensus.cc:493] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.399358 17555 raft_consensus.cc:3060] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.399788 18847 raft_consensus.cc:515] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:07:32.400368 17450 leader_election.cc:336] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.401530 18822 raft_consensus.cc:493] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.401700 18839 leader_election.cc:290] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 4 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:32.402143 18822 raft_consensus.cc:515] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:32.404165 18847 leader_election.cc:290] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.409912 18822 leader_election.cc:290] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:32.410166 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:32.411029 17479 raft_consensus.cc:2468] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:32.396613 18842 consensus_queue.cc:237] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:07:32.412709 17450 leader_election.cc:336] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.420701 17630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:32.421224 17555 raft_consensus.cc:2468] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 5.
I20260709 13:07:32.421567 17630 raft_consensus.cc:2468] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 4.
I20260709 13:07:32.421885 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:32.421790 18861 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.424218 18861 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.424798 17554 raft_consensus.cc:2393] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a9a65d4c22be403682be9bc817650ada in current term 5: Already voted for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in this term.
W20260709 13:07:32.420567 17598 leader_election.cc:336] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.427801 17596 leader_election.cc:304] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
W20260709 13:07:32.428416 17524 leader_election.cc:336] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.429006 18868 raft_consensus.cc:2804] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:07:32.429129 17524 leader_election.cc:336] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.429553 18868 raft_consensus.cc:493] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.429700 17524 leader_election.cc:304] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:32.429992 18868 raft_consensus.cc:3060] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.433892 18861 raft_consensus.cc:3060] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:32.435410 17598 leader_election.cc:336] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
W20260709 13:07:32.436218 17598 leader_election.cc:336] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.440909 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:32.441398 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd407ea24c014a8690ac37978d4b0b50" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:32.446048 18822 raft_consensus.cc:493] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.446914 17599 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 3e3a92cfda7b4a9c8ddf43c0c9f67317
I20260709 13:07:32.447198 18822 raft_consensus.cc:515] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:32.441845 17554 raft_consensus.cc:2468] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 2.
I20260709 13:07:32.445947 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:32.448988 17631 raft_consensus.cc:2393] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in current term 5: Already voted for candidate a9a65d4c22be403682be9bc817650ada in this term.
I20260709 13:07:32.449635 18822 leader_election.cc:290] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
W20260709 13:07:32.456432 17450 leader_election.cc:336] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.460698 18859 raft_consensus.cc:2749] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:32.462198 17599 leader_election.cc:304] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.463531 18859 raft_consensus.cc:2804] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:32.464089 18859 raft_consensus.cc:697] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 5 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:32.465730 18859 consensus_queue.cc:237] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.473877 17524 leader_election.cc:304] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.475617 18838 raft_consensus.cc:2804] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:32.476239 18838 raft_consensus.cc:493] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.476783 18838 raft_consensus.cc:3060] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.477744 18868 raft_consensus.cc:515] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:07:32.488611 17598 leader_election.cc:336] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.489208 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2226d75e22544facaa146d281f852f27" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:32.490139 17479 raft_consensus.cc:3060] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:32.488519 17524 leader_election.cc:336] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.493899 17524 leader_election.cc:304] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:32.500602 18865 raft_consensus.cc:2749] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:32.500357 18838 raft_consensus.cc:515] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.501439 18859 raft_consensus.cc:493] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.502085 18859 raft_consensus.cc:515] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.501526 18868 leader_election.cc:290] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.503011 18838 leader_election.cc:290] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.505250 17451 leader_election.cc:304] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.509666 18822 raft_consensus.cc:2804] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:32.510149 18822 raft_consensus.cc:493] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.510547 18822 raft_consensus.cc:3060] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.478678 18861 raft_consensus.cc:2749] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:07:32.513116 17598 leader_election.cc:336] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.514114 18859 leader_election.cc:290] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.516901 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:32.517786 17554 raft_consensus.cc:2468] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 2.
I20260709 13:07:32.519613 18868 raft_consensus.cc:493] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.520287 18868 raft_consensus.cc:515] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.523147 18859 raft_consensus.cc:493] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.523989 18859 raft_consensus.cc:515] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.526408 17479 raft_consensus.cc:2468] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 3.
I20260709 13:07:32.528602 17555 raft_consensus.cc:2468] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 4.
I20260709 13:07:32.530416 17484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6633559a9d2643a29622caab848e5442" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
W20260709 13:07:32.538462 17598 leader_election.cc:336] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.539254 17596 leader_election.cc:304] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
W20260709 13:07:32.539475 17598 leader_election.cc:336] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.540270 18858 raft_consensus.cc:2804] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:32.540601 17596 leader_election.cc:304] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c, 4df2ca35be5341afa5a621233a4da94e
I20260709 13:07:32.540834 18858 raft_consensus.cc:697] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 3 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:32.531945 17631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445a4c9a8b0441f0ad2725785483a5d4" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
I20260709 13:07:32.541846 17631 raft_consensus.cc:3060] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.546965 18842 raft_consensus.cc:2749] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:32.542002 18858 consensus_queue.cc:237] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
W20260709 13:07:32.533107 17524 leader_election.cc:336] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.551510 17524 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317; no voters: 337bbc857b9b4e329c817b81bfb0971c, a9a65d4c22be403682be9bc817650ada
I20260709 13:07:32.554864 18861 raft_consensus.cc:2749] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:32.557340 18862 raft_consensus.cc:493] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.558497 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:32.559348 17479 raft_consensus.cc:2468] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:32.561957 17596 leader_election.cc:304] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.562866 18847 raft_consensus.cc:2804] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:32.563318 18847 raft_consensus.cc:493] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.563746 18847 raft_consensus.cc:3060] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.565549 18862 raft_consensus.cc:515] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.569424 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6633559a9d2643a29622caab848e5442" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada" is_pre_election: true
I20260709 13:07:32.570044 17631 raft_consensus.cc:2468] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 5.
I20260709 13:07:32.570269 17628 raft_consensus.cc:2468] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 2.
I20260709 13:07:32.571900 17450 leader_election.cc:304] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:32.572896 18839 raft_consensus.cc:2804] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:32.573348 18839 raft_consensus.cc:493] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.573745 18839 raft_consensus.cc:3060] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.574949 18862 leader_election.cc:290] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.563881 17303 catalog_manager.cc:5697] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 2 to 3, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 3 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:32.579290 17450 leader_election.cc:336] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.580052 18822 raft_consensus.cc:515] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:32.582798 18822 leader_election.cc:290] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:32.533605 18859 leader_election.cc:290] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.577980 17303 catalog_manager.cc:5697] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 2 to 3, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 3 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.585070 17303 catalog_manager.cc:5697] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 4 to 5, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 5 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.558470 18868 leader_election.cc:290] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.588587 17451 leader_election.cc:304] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.592020 17554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:32.592890 17554 raft_consensus.cc:2468] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 2.
I20260709 13:07:32.593958 18863 raft_consensus.cc:2804] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:32.594455 18863 raft_consensus.cc:493] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.594911 18863 raft_consensus.cc:3060] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.594995 18847 raft_consensus.cc:515] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.598551 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e00eadc811946b48dfdf947398a84c1" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:32.599227 17479 raft_consensus.cc:3060] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:32.604914 17598 leader_election.cc:336] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.612967 17479 raft_consensus.cc:2468] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 3.
W20260709 13:07:32.614117 17450 leader_election.cc:336] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.617158 17596 leader_election.cc:304] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.619387 18858 raft_consensus.cc:2804] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:32.620811 18858 raft_consensus.cc:697] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 3 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:32.625337 18822 raft_consensus.cc:493] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.627005 18847 leader_election.cc:290] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.628087 18863 raft_consensus.cc:515] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:32.630553 18863 leader_election.cc:290] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:32.631165 18822 raft_consensus.cc:515] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:32.625437 18858 consensus_queue.cc:237] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.639616 18839 raft_consensus.cc:515] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.640373 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1f46bc9df3a44dd927d40f3778a5ec3" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:32.641093 17558 raft_consensus.cc:3060] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.642109 18839 leader_election.cc:290] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 election: Requested vote from peers a9a65d4c22be403682be9bc817650ada (127.7.89.132:40637), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.646344 17628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6633559a9d2643a29622caab848e5442" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a9a65d4c22be403682be9bc817650ada"
W20260709 13:07:32.647030 17450 leader_election.cc:336] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.647243 17628 raft_consensus.cc:3060] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:32.647732 17450 leader_election.cc:336] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.649338 18822 leader_election.cc:290] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851)
I20260709 13:07:32.654110 17552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ead71306714d3a9784b47ddaa7588c" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:32.654884 18861 raft_consensus.cc:493] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.654350 17555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "4df2ca35be5341afa5a621233a4da94e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:32.655004 17552 raft_consensus.cc:3060] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:32.655516 17450 leader_election.cc:336] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.655625 18861 raft_consensus.cc:515] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:32.659039 17555 raft_consensus.cc:2468] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 2.
I20260709 13:07:32.659633 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e" is_pre_election: true
I20260709 13:07:32.660359 17479 raft_consensus.cc:2468] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 2.
I20260709 13:07:32.661463 17522 leader_election.cc:304] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:32.662354 18865 raft_consensus.cc:2804] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:32.662806 18865 raft_consensus.cc:493] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.663199 18865 raft_consensus.cc:3060] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.668689 17552 raft_consensus.cc:2468] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 3.
I20260709 13:07:32.673036 18861 leader_election.cc:290] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.673355 17628 raft_consensus.cc:2468] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 3.
I20260709 13:07:32.680218 17451 leader_election.cc:304] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.686196 18862 raft_consensus.cc:2804] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:32.686314 17558 raft_consensus.cc:2468] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df2ca35be5341afa5a621233a4da94e in term 3.
I20260709 13:07:32.686856 18862 raft_consensus.cc:697] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 4 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:32.687871 18862 consensus_queue.cc:237] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:32.688983 17599 leader_election.cc:304] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.691370 18847 raft_consensus.cc:2804] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:32.691910 18847 raft_consensus.cc:493] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Starting leader election (detected failure of leader 337bbc857b9b4e329c817b81bfb0971c)
I20260709 13:07:32.692379 18847 raft_consensus.cc:3060] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.678707 18865 raft_consensus.cc:515] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:32.711138 17479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6170cc45e2ea48598650c74f9c06b644" candidate_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4df2ca35be5341afa5a621233a4da94e"
I20260709 13:07:32.712076 17479 raft_consensus.cc:3060] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.710381 17303 catalog_manager.cc:5697] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 2 to 3, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 3 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.713924 17303 catalog_manager.cc:5697] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 2 to 3, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 3 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.731173 17450 leader_election.cc:304] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df2ca35be5341afa5a621233a4da94e, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.732839 18822 raft_consensus.cc:2804] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:32.733461 18822 raft_consensus.cc:697] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 3 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:32.734460 18822 consensus_queue.cc:237] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } }
I20260709 13:07:32.751497 18865 leader_election.cc:290] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: Requested vote from peers 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943), 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599)
I20260709 13:07:32.754892 18847 raft_consensus.cc:515] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.757477 18847 leader_election.cc:290] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:32.762310 17303 catalog_manager.cc:5697] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 3 to 4, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 4 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:32.765089 17598 leader_election.cc:336] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
W20260709 13:07:32.776336 17598 consensus_peers.cc:597] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:32.777042 17598 consensus_peers.cc:597] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:32.779973 17524 leader_election.cc:304] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:32.780911 18865 raft_consensus.cc:2804] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:32.781036 17479 raft_consensus.cc:2468] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3a92cfda7b4a9c8ddf43c0c9f67317 in term 3.
I20260709 13:07:32.781483 18865 raft_consensus.cc:697] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 5 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:32.782328 17522 leader_election.cc:304] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 
I20260709 13:07:32.782429 18865 consensus_queue.cc:237] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.783552 18793 raft_consensus.cc:2804] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:32.784104 18793 raft_consensus.cc:697] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Becoming Leader. State: Replica: 3e3a92cfda7b4a9c8ddf43c0c9f67317, State: Running, Role: LEADER
I20260709 13:07:32.785094 18793 consensus_queue.cc:237] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:32.794265 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b795b9f53f64ad387b207e8da82ff6f" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:32.795022 17558 raft_consensus.cc:3060] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.796325 17303 catalog_manager.cc:5697] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 2 to 3, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 3 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.804118 17558 raft_consensus.cc:2468] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 3.
I20260709 13:07:32.807968 17451 leader_election.cc:304] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.809346 18839 raft_consensus.cc:2764] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:07:32.812023 17524 leader_election.cc:336] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.815231 17451 leader_election.cc:304] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
W20260709 13:07:32.815783 17450 consensus_peers.cc:597] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:32.816567 18839 raft_consensus.cc:2804] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:32.817152 18839 raft_consensus.cc:697] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 3 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:32.818076 18839 consensus_queue.cc:237] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
I20260709 13:07:32.820042 17451 leader_election.cc:304] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, 4df2ca35be5341afa5a621233a4da94e; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.821301 18876 raft_consensus.cc:2804] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:32.821857 18876 raft_consensus.cc:697] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 3 LEADER]: Becoming Leader. State: Replica: 4df2ca35be5341afa5a621233a4da94e, State: Running, Role: LEADER
I20260709 13:07:32.822872 18876 consensus_queue.cc:237] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } }
W20260709 13:07:32.865597 17524 leader_election.cc:336] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:32.871654 17303 catalog_manager.cc:5697] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 4 to 5, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 5 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.876523 17599 leader_election.cc:304] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:32.878885 18842 raft_consensus.cc:2804] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:32.879469 18842 raft_consensus.cc:697] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 3 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:32.880528 18842 consensus_queue.cc:237] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:32.885735 17479 raft_consensus.cc:1275] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:32.883213 17306 catalog_manager.cc:5697] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 2 to 3, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 3 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.873726 17303 catalog_manager.cc:5697] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 reported cstate change: term changed from 2 to 3, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:32.894093 17598 consensus_peers.cc:597] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:32.895462 18842 consensus_queue.cc:1048] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:32.904721 17479 raft_consensus.cc:1275] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:07:32.912077 17598 consensus_peers.cc:597] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:32.912784 17450 consensus_peers.cc:597] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:32.921943 17304 catalog_manager.cc:5697] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 2 to 3, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 3 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:32.930922 17524 leader_election.cc:336] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
W20260709 13:07:32.932649 17524 consensus_peers.cc:597] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:32.942921 17524 consensus_peers.cc:597] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:32.943495 17524 consensus_peers.cc:597] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:32.955688 17598 consensus_peers.cc:597] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:32.955925 18859 consensus_queue.cc:1048] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:32.954517 17304 catalog_manager.cc:5697] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e reported cstate change: term changed from 2 to 3, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to 4df2ca35be5341afa5a621233a4da94e (127.7.89.130). New cstate: current_term: 3 leader_uuid: "4df2ca35be5341afa5a621233a4da94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.978803 17555 raft_consensus.cc:1275] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 5 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:07:32.984118 18859 consensus_queue.cc:1048] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:07:32.996140 17524 consensus_peers.cc:597] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.008895 17598 consensus_peers.cc:597] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:33.010783 17479 raft_consensus.cc:1275] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:33.012575 18842 consensus_queue.cc:1048] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:07:33.043359 17450 consensus_peers.cc:597] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.067621 17598 consensus_peers.cc:597] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:33.073869 17484 raft_consensus.cc:1275] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:33.080291 18842 consensus_queue.cc:1048] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
W20260709 13:07:33.082903 17598 consensus_peers.cc:597] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:33.092921 17450 consensus_peers.cc:597] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.094048 17598 consensus_peers.cc:597] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:33.095048 17598 consensus_peers.cc:597] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:33.095710 17598 consensus_peers.cc:597] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:33.098193 17479 raft_consensus.cc:1275] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:33.101403 18842 consensus_queue.cc:1048] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:07:33.116288 17524 consensus_peers.cc:597] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.135097 17598 consensus_peers.cc:597] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.135943 17598 consensus_peers.cc:597] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:33.137645 17598 consensus_peers.cc:597] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:33.149889 17631 raft_consensus.cc:1275] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Refusing update from remote peer 4df2ca35be5341afa5a621233a4da94e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:33.151952 18863 consensus_queue.cc:1048] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:07:33.173741 17524 consensus_peers.cc:597] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.179550 17450 consensus_peers.cc:597] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:33.186033 17598 consensus_peers.cc:597] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.190557 17524 consensus_peers.cc:597] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.191335 17524 consensus_peers.cc:597] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:33.213778 17524 consensus_peers.cc:597] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:33.214522 17524 consensus_peers.cc:597] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.219318 17524 consensus_peers.cc:597] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:33.229406 17554 raft_consensus.cc:1275] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Refusing update from remote peer 4df2ca35be5341afa5a621233a4da94e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:33.230957 18822 consensus_queue.cc:1048] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:07:33.263512 17450 consensus_peers.cc:597] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:33.266815 17628 raft_consensus.cc:1275] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:07:33.279393 18793 consensus_queue.cc:1048] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:07:33.299144 17598 consensus_peers.cc:597] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.300128 17450 consensus_peers.cc:597] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.308897 17450 consensus_peers.cc:597] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.309808 17450 consensus_peers.cc:597] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:33.322829 17554 raft_consensus.cc:1275] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:07:33.343647 17598 consensus_peers.cc:597] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:33.346988 18858 consensus_queue.cc:1048] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:33.352483 17552 raft_consensus.cc:1275] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Refusing update from remote peer 4df2ca35be5341afa5a621233a4da94e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:33.355914 17479 raft_consensus.cc:1275] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Refusing update from remote peer 3e3a92cfda7b4a9c8ddf43c0c9f67317: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:33.357767 18793 consensus_queue.cc:1048] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df2ca35be5341afa5a621233a4da94e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:07:33.363952 17450 consensus_peers.cc:597] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.370520  7526 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 18459)
I20260709 13:07:33.372607 18822 consensus_queue.cc:1048] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
W20260709 13:07:33.392263 17598 consensus_peers.cc:597] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:33.399435 17450 consensus_peers.cc:597] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:33.369972 17524 consensus_peers.cc:597] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:33.420328 17554 raft_consensus.cc:1275] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Refusing update from remote peer 4df2ca35be5341afa5a621233a4da94e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:33.426998 18862 consensus_queue.cc:1048] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:07:33.451305 17450 consensus_peers.cc:597] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:33.473140 17450 consensus_peers.cc:597] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.658995 17524 consensus_peers.cc:597] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:33.992065 18868 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.992766 18868 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:33.995424 18868 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:33.997038 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" is_pre_election: true
I20260709 13:07:33.997891 17558 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 5.
I20260709 13:07:33.999591 17599 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 
I20260709 13:07:34.000761 18842 raft_consensus.cc:2804] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:34.001317 18842 raft_consensus.cc:493] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.001729 18842 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:34.007445 17598 leader_election.cc:336] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:34.011261 18842 raft_consensus.cc:515] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:34.015372 18842 leader_election.cc:290] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 6 election: Requested vote from peers 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851), 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943)
I20260709 13:07:34.020052 17558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8259e46a7776417a80d1be42d4419a13" candidate_uuid: "a9a65d4c22be403682be9bc817650ada" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317"
I20260709 13:07:34.020867 17558 raft_consensus.cc:3060] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:34.022161 17598 leader_election.cc:336] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111)
I20260709 13:07:34.029876 17558 raft_consensus.cc:2468] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a65d4c22be403682be9bc817650ada in term 6.
I20260709 13:07:34.031343 17599 leader_election.cc:304] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e3a92cfda7b4a9c8ddf43c0c9f67317, a9a65d4c22be403682be9bc817650ada; no voters: 337bbc857b9b4e329c817b81bfb0971c
I20260709 13:07:34.032305 18842 raft_consensus.cc:2804] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:34.032830 18842 raft_consensus.cc:697] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 6 LEADER]: Becoming Leader. State: Replica: a9a65d4c22be403682be9bc817650ada, State: Running, Role: LEADER
I20260709 13:07:34.033644 18842 consensus_queue.cc:237] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } }
I20260709 13:07:34.045764 17304 catalog_manager.cc:5697] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada reported cstate change: term changed from 4 to 6, leader changed from 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129) to a9a65d4c22be403682be9bc817650ada (127.7.89.132). New cstate: current_term: 6 leader_uuid: "a9a65d4c22be403682be9bc817650ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a65d4c22be403682be9bc817650ada" member_type: VOTER last_known_addr { host: "127.7.89.132" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337bbc857b9b4e329c817b81bfb0971c" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 34943 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:34.377586  7526 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 18459)
W20260709 13:07:34.481128 17598 consensus_peers.cc:597] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:34.558240 17554 raft_consensus.cc:1275] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 6 FOLLOWER]: Refusing update from remote peer a9a65d4c22be403682be9bc817650ada: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:34.560709 18868 consensus_queue.cc:1048] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3a92cfda7b4a9c8ddf43c0c9f67317" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 40851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:07:34.806542 17524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.89.129:34943, user_credentials={real_user=slave}} blocked reactor thread for 61458.3us
W20260709 13:07:35.120309 17598 consensus_peers.cc:597] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.187738 17450 consensus_peers.cc:597] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.215493 17450 consensus_peers.cc:597] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.286900 17524 consensus_peers.cc:597] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.292259 17598 consensus_peers.cc:597] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:35.333160 17598 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111) [suppressed 413 similar messages]
W20260709 13:07:35.378837  7526 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 18459)
W20260709 13:07:35.391391 17450 consensus_peers.cc:597] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.397086 17598 consensus_peers.cc:597] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:35.456041 17598 consensus_peers.cc:597] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.458345 17598 consensus_peers.cc:597] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.501024 17524 consensus_peers.cc:597] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.504886 17598 consensus_peers.cc:597] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.541005 17450 consensus_peers.cc:597] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.561882 17524 consensus_peers.cc:597] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.566071 17450 consensus_peers.cc:597] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.584496 17524 consensus_peers.cc:597] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.635397 17598 consensus_peers.cc:597] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.639557 17598 consensus_peers.cc:597] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.652904 17524 consensus_peers.cc:597] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.660418 17524 consensus_peers.cc:597] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.664535 17524 consensus_peers.cc:597] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:35.697501 17524 consensus_peers.cc:597] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:35.700418 17450 consensus_peers.cc:597] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.710654 17598 consensus_peers.cc:597] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:35.712209 17450 consensus_peers.cc:597] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:35.743717 17524 consensus_peers.cc:597] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.744169 17450 consensus_peers.cc:597] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:35.751936 17598 consensus_peers.cc:597] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.755978 17598 consensus_peers.cc:597] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.760887 17598 consensus_peers.cc:597] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:35.761339 17450 consensus_peers.cc:597] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.764250 17598 consensus_peers.cc:597] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:35.789325 17598 consensus_peers.cc:597] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:35.790422 17524 consensus_peers.cc:597] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.834251 17598 consensus_peers.cc:597] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.834276 17524 consensus_peers.cc:597] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.892997 17598 consensus_peers.cc:597] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.906512 17450 consensus_peers.cc:597] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:35.918709 17450 consensus_peers.cc:597] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.949561 17450 consensus_peers.cc:597] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.991955 17450 consensus_peers.cc:597] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:36.011864 17524 consensus_peers.cc:597] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:36.083189 17598 consensus_peers.cc:597] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:36.106321 17598 consensus_peers.cc:597] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:36.122385 17524 consensus_peers.cc:597] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:36.279951  7526 tablet_server.cc:196] TabletServer@127.7.89.129:34943 shutdown complete.
I20260709 13:07:36.400467  7526 tablet_server.cc:179] TabletServer@127.7.89.130:0 shutting down...
W20260709 13:07:36.411692 17596 consensus_peers.cc:597] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.427295 17596 consensus_peers.cc:597] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:36.467187  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:36.468331  7526 tablet_replica.cc:333] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.469350  7526 raft_consensus.cc:2243] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.470381  7526 raft_consensus.cc:2272] T f55f8c5bd47d4a31a9562ab2b75ab56f P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.473369  7526 tablet_replica.cc:333] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.474290  7526 raft_consensus.cc:2243] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.474944  7526 raft_consensus.cc:2272] T c295096a121e4940b38125740cea67cb P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:36.478142 17596 consensus_peers.cc:597] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:36.478417  7526 tablet_replica.cc:333] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.479650  7526 raft_consensus.cc:2243] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.481047  7526 raft_consensus.cc:2272] T cdcc480cc6534780aa5a81e4c97e8bef P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:36.482515 17522 consensus_peers.cc:597] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:36.484531  7526 tablet_replica.cc:333] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.485636  7526 raft_consensus.cc:2243] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.486786  7526 raft_consensus.cc:2272] T 1b10c75b663e4cbcb3237b59443f77ca P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.490109  7526 tablet_replica.cc:333] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.491021  7526 raft_consensus.cc:2243] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.491616  7526 raft_consensus.cc:2272] T 0b2d2a20fb474604a1077ce9af500e42 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.494966  7526 tablet_replica.cc:333] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.495894  7526 raft_consensus.cc:2243] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.497138  7526 raft_consensus.cc:2272] T 20df005b7408432da9ac4fcf110a20b9 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:36.497843 17522 consensus_peers.cc:597] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:36.500670  7526 tablet_replica.cc:333] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.502063  7526 raft_consensus.cc:2243] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.503324  7526 raft_consensus.cc:2272] T def346af33c747faaf6c9cf0414510f0 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:36.504379 17596 consensus_peers.cc:597] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:36.506418  7526 tablet_replica.cc:333] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.507388  7526 raft_consensus.cc:2243] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.508435  7526 raft_consensus.cc:2272] T 22993c01d4634a3a92c5a2903aa07ffe P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.511811  7526 tablet_replica.cc:333] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.513381  7526 raft_consensus.cc:2243] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.514257  7526 raft_consensus.cc:2272] T fa3738962c3242b5aa69f93b778bea53 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:36.517796 17596 consensus_peers.cc:597] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:36.517822  7526 tablet_replica.cc:333] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.519731  7526 raft_consensus.cc:2243] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.520344  7526 raft_consensus.cc:2272] T dfc142508c684e6eb9716cb26e7047b8 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.523263  7526 tablet_replica.cc:333] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.524312  7526 raft_consensus.cc:2243] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.525141  7526 raft_consensus.cc:2272] T 6170cc45e2ea48598650c74f9c06b644 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.527855  7526 tablet_replica.cc:333] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.528815  7526 raft_consensus.cc:2243] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.530151  7526 raft_consensus.cc:2272] T b85fb92208674e2ba20983a453290a8b P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.533617  7526 tablet_replica.cc:333] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.534703  7526 raft_consensus.cc:2243] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.535425  7526 raft_consensus.cc:2272] T 1e00eadc811946b48dfdf947398a84c1 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.538290  7526 tablet_replica.cc:333] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.539021  7526 raft_consensus.cc:2243] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.539594  7526 raft_consensus.cc:2272] T 2226d75e22544facaa146d281f852f27 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.542357  7526 tablet_replica.cc:333] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.543150  7526 raft_consensus.cc:2243] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.543709  7526 raft_consensus.cc:2272] T 626dcd42c0f74fccb9c2a42ee7d443ee P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.546252  7526 tablet_replica.cc:333] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.547012  7526 raft_consensus.cc:2243] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.548054  7526 raft_consensus.cc:2272] T b1f46bc9df3a44dd927d40f3778a5ec3 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.551286  7526 tablet_replica.cc:333] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.552513  7526 raft_consensus.cc:2243] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.553611  7526 raft_consensus.cc:2272] T 62f330683b8d4b0696fad129a8f2ce23 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.557035  7526 tablet_replica.cc:333] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.558066  7526 raft_consensus.cc:2243] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.558791  7526 raft_consensus.cc:2272] T 2ccac61a2bab40708cfe17c17ae1de65 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.562220  7526 tablet_replica.cc:333] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.563162  7526 raft_consensus.cc:2243] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.564381  7526 raft_consensus.cc:2272] T 14ead71306714d3a9784b47ddaa7588c P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:36.566339 17596 consensus_peers.cc:597] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:36.567692  7526 tablet_replica.cc:333] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.568593  7526 raft_consensus.cc:2243] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.569569  7526 raft_consensus.cc:2272] T deeb668bf4814b1bb7e01791d6f96914 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.572643  7526 tablet_replica.cc:333] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.573343  7526 raft_consensus.cc:2243] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.573838  7526 raft_consensus.cc:2272] T dcb6246353144b0a8b9648ac089a1da6 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.576375  7526 tablet_replica.cc:333] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.577085  7526 raft_consensus.cc:2243] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.577632  7526 raft_consensus.cc:2272] T a74eee34cc514b79b981a6ef6c45d7e2 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.580385  7526 tablet_replica.cc:333] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.581190  7526 raft_consensus.cc:2243] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.582206  7526 raft_consensus.cc:2272] T a7f14faf54714af3abeff5693ace0e8c P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.584872  7526 tablet_replica.cc:333] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.585572  7526 raft_consensus.cc:2243] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.586202  7526 raft_consensus.cc:2272] T a81d2393d5664464837df0b9d2a0d25b P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.589438  7526 tablet_replica.cc:333] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.590247  7526 raft_consensus.cc:2243] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.591210  7526 raft_consensus.cc:2272] T 2e4b5ba400ad420f89c16a97b71e3b70 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.593461  7526 tablet_replica.cc:333] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.594058  7526 raft_consensus.cc:2243] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.595100  7526 raft_consensus.cc:2272] T 2d047496a466464ba1777cf63c5c84b0 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.598274  7526 tablet_replica.cc:333] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.599092  7526 raft_consensus.cc:2243] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.599635  7526 raft_consensus.cc:2272] T e559727ff02540749beb944b741d471f P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.602314  7526 tablet_replica.cc:333] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.603013  7526 raft_consensus.cc:2243] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.603503  7526 raft_consensus.cc:2272] T 37b734b40edd440fad746ea61f858c1b P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.605912  7526 tablet_replica.cc:333] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.606556  7526 raft_consensus.cc:2243] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.607121  7526 raft_consensus.cc:2272] T 242531b3d6ad4ea4bdd979f393b59f29 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.609441  7526 tablet_replica.cc:333] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.610041  7526 raft_consensus.cc:2243] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.610518  7526 raft_consensus.cc:2272] T eef6e41b27ca4b229606bc1df6b023fb P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.612807  7526 tablet_replica.cc:333] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.613399  7526 raft_consensus.cc:2243] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.614380  7526 raft_consensus.cc:2272] T 050a17ead1c04486b472a25b0bb179ca P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.616607  7526 tablet_replica.cc:333] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.617183  7526 raft_consensus.cc:2243] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.618054  7526 raft_consensus.cc:2272] T 5d19239b38b1485ca53f239a71a11444 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.620492  7526 tablet_replica.cc:333] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.621081  7526 raft_consensus.cc:2243] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.621593  7526 raft_consensus.cc:2272] T 0f7e0518788149c19cf493a65bc598c2 P 4df2ca35be5341afa5a621233a4da94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.624058  7526 tablet_replica.cc:333] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.624677  7526 raft_consensus.cc:2243] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.625159  7526 raft_consensus.cc:2272] T 01c85566c0a845438507d71c6936d507 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.628058  7526 tablet_replica.cc:333] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.628855  7526 raft_consensus.cc:2243] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.629477  7526 raft_consensus.cc:2272] T 5a9a47a9cc56480fada4e8084a44682a P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.632731  7526 tablet_replica.cc:333] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.633543  7526 raft_consensus.cc:2243] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.634778  7526 raft_consensus.cc:2272] T 09bc57cc906f4179a8f6ff86c42b0247 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.637813  7526 tablet_replica.cc:333] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.638700  7526 raft_consensus.cc:2243] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.639339  7526 raft_consensus.cc:2272] T 36168ca382614c3f9061a116e5d12df5 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.642889  7526 tablet_replica.cc:333] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.643888  7526 raft_consensus.cc:2243] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.644678  7526 raft_consensus.cc:2272] T cac16fb87c954fd9b4cef5850ffc16e9 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.648000  7526 tablet_replica.cc:333] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.648837  7526 raft_consensus.cc:2243] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.650050  7526 raft_consensus.cc:2272] T bd407ea24c014a8690ac37978d4b0b50 P 4df2ca35be5341afa5a621233a4da94e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.652814  7526 tablet_replica.cc:333] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.653625  7526 raft_consensus.cc:2243] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.654247  7526 raft_consensus.cc:2272] T 1995e080f6c048fe935afd95f336b109 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.657014  7526 tablet_replica.cc:333] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.657593  7526 raft_consensus.cc:2243] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.658046  7526 raft_consensus.cc:2272] T b5fb377eea3d41e68cb4f46212e6e857 P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.660370  7526 tablet_replica.cc:333] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.660992  7526 raft_consensus.cc:2243] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.661489  7526 raft_consensus.cc:2272] T 8b8583086b304aa484d68b0a8175ceb4 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.664043  7526 tablet_replica.cc:333] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.664750  7526 raft_consensus.cc:2243] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.665938  7526 raft_consensus.cc:2272] T 6633559a9d2643a29622caab848e5442 P 4df2ca35be5341afa5a621233a4da94e [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:36.668473 17596 consensus_peers.cc:597] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:36.669411  7526 tablet_replica.cc:333] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e: stopping tablet replica
I20260709 13:07:36.670257  7526 raft_consensus.cc:2243] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.671365  7526 raft_consensus.cc:2272] T 800f4d4505654dc58f390fcc9f0d90db P 4df2ca35be5341afa5a621233a4da94e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:36.689399 17522 consensus_peers.cc:597] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.695365 17522 consensus_peers.cc:597] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.710387 17596 consensus_peers.cc:597] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.726203 17596 consensus_peers.cc:597] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.729894 17596 consensus_peers.cc:597] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.734810 17596 consensus_peers.cc:597] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.735327 17522 consensus_peers.cc:597] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:36.769785  7526 tablet_server.cc:196] TabletServer@127.7.89.130:0 shutdown complete.
W20260709 13:07:36.793664 17596 consensus_peers.cc:597] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:36599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.797546 17596 consensus_peers.cc:597] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:36599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.819479 17596 consensus_peers.cc:597] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:36599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.820039 17596 consensus_peers.cc:597] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:36599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.826224 17522 consensus_peers.cc:597] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:36599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.842604 17522 consensus_peers.cc:597] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:36599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.864711 17522 consensus_peers.cc:597] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:36599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:36.870082  7526 tablet_server.cc:179] TabletServer@127.7.89.131:0 shutting down...
W20260709 13:07:36.884968 17598 consensus_peers.cc:597] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:36.895103 17599 consensus_peers.cc:597] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada -> Peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Couldn't send request to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.951257 17522 consensus_peers.cc:597] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:36599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.952704 17596 consensus_peers.cc:597] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada -> Peer 4df2ca35be5341afa5a621233a4da94e (127.7.89.130:36599): Couldn't send request to peer 4df2ca35be5341afa5a621233a4da94e. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:36599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:36.957648 17599 consensus_peers.cc:597] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada -> Peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Couldn't send request to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:36.959652  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:36.960871  7526 tablet_replica.cc:333] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:36.962006  7526 raft_consensus.cc:2243] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.962679  7526 raft_consensus.cc:2272] T f55f8c5bd47d4a31a9562ab2b75ab56f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:36.965055 17599 consensus_peers.cc:597] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada -> Peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Couldn't send request to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:36.965974  7526 tablet_replica.cc:333] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:36.967075  7526 raft_consensus.cc:2243] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.968281  7526 raft_consensus.cc:2272] T eea055e03bf849bea8fdc851e10a8cbe P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.971726  7526 tablet_replica.cc:333] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:36.972842  7526 raft_consensus.cc:2243] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.973539  7526 raft_consensus.cc:2272] T 307acc498b0c4da7b7953e895b049c2a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.977787  7526 tablet_replica.cc:333] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:36.978884  7526 raft_consensus.cc:2243] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.979482  7526 raft_consensus.cc:2272] T c295096a121e4940b38125740cea67cb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.982004  7526 tablet_replica.cc:333] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:36.983039  7526 raft_consensus.cc:2243] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.983673  7526 raft_consensus.cc:2272] T def346af33c747faaf6c9cf0414510f0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.986076  7526 tablet_replica.cc:333] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:36.986964  7526 raft_consensus.cc:2243] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.987506  7526 raft_consensus.cc:2272] T cdcc480cc6534780aa5a81e4c97e8bef P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.989811  7526 tablet_replica.cc:333] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:36.990480  7526 raft_consensus.cc:2243] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.991099  7526 raft_consensus.cc:2272] T 0b795b9f53f64ad387b207e8da82ff6f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.993950  7526 tablet_replica.cc:333] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:36.994782  7526 raft_consensus.cc:2243] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:36.995321  7526 raft_consensus.cc:2272] T b85fb92208674e2ba20983a453290a8b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:36.998034  7526 tablet_replica.cc:333] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:36.998890  7526 raft_consensus.cc:2243] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:36.999948  7526 raft_consensus.cc:2272] T fa3738962c3242b5aa69f93b778bea53 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.002378  7526 tablet_replica.cc:333] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.003170  7526 raft_consensus.cc:2243] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.004194  7526 raft_consensus.cc:2272] T 032dc48c2ae44160841c456abd340e40 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.006712  7526 tablet_replica.cc:333] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.007495  7526 raft_consensus.cc:2243] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.008045  7526 raft_consensus.cc:2272] T dfc142508c684e6eb9716cb26e7047b8 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.010929  7526 tablet_replica.cc:333] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.011724  7526 raft_consensus.cc:2243] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.012949  7526 raft_consensus.cc:2272] T 6170cc45e2ea48598650c74f9c06b644 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.016052  7526 tablet_replica.cc:333] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.017082  7526 raft_consensus.cc:2243] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.017786  7526 raft_consensus.cc:2272] T 14ead71306714d3a9784b47ddaa7588c P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:37.018169 17599 consensus_peers.cc:597] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada -> Peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Couldn't send request to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:37.020961  7526 tablet_replica.cc:333] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.022049  7526 raft_consensus.cc:2243] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.022789  7526 raft_consensus.cc:2272] T deeb668bf4814b1bb7e01791d6f96914 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.026258  7526 tablet_replica.cc:333] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.027222  7526 raft_consensus.cc:2243] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.028754  7526 raft_consensus.cc:2272] T 1b10c75b663e4cbcb3237b59443f77ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.032056  7526 tablet_replica.cc:333] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.032814  7526 raft_consensus.cc:2243] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.033378  7526 raft_consensus.cc:2272] T 20df005b7408432da9ac4fcf110a20b9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.035807  7526 tablet_replica.cc:333] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.036607  7526 raft_consensus.cc:2243] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.037699  7526 raft_consensus.cc:2272] T 5b1eabbf0fc942fabea0c59e6997ad23 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.041520  7526 tablet_replica.cc:333] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.042433  7526 raft_consensus.cc:2243] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.043627  7526 raft_consensus.cc:2272] T e84a923caa9c4755bcb78a35143dfbd9 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.046859  7526 tablet_replica.cc:333] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.047827  7526 raft_consensus.cc:2243] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.048488  7526 raft_consensus.cc:2272] T b1f46bc9df3a44dd927d40f3778a5ec3 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.052279  7526 tablet_replica.cc:333] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
W20260709 13:07:37.052050 17599 consensus_peers.cc:597] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada -> Peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Couldn't send request to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:37.053224  7526 raft_consensus.cc:2243] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.054513  7526 raft_consensus.cc:2272] T 626dcd42c0f74fccb9c2a42ee7d443ee P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.057920  7526 tablet_replica.cc:333] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.059161  7526 raft_consensus.cc:2243] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.060500  7526 raft_consensus.cc:2272] T 445a4c9a8b0441f0ad2725785483a5d4 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.063951  7526 tablet_replica.cc:333] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.064910  7526 raft_consensus.cc:2243] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.065719  7526 raft_consensus.cc:2272] T dcb6246353144b0a8b9648ac089a1da6 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:37.066457 17599 consensus_peers.cc:597] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada -> Peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Couldn't send request to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:37.068599  7526 tablet_replica.cc:333] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.069424  7526 raft_consensus.cc:2243] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.070497  7526 raft_consensus.cc:2272] T a74eee34cc514b79b981a6ef6c45d7e2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.073087  7526 tablet_replica.cc:333] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.073772  7526 raft_consensus.cc:2243] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.074318  7526 raft_consensus.cc:2272] T 944f8419f8c94888bb53565c46d6f69b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.076524  7526 tablet_replica.cc:333] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.077337  7526 raft_consensus.cc:2243] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.078003  7526 raft_consensus.cc:2272] T 800f4d4505654dc58f390fcc9f0d90db P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.081228  7526 tablet_replica.cc:333] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.081952  7526 raft_consensus.cc:2243] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:07:37.082772 17599 consensus_peers.cc:597] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada -> Peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Couldn't send request to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:37.082993  7526 raft_consensus.cc:2272] T 37b734b40edd440fad746ea61f858c1b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.086215  7526 tablet_replica.cc:333] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.087129  7526 raft_consensus.cc:2243] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.087741  7526 raft_consensus.cc:2272] T 8259e46a7776417a80d1be42d4419a13 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.090822  7526 tablet_replica.cc:333] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.091709  7526 raft_consensus.cc:2243] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.092864  7526 raft_consensus.cc:2272] T eef6e41b27ca4b229606bc1df6b023fb P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.096040  7526 tablet_replica.cc:333] T a81d2393d5664464837df0b9d2a0d25b P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.096716  7526 raft_consensus.cc:2243] T a81d2393d5664464837df0b9d2a0d25b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.097229  7526 raft_consensus.cc:2272] T a81d2393d5664464837df0b9d2a0d25b P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.099860  7526 tablet_replica.cc:333] T 2e4b5ba400ad420f89c16a97b71e3b70 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.100648  7526 raft_consensus.cc:2243] T 2e4b5ba400ad420f89c16a97b71e3b70 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.101205  7526 raft_consensus.cc:2272] T 2e4b5ba400ad420f89c16a97b71e3b70 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.103878  7526 tablet_replica.cc:333] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.104559  7526 raft_consensus.cc:2243] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.105137  7526 raft_consensus.cc:2272] T 2d047496a466464ba1777cf63c5c84b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.107931  7526 tablet_replica.cc:333] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.108641  7526 raft_consensus.cc:2243] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.109546  7526 raft_consensus.cc:2272] T 05345b561dfb497c8f1e081b05c1bb5d P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.111788  7526 tablet_replica.cc:333] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.112358  7526 raft_consensus.cc:2243] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.113214  7526 raft_consensus.cc:2272] T e559727ff02540749beb944b741d471f P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.115558  7526 tablet_replica.cc:333] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.116166  7526 raft_consensus.cc:2243] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.116991  7526 raft_consensus.cc:2272] T 5a9a47a9cc56480fada4e8084a44682a P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.119297  7526 tablet_replica.cc:333] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.119830  7526 raft_consensus.cc:2243] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.120225  7526 raft_consensus.cc:2272] T 09bc57cc906f4179a8f6ff86c42b0247 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.122697  7526 tablet_replica.cc:333] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.123262  7526 raft_consensus.cc:2243] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.123736  7526 raft_consensus.cc:2272] T abe7eea0c47a4c6998274f4b6d14dfb7 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.126490  7526 tablet_replica.cc:333] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.127288  7526 raft_consensus.cc:2243] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.128324  7526 raft_consensus.cc:2272] T 0f7e0518788149c19cf493a65bc598c2 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.131186  7526 tablet_replica.cc:333] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.131963  7526 raft_consensus.cc:2243] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.133082  7526 raft_consensus.cc:2272] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.136478  7526 tablet_replica.cc:333] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.137327  7526 raft_consensus.cc:2243] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.137799  7526 raft_consensus.cc:2272] T 34919e11221f465cbcef1151a1c5d4b0 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.140123  7526 tablet_replica.cc:333] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.140712  7526 raft_consensus.cc:2243] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.141199  7526 raft_consensus.cc:2272] T bd407ea24c014a8690ac37978d4b0b50 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.144043  7526 tablet_replica.cc:333] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.144848  7526 raft_consensus.cc:2243] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.145471  7526 raft_consensus.cc:2272] T 050a17ead1c04486b472a25b0bb179ca P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.149137  7526 tablet_replica.cc:333] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.150012  7526 raft_consensus.cc:2243] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.150689  7526 raft_consensus.cc:2272] T 5d19239b38b1485ca53f239a71a11444 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:37.151979 17599 consensus_peers.cc:597] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada -> Peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Couldn't send request to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:37.153975  7526 tablet_replica.cc:333] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.154932  7526 raft_consensus.cc:2243] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.155567  7526 raft_consensus.cc:2272] T 61235f863ca44ed3b6c4715dfd68e223 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.159049  7526 tablet_replica.cc:333] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317: stopping tablet replica
I20260709 13:07:37.159791  7526 raft_consensus.cc:2243] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.160959  7526 raft_consensus.cc:2272] T a5414b9916b148bba0932c91f3770f57 P 3e3a92cfda7b4a9c8ddf43c0c9f67317 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:37.189563 17599 consensus_peers.cc:597] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada -> Peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Couldn't send request to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:37.293701  7526 tablet_server.cc:196] TabletServer@127.7.89.131:0 shutdown complete.
W20260709 13:07:37.335292 17599 consensus_peers.cc:597] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada -> Peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Couldn't send request to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.131:40851: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:37.359318 17598 consensus_peers.cc:597] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada -> Peer 337bbc857b9b4e329c817b81bfb0971c (127.7.89.129:34943): Couldn't send request to peer 337bbc857b9b4e329c817b81bfb0971c. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:34943: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:37.395645  7526 tablet_server.cc:179] TabletServer@127.7.89.132:0 shutting down...
W20260709 13:07:37.426224 17599 consensus_peers.cc:597] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada -> Peer 3e3a92cfda7b4a9c8ddf43c0c9f67317 (127.7.89.131:40851): Couldn't send request to peer 3e3a92cfda7b4a9c8ddf43c0c9f67317. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.131:40851: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:37.467080  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:37.468255  7526 tablet_replica.cc:333] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.469324  7526 raft_consensus.cc:2243] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.470537  7526 raft_consensus.cc:2272] T dfc142508c684e6eb9716cb26e7047b8 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.473506  7526 tablet_replica.cc:333] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.474189  7526 raft_consensus.cc:2243] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.474773  7526 raft_consensus.cc:2272] T eea055e03bf849bea8fdc851e10a8cbe P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.477118  7526 tablet_replica.cc:333] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.477783  7526 raft_consensus.cc:2243] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.478734  7526 raft_consensus.cc:2272] T c295096a121e4940b38125740cea67cb P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.481353  7526 tablet_replica.cc:333] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.482188  7526 raft_consensus.cc:2243] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.482807  7526 raft_consensus.cc:2272] T cdcc480cc6534780aa5a81e4c97e8bef P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.486038  7526 tablet_replica.cc:333] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.487010  7526 raft_consensus.cc:2243] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.488165  7526 raft_consensus.cc:2272] T dcb6246353144b0a8b9648ac089a1da6 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.491288  7526 tablet_replica.cc:333] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.492002  7526 raft_consensus.cc:2243] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.493158  7526 raft_consensus.cc:2272] T cac16fb87c954fd9b4cef5850ffc16e9 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.495675  7526 tablet_replica.cc:333] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.496538  7526 raft_consensus.cc:2243] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.497478  7526 raft_consensus.cc:2272] T 34919e11221f465cbcef1151a1c5d4b0 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.499945  7526 tablet_replica.cc:333] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.500636  7526 raft_consensus.cc:2243] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.501187  7526 raft_consensus.cc:2272] T 22993c01d4634a3a92c5a2903aa07ffe P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.538699  7526 tablet_replica.cc:333] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.542989  7526 raft_consensus.cc:2243] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.552913  7526 raft_consensus.cc:2272] T 0b795b9f53f64ad387b207e8da82ff6f P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.559455  7526 tablet_replica.cc:333] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.561350  7526 raft_consensus.cc:2243] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.562525  7526 raft_consensus.cc:2272] T 032dc48c2ae44160841c456abd340e40 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.567050  7526 tablet_replica.cc:333] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.568499  7526 raft_consensus.cc:2243] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.569437  7526 raft_consensus.cc:2272] T b85fb92208674e2ba20983a453290a8b P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.573271  7526 tablet_replica.cc:333] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.574249  7526 raft_consensus.cc:2243] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.576498  7526 raft_consensus.cc:2272] T 1e00eadc811946b48dfdf947398a84c1 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.579809  7526 tablet_replica.cc:333] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.581015  7526 raft_consensus.cc:2243] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.582841  7526 raft_consensus.cc:2272] T a81d2393d5664464837df0b9d2a0d25b P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.588127  7526 tablet_replica.cc:333] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.590504  7526 raft_consensus.cc:2243] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.591694  7526 raft_consensus.cc:2272] T 2e4b5ba400ad420f89c16a97b71e3b70 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.596237  7526 tablet_replica.cc:333] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.597885  7526 raft_consensus.cc:2243] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.599004  7526 raft_consensus.cc:2272] T e84a923caa9c4755bcb78a35143dfbd9 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.605077  7526 tablet_replica.cc:333] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.607802  7526 raft_consensus.cc:2243] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.609197  7526 raft_consensus.cc:2272] T 626dcd42c0f74fccb9c2a42ee7d443ee P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.615219  7526 tablet_replica.cc:333] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.616626  7526 raft_consensus.cc:2243] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.619534  7526 raft_consensus.cc:2272] T 62f330683b8d4b0696fad129a8f2ce23 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.624423  7526 tablet_replica.cc:333] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.625648  7526 raft_consensus.cc:2243] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.628403  7526 raft_consensus.cc:2272] T 0b2d2a20fb474604a1077ce9af500e42 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.634074  7526 tablet_replica.cc:333] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.635768  7526 raft_consensus.cc:2243] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.637023  7526 raft_consensus.cc:2272] T 5b1eabbf0fc942fabea0c59e6997ad23 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.641561  7526 tablet_replica.cc:333] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.642818  7526 raft_consensus.cc:2243] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.643918  7526 raft_consensus.cc:2272] T deeb668bf4814b1bb7e01791d6f96914 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.648697  7526 tablet_replica.cc:333] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.650199  7526 raft_consensus.cc:2243] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.652567  7526 raft_consensus.cc:2272] T 944f8419f8c94888bb53565c46d6f69b P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.657258  7526 tablet_replica.cc:333] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.658532  7526 raft_consensus.cc:2243] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.661563  7526 raft_consensus.cc:2272] T 2226d75e22544facaa146d281f852f27 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.666391  7526 tablet_replica.cc:333] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.667642  7526 raft_consensus.cc:2243] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.668535  7526 raft_consensus.cc:2272] T 445a4c9a8b0441f0ad2725785483a5d4 P a9a65d4c22be403682be9bc817650ada [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.673282  7526 tablet_replica.cc:333] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.674314  7526 raft_consensus.cc:2243] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.675428  7526 raft_consensus.cc:2272] T 5d19239b38b1485ca53f239a71a11444 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.681208  7526 tablet_replica.cc:333] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.682359  7526 raft_consensus.cc:2243] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.684887  7526 raft_consensus.cc:2272] T 61235f863ca44ed3b6c4715dfd68e223 P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.689599  7526 tablet_replica.cc:333] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.690658  7526 raft_consensus.cc:2243] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.691587  7526 raft_consensus.cc:2272] T a7f14faf54714af3abeff5693ace0e8c P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.695612  7526 tablet_replica.cc:333] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.696673  7526 raft_consensus.cc:2243] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.697590  7526 raft_consensus.cc:2272] T 05345b561dfb497c8f1e081b05c1bb5d P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.700909  7526 tablet_replica.cc:333] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.701650  7526 raft_consensus.cc:2243] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.702380  7526 raft_consensus.cc:2272] T e559727ff02540749beb944b741d471f P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.705346  7526 tablet_replica.cc:333] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.706130  7526 raft_consensus.cc:2243] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 6 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.707990  7526 raft_consensus.cc:2272] T 8259e46a7776417a80d1be42d4419a13 P a9a65d4c22be403682be9bc817650ada [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.711036  7526 tablet_replica.cc:333] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.711737  7526 raft_consensus.cc:2243] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.713171  7526 raft_consensus.cc:2272] T 242531b3d6ad4ea4bdd979f393b59f29 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.716112  7526 tablet_replica.cc:333] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.716666  7526 raft_consensus.cc:2243] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.717240  7526 raft_consensus.cc:2272] T eef6e41b27ca4b229606bc1df6b023fb P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.720203  7526 tablet_replica.cc:333] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.720758  7526 raft_consensus.cc:2243] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.722030  7526 raft_consensus.cc:2272] T 307acc498b0c4da7b7953e895b049c2a P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.725073  7526 tablet_replica.cc:333] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.725725  7526 raft_consensus.cc:2243] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.727344  7526 raft_consensus.cc:2272] T 8b8583086b304aa484d68b0a8175ceb4 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.730062  7526 tablet_replica.cc:333] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.731204  7526 raft_consensus.cc:2243] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.732239  7526 raft_consensus.cc:2272] T 6633559a9d2643a29622caab848e5442 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.735553  7526 tablet_replica.cc:333] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.736492  7526 raft_consensus.cc:2243] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.738278  7526 raft_consensus.cc:2272] T 2ccac61a2bab40708cfe17c17ae1de65 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.741369  7526 tablet_replica.cc:333] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.742035  7526 raft_consensus.cc:2243] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.743547  7526 raft_consensus.cc:2272] T 1995e080f6c048fe935afd95f336b109 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.746799  7526 tablet_replica.cc:333] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.747485  7526 raft_consensus.cc:2243] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.748800  7526 raft_consensus.cc:2272] T b5fb377eea3d41e68cb4f46212e6e857 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.751644  7526 tablet_replica.cc:333] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.752319  7526 raft_consensus.cc:2243] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.753034  7526 raft_consensus.cc:2272] T a5414b9916b148bba0932c91f3770f57 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.756034  7526 tablet_replica.cc:333] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.756664  7526 raft_consensus.cc:2243] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.757227  7526 raft_consensus.cc:2272] T 5a9a47a9cc56480fada4e8084a44682a P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.759755  7526 tablet_replica.cc:333] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.760375  7526 raft_consensus.cc:2243] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.761620  7526 raft_consensus.cc:2272] T 01c85566c0a845438507d71c6936d507 P a9a65d4c22be403682be9bc817650ada [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.764533  7526 tablet_replica.cc:333] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.765116  7526 raft_consensus.cc:2243] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.765735  7526 raft_consensus.cc:2272] T 09bc57cc906f4179a8f6ff86c42b0247 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.768509  7526 tablet_replica.cc:333] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.769093  7526 raft_consensus.cc:2243] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.770334  7526 raft_consensus.cc:2272] T 36168ca382614c3f9061a116e5d12df5 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.773063  7526 tablet_replica.cc:333] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.773679  7526 raft_consensus.cc:2243] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:37.774983  7526 raft_consensus.cc:2272] T abe7eea0c47a4c6998274f4b6d14dfb7 P a9a65d4c22be403682be9bc817650ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.777661  7526 tablet_replica.cc:333] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada: stopping tablet replica
I20260709 13:07:37.778164  7526 raft_consensus.cc:2243] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:37.778785  7526 raft_consensus.cc:2272] T 8f8bf7f6c34a4edfb28a4914ad4ce49e P a9a65d4c22be403682be9bc817650ada [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:37.912714  7526 tablet_server.cc:196] TabletServer@127.7.89.132:0 shutdown complete.
I20260709 13:07:38.008677  7526 master.cc:562] Master@127.7.89.190:45519 shutting down...
I20260709 13:07:38.053231  7526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:38.054054  7526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:38.054426  7526 tablet_replica.cc:333] T 00000000000000000000000000000000 P dead242a90f4462c89f826224a05a3de: stopping tablet replica
I20260709 13:07:38.081199  7526 master.cc:584] Master@127.7.89.190:45519 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (50095 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:07:38.197958  7526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.89.190:43061
I20260709 13:07:38.199971  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:38.207300 18926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:38.207537 18927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:38.207803 18929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:38.208457  7526 server_base.cc:1061] running on GCE node
I20260709 13:07:38.209904  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:38.210099  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:38.210297  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602458210262 us; error 0 us; skew 500 ppm
I20260709 13:07:38.210980  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:38.213827  7526 webserver.cc:533] Webserver started at http://127.7.89.190:37505/ using document root <none> and password file <none>
I20260709 13:07:38.214399  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:38.214602  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:38.214928  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:38.216444  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/master-0-root/instance:
uuid: "c997592ab6304092a0cba7c630ef9b2d"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-h0rz"
I20260709 13:07:38.221604  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:07:38.225603 18934 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.226699  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:07:38.227085  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/master-0-root
uuid: "c997592ab6304092a0cba7c630ef9b2d"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-h0rz"
I20260709 13:07:38.227551  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:38.265285  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:38.266835  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:38.313931  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.190:43061
I20260709 13:07:38.314039 18985 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.190:43061 every 8 connection(s)
I20260709 13:07:38.318599 18986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:38.333485 18986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d: Bootstrap starting.
I20260709 13:07:38.339880 18986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.345165 18986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d: No bootstrap required, opened a new log
I20260709 13:07:38.348011 18986 raft_consensus.cc:359] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c997592ab6304092a0cba7c630ef9b2d" member_type: VOTER }
I20260709 13:07:38.348562 18986 raft_consensus.cc:385] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.348773 18986 raft_consensus.cc:740] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c997592ab6304092a0cba7c630ef9b2d, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.349342 18986 consensus_queue.cc:260] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c997592ab6304092a0cba7c630ef9b2d" member_type: VOTER }
I20260709 13:07:38.349818 18986 raft_consensus.cc:399] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:38.350033 18986 raft_consensus.cc:493] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:38.350273 18986 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.356302 18986 raft_consensus.cc:515] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c997592ab6304092a0cba7c630ef9b2d" member_type: VOTER }
I20260709 13:07:38.356894 18986 leader_election.cc:304] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c997592ab6304092a0cba7c630ef9b2d; no voters: 
I20260709 13:07:38.358212 18986 leader_election.cc:290] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:38.358695 18989 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:38.360317 18989 raft_consensus.cc:697] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [term 1 LEADER]: Becoming Leader. State: Replica: c997592ab6304092a0cba7c630ef9b2d, State: Running, Role: LEADER
I20260709 13:07:38.361126 18989 consensus_queue.cc:237] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c997592ab6304092a0cba7c630ef9b2d" member_type: VOTER }
I20260709 13:07:38.361624 18986 sys_catalog.cc:565] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:38.368479 18990 sys_catalog.cc:455] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c997592ab6304092a0cba7c630ef9b2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c997592ab6304092a0cba7c630ef9b2d" member_type: VOTER } }
I20260709 13:07:38.369272 18990 sys_catalog.cc:458] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:38.370656 18991 sys_catalog.cc:455] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c997592ab6304092a0cba7c630ef9b2d. Latest consensus state: current_term: 1 leader_uuid: "c997592ab6304092a0cba7c630ef9b2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c997592ab6304092a0cba7c630ef9b2d" member_type: VOTER } }
I20260709 13:07:38.371331 18991 sys_catalog.cc:458] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:38.371872 18997 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:38.380746 18997 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:38.381911  7526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:38.392127 18997 catalog_manager.cc:1383] Generated new cluster ID: 77319000f57f42b8a4ede64b842125b2
I20260709 13:07:38.392504 18997 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:38.413218 18997 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:38.414942 18997 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:38.432883 18997 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d: Generated new TSK 0
I20260709 13:07:38.433674 18997 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:38.449131  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:38.456454 19007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:38.457993 19008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:38.459967 19010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:38.465384  7526 server_base.cc:1061] running on GCE node
I20260709 13:07:38.467329  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:38.467590  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:38.467718  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602458467707 us; error 0 us; skew 500 ppm
I20260709 13:07:38.468288  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:38.471176  7526 webserver.cc:533] Webserver started at http://127.7.89.129:42915/ using document root <none> and password file <none>
I20260709 13:07:38.471699  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:38.471877  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:38.472118  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:38.473623  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-0-root/instance:
uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-h0rz"
I20260709 13:07:38.479005  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 13:07:38.482918 19015 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.483749  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:07:38.484042  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-0-root
uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-h0rz"
I20260709 13:07:38.484315  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:38.508641  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:38.510046  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:38.511643  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:38.514178  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:38.514386  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.514680  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:38.514854  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.564816  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.129:33445
I20260709 13:07:38.565166 19077 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.129:33445 every 8 connection(s)
I20260709 13:07:38.570979  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:38.580277 19082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:38.580300 19083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:38.584592  7526 server_base.cc:1061] running on GCE node
W20260709 13:07:38.583350 19085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:38.586679  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:38.586933  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:38.587152  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602458587129 us; error 0 us; skew 500 ppm
I20260709 13:07:38.587953  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:38.591137 19078 heartbeater.cc:344] Connected to a master server at 127.7.89.190:43061
I20260709 13:07:38.591183  7526 webserver.cc:533] Webserver started at http://127.7.89.130:38693/ using document root <none> and password file <none>
I20260709 13:07:38.591616 19078 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:38.591943  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:38.592254  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:38.592559  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:38.592564 19078 heartbeater.cc:507] Master 127.7.89.190:43061 requested a full tablet report, sending...
I20260709 13:07:38.594007  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-1-root/instance:
uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-h0rz"
I20260709 13:07:38.595168 18951 ts_manager.cc:194] Registered new tserver with Master: f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445)
I20260709 13:07:38.597328 18951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57466
I20260709 13:07:38.599619  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:07:38.603860 19090 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.604761  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:07:38.605132  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-1-root
uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-h0rz"
I20260709 13:07:38.605472  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:38.624312  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:38.625878  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:38.627791  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:38.630999  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:38.631239  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.631582  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:38.631760  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.682708  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.130:33415
I20260709 13:07:38.682847 19152 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.130:33415 every 8 connection(s)
I20260709 13:07:38.688202  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:38.696887 19156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:38.698782 19157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:38.701548 19159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:38.701843  7526 server_base.cc:1061] running on GCE node
I20260709 13:07:38.703294  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:07:38.703454 19153 heartbeater.cc:344] Connected to a master server at 127.7.89.190:43061
W20260709 13:07:38.703626  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:38.704017  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602458703998 us; error 0 us; skew 500 ppm
I20260709 13:07:38.704082 19153 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:38.704710  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:38.705109 19153 heartbeater.cc:507] Master 127.7.89.190:43061 requested a full tablet report, sending...
I20260709 13:07:38.708113  7526 webserver.cc:533] Webserver started at http://127.7.89.131:43225/ using document root <none> and password file <none>
I20260709 13:07:38.708145 18951 ts_manager.cc:194] Registered new tserver with Master: 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:38.709074  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:38.709409  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:38.709751  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:38.710489 18951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57476
I20260709 13:07:38.711621  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-2-root/instance:
uuid: "07cd48e27e034bdda1af81e80009363c"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-h0rz"
I20260709 13:07:38.717146  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:07:38.721259 19164 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.722208  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:07:38.722577  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-2-root
uuid: "07cd48e27e034bdda1af81e80009363c"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-h0rz"
I20260709 13:07:38.722996  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:38.745468  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:38.747054  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:38.748847  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:38.751540  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:38.751806  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.752056  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:38.752231  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:38.800944  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.131:44617
I20260709 13:07:38.801070 19226 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.131:44617 every 8 connection(s)
I20260709 13:07:38.818101 19227 heartbeater.cc:344] Connected to a master server at 127.7.89.190:43061
I20260709 13:07:38.818751 19227 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:38.819713 19227 heartbeater.cc:507] Master 127.7.89.190:43061 requested a full tablet report, sending...
I20260709 13:07:38.822347 18951 ts_manager.cc:194] Registered new tserver with Master: 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:38.823150  7526 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018336131s
I20260709 13:07:38.824081 18951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57478
I20260709 13:07:38.854570 18951 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57484:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:38.858186 18951 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:38.949551 19043 tablet_service.cc:1511] Processing CreateTablet for tablet 7816d5b785b0476f9243429fc021d806 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:38.949821 19042 tablet_service.cc:1511] Processing CreateTablet for tablet 6d7f014f1dfc4d1680a708a717ee0eb5 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:38.951390 19043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7816d5b785b0476f9243429fc021d806. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:38.952145 19042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d7f014f1dfc4d1680a708a717ee0eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:38.956804 19041 tablet_service.cc:1511] Processing CreateTablet for tablet 280ee5e3334b4646b9baf8ea4c11b8e9 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:38.958796 19041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 280ee5e3334b4646b9baf8ea4c11b8e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:38.962469 19040 tablet_service.cc:1511] Processing CreateTablet for tablet 05278d681f5f466e9cfc401e8f7e191d (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:38.964201 19191 tablet_service.cc:1511] Processing CreateTablet for tablet 6d7f014f1dfc4d1680a708a717ee0eb5 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:38.966136 19191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d7f014f1dfc4d1680a708a717ee0eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:38.962158 19192 tablet_service.cc:1511] Processing CreateTablet for tablet 7816d5b785b0476f9243429fc021d806 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:38.968128 19192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7816d5b785b0476f9243429fc021d806. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:38.964319 19040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05278d681f5f466e9cfc401e8f7e191d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:38.972313 19039 tablet_service.cc:1511] Processing CreateTablet for tablet 897c954cea374c7bbfb017af4c503055 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:38.974169 19039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 897c954cea374c7bbfb017af4c503055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:38.978363 19038 tablet_service.cc:1511] Processing CreateTablet for tablet 75b9646c958c4e8c9e6133698a77d51b (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:38.981755 19190 tablet_service.cc:1511] Processing CreateTablet for tablet 280ee5e3334b4646b9baf8ea4c11b8e9 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:38.981854 19038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b9646c958c4e8c9e6133698a77d51b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:38.984774 19190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 280ee5e3334b4646b9baf8ea4c11b8e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:38.987138 19189 tablet_service.cc:1511] Processing CreateTablet for tablet 05278d681f5f466e9cfc401e8f7e191d (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:38.988940 19188 tablet_service.cc:1511] Processing CreateTablet for tablet 897c954cea374c7bbfb017af4c503055 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:38.993552 19187 tablet_service.cc:1511] Processing CreateTablet for tablet 75b9646c958c4e8c9e6133698a77d51b (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:38.988997 19189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05278d681f5f466e9cfc401e8f7e191d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.000862 19037 tablet_service.cc:1511] Processing CreateTablet for tablet 9b8743904619452f81b5552cadd9f491 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:39.002789 19037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b8743904619452f81b5552cadd9f491. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.004233 19188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 897c954cea374c7bbfb017af4c503055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.008215 19186 tablet_service.cc:1511] Processing CreateTablet for tablet 9b8743904619452f81b5552cadd9f491 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:39.010206 19186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b8743904619452f81b5552cadd9f491. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.023650 19187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b9646c958c4e8c9e6133698a77d51b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.030078 19036 tablet_service.cc:1511] Processing CreateTablet for tablet 4c504c2e2f0b49b4aa1b80600f93b37b (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:39.031963 19036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c504c2e2f0b49b4aa1b80600f93b37b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.034447 19185 tablet_service.cc:1511] Processing CreateTablet for tablet 4c504c2e2f0b49b4aa1b80600f93b37b (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:39.035089 19242 tablet_bootstrap.cc:492] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:39.036335 19185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c504c2e2f0b49b4aa1b80600f93b37b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.036600 19035 tablet_service.cc:1511] Processing CreateTablet for tablet 375340f82f454f609b1dc1f347042929 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:39.038134 19035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 375340f82f454f609b1dc1f347042929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.042902 19118 tablet_service.cc:1511] Processing CreateTablet for tablet 7816d5b785b0476f9243429fc021d806 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:39.044621 19118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7816d5b785b0476f9243429fc021d806. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.045035 19116 tablet_service.cc:1511] Processing CreateTablet for tablet 280ee5e3334b4646b9baf8ea4c11b8e9 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:39.046581 19116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 280ee5e3334b4646b9baf8ea4c11b8e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.048036 19117 tablet_service.cc:1511] Processing CreateTablet for tablet 6d7f014f1dfc4d1680a708a717ee0eb5 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:39.060635 19117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d7f014f1dfc4d1680a708a717ee0eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.048043 19115 tablet_service.cc:1511] Processing CreateTablet for tablet 05278d681f5f466e9cfc401e8f7e191d (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:39.064447 19115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05278d681f5f466e9cfc401e8f7e191d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.049499 19114 tablet_service.cc:1511] Processing CreateTablet for tablet 897c954cea374c7bbfb017af4c503055 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:39.068420 19114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 897c954cea374c7bbfb017af4c503055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.051110 19113 tablet_service.cc:1511] Processing CreateTablet for tablet 75b9646c958c4e8c9e6133698a77d51b (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:39.072570 19113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b9646c958c4e8c9e6133698a77d51b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.053355 19112 tablet_service.cc:1511] Processing CreateTablet for tablet 9b8743904619452f81b5552cadd9f491 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:39.075098 19184 tablet_service.cc:1511] Processing CreateTablet for tablet 375340f82f454f609b1dc1f347042929 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:39.075966 19112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b8743904619452f81b5552cadd9f491. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.076606 19184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 375340f82f454f609b1dc1f347042929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.055727 19111 tablet_service.cc:1511] Processing CreateTablet for tablet 4c504c2e2f0b49b4aa1b80600f93b37b (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:39.080549 19111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c504c2e2f0b49b4aa1b80600f93b37b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.088613 19242 tablet_bootstrap.cc:654] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.088835 19110 tablet_service.cc:1511] Processing CreateTablet for tablet 375340f82f454f609b1dc1f347042929 (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:39.096902 19110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 375340f82f454f609b1dc1f347042929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.112788 19109 tablet_service.cc:1511] Processing CreateTablet for tablet 5f04a57ced394d7db3587eb966026f3a (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:39.114593 19109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f04a57ced394d7db3587eb966026f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.123255 19041 tablet_service.cc:1511] Processing CreateTablet for tablet 5f04a57ced394d7db3587eb966026f3a (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:39.125227 19041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f04a57ced394d7db3587eb966026f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.125995 19190 tablet_service.cc:1511] Processing CreateTablet for tablet 5f04a57ced394d7db3587eb966026f3a (DEFAULT_TABLE table=test-workload [id=004622d8b1e3488eb843a783ef36c733]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:39.127646 19190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f04a57ced394d7db3587eb966026f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:39.144182 19243 tablet_bootstrap.cc:492] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c: Bootstrap starting.
I20260709 13:07:39.179723 19244 tablet_bootstrap.cc:492] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf: Bootstrap starting.
I20260709 13:07:39.207149 19242 tablet_bootstrap.cc:492] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8: No bootstrap required, opened a new log
I20260709 13:07:39.207494 19243 tablet_bootstrap.cc:654] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.207907 19242 ts_tablet_manager.cc:1403] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.173s	user 0.011s	sys 0.007s
I20260709 13:07:39.217684 19242 raft_consensus.cc:359] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } }
I20260709 13:07:39.218541 19242 raft_consensus.cc:385] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.218914 19242 raft_consensus.cc:740] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.219916 19242 consensus_queue.cc:260] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } }
I20260709 13:07:39.222288 19244 tablet_bootstrap.cc:654] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.236091 19244 tablet_bootstrap.cc:492] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf: No bootstrap required, opened a new log
I20260709 13:07:39.236843 19244 ts_tablet_manager.cc:1403] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent bootstrapping tablet: real 0.058s	user 0.008s	sys 0.010s
I20260709 13:07:39.242226 19244 raft_consensus.cc:359] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.243057 19244 raft_consensus.cc:385] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.243377 19244 raft_consensus.cc:740] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.251907 19242 ts_tablet_manager.cc:1434] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.044s	user 0.016s	sys 0.014s
I20260709 13:07:39.253680 19078 heartbeater.cc:499] Master 127.7.89.190:43061 was elected leader, sending a full tablet report...
I20260709 13:07:39.254036 19242 tablet_bootstrap.cc:492] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:39.257696 19244 consensus_queue.cc:260] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.265414 19242 tablet_bootstrap.cc:654] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.267685 19243 tablet_bootstrap.cc:492] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c: No bootstrap required, opened a new log
I20260709 13:07:39.268474 19243 ts_tablet_manager.cc:1403] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c: Time spent bootstrapping tablet: real 0.125s	user 0.013s	sys 0.007s
I20260709 13:07:39.272812 19243 raft_consensus.cc:359] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } }
I20260709 13:07:39.273715 19243 raft_consensus.cc:385] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.274286 19243 raft_consensus.cc:740] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.275283 19243 consensus_queue.cc:260] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } }
I20260709 13:07:39.281896 19244 ts_tablet_manager.cc:1434] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent starting tablet: real 0.045s	user 0.020s	sys 0.009s
I20260709 13:07:39.283888 19153 heartbeater.cc:499] Master 127.7.89.190:43061 was elected leader, sending a full tablet report...
I20260709 13:07:39.285900 19227 heartbeater.cc:499] Master 127.7.89.190:43061 was elected leader, sending a full tablet report...
I20260709 13:07:39.286535 19243 ts_tablet_manager.cc:1434] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.012s
I20260709 13:07:39.287868 19243 tablet_bootstrap.cc:492] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c: Bootstrap starting.
I20260709 13:07:39.289808 19244 tablet_bootstrap.cc:492] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf: Bootstrap starting.
I20260709 13:07:39.298588 19242 tablet_bootstrap.cc:492] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8: No bootstrap required, opened a new log
I20260709 13:07:39.299180 19244 tablet_bootstrap.cc:654] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.299180 19243 tablet_bootstrap.cc:654] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.299417 19242 ts_tablet_manager.cc:1403] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.046s	user 0.021s	sys 0.007s
W20260709 13:07:39.306016 19228 tablet.cc:2406] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:07:39.304818 19242 raft_consensus.cc:359] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.308494 19242 raft_consensus.cc:385] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.309051 19242 raft_consensus.cc:740] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.310228 19242 consensus_queue.cc:260] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.313436 19243 tablet_bootstrap.cc:492] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c: No bootstrap required, opened a new log
I20260709 13:07:39.314531 19243 ts_tablet_manager.cc:1403] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.010s
I20260709 13:07:39.315887 19242 ts_tablet_manager.cc:1434] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.001s
I20260709 13:07:39.317476 19242 tablet_bootstrap.cc:492] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:39.318369 19244 tablet_bootstrap.cc:492] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf: No bootstrap required, opened a new log
I20260709 13:07:39.319108 19244 ts_tablet_manager.cc:1403] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.014s
I20260709 13:07:39.319011 19243 raft_consensus.cc:359] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.320034 19243 raft_consensus.cc:385] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.320391 19243 raft_consensus.cc:740] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.321529 19243 consensus_queue.cc:260] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.324759 19243 ts_tablet_manager.cc:1434] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:07:39.325040 19244 raft_consensus.cc:359] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.325989 19244 raft_consensus.cc:385] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.326231 19243 tablet_bootstrap.cc:492] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c: Bootstrap starting.
I20260709 13:07:39.326354 19244 raft_consensus.cc:740] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.327108 19242 tablet_bootstrap.cc:654] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.327677 19244 consensus_queue.cc:260] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.330488 19244 ts_tablet_manager.cc:1434] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:07:39.331979 19244 tablet_bootstrap.cc:492] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf: Bootstrap starting.
I20260709 13:07:39.335589 19243 tablet_bootstrap.cc:654] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.340188 19244 tablet_bootstrap.cc:654] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.349685 19243 tablet_bootstrap.cc:492] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c: No bootstrap required, opened a new log
I20260709 13:07:39.350562 19243 ts_tablet_manager.cc:1403] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.008s
I20260709 13:07:39.355295 19244 tablet_bootstrap.cc:492] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf: No bootstrap required, opened a new log
I20260709 13:07:39.355989 19244 ts_tablet_manager.cc:1403] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.003s
I20260709 13:07:39.359804 19244 raft_consensus.cc:359] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.360692 19244 raft_consensus.cc:385] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.361011 19244 raft_consensus.cc:740] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.362133 19244 consensus_queue.cc:260] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.365197 19244 ts_tablet_manager.cc:1434] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:07:39.354542 19243 raft_consensus.cc:359] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.366190 19243 raft_consensus.cc:385] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.366420 19244 tablet_bootstrap.cc:492] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf: Bootstrap starting.
I20260709 13:07:39.366554 19243 raft_consensus.cc:740] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.367985 19243 consensus_queue.cc:260] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
W20260709 13:07:39.375151 18999 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:39.375402 19244 tablet_bootstrap.cc:654] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.377218 19243 ts_tablet_manager.cc:1434] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:07:39.378796 19243 tablet_bootstrap.cc:492] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c: Bootstrap starting.
I20260709 13:07:39.388496 19243 tablet_bootstrap.cc:654] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.388602 19244 tablet_bootstrap.cc:492] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf: No bootstrap required, opened a new log
I20260709 13:07:39.389389 19244 ts_tablet_manager.cc:1403] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.011s
I20260709 13:07:39.392997 19242 tablet_bootstrap.cc:492] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8: No bootstrap required, opened a new log
I20260709 13:07:39.393812 19242 ts_tablet_manager.cc:1403] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.077s	user 0.035s	sys 0.040s
I20260709 13:07:39.393268 19244 raft_consensus.cc:359] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.394522 19244 raft_consensus.cc:385] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.395035 19244 raft_consensus.cc:740] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.396411 19244 consensus_queue.cc:260] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.397881 19242 raft_consensus.cc:359] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.398958 19242 raft_consensus.cc:385] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.399371 19242 raft_consensus.cc:740] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.400530 19242 consensus_queue.cc:260] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.402220 19243 tablet_bootstrap.cc:492] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c: No bootstrap required, opened a new log
I20260709 13:07:39.402923 19243 ts_tablet_manager.cc:1403] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.007s
I20260709 13:07:39.403227 19244 ts_tablet_manager.cc:1434] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:07:39.403281 19242 ts_tablet_manager.cc:1434] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:07:39.404805 19244 tablet_bootstrap.cc:492] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf: Bootstrap starting.
I20260709 13:07:39.404795 19242 tablet_bootstrap.cc:492] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:39.406497 19243 raft_consensus.cc:359] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.407436 19243 raft_consensus.cc:385] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.407770 19243 raft_consensus.cc:740] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.408901 19243 consensus_queue.cc:260] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.412472 19249 raft_consensus.cc:493] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.413173 19249 raft_consensus.cc:515] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.414003 19243 ts_tablet_manager.cc:1434] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:07:39.415019 19242 tablet_bootstrap.cc:654] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.415323 19243 tablet_bootstrap.cc:492] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c: Bootstrap starting.
I20260709 13:07:39.425832 19249 leader_election.cc:290] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:39.433475 19244 tablet_bootstrap.cc:654] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.446332 19243 tablet_bootstrap.cc:654] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.449005 19053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c504c2e2f0b49b4aa1b80600f93b37b" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" is_pre_election: true
W20260709 13:07:39.451352 19094 leader_election.cc:343] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.453593 19244 tablet_bootstrap.cc:492] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf: No bootstrap required, opened a new log
I20260709 13:07:39.454651 19244 ts_tablet_manager.cc:1403] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.003s
I20260709 13:07:39.459882 19244 raft_consensus.cc:359] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.461200 19244 raft_consensus.cc:385] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.461625 19244 raft_consensus.cc:740] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.462905 19244 consensus_queue.cc:260] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.465148 19242 tablet_bootstrap.cc:492] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8: No bootstrap required, opened a new log
I20260709 13:07:39.465953 19242 ts_tablet_manager.cc:1403] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.061s	user 0.036s	sys 0.018s
I20260709 13:07:39.467267 19244 ts_tablet_manager.cc:1434] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 13:07:39.469086 19244 tablet_bootstrap.cc:492] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf: Bootstrap starting.
I20260709 13:07:39.470280 19242 raft_consensus.cc:359] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.471426 19242 raft_consensus.cc:385] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.471868 19242 raft_consensus.cc:740] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.473007 19242 consensus_queue.cc:260] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.480492 19242 ts_tablet_manager.cc:1434] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.004s
I20260709 13:07:39.480739 19244 tablet_bootstrap.cc:654] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.481973 19242 tablet_bootstrap.cc:492] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:39.489387 19243 tablet_bootstrap.cc:492] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c: No bootstrap required, opened a new log
I20260709 13:07:39.490139 19243 ts_tablet_manager.cc:1403] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c: Time spent bootstrapping tablet: real 0.075s	user 0.026s	sys 0.022s
I20260709 13:07:39.491127 19202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c504c2e2f0b49b4aa1b80600f93b37b" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c" is_pre_election: true
I20260709 13:07:39.492269 19202 raft_consensus.cc:2468] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 0.
I20260709 13:07:39.493999 19243 raft_consensus.cc:359] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.494953 19243 raft_consensus.cc:385] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.494788 19092 leader_election.cc:304] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:39.495560 19243 raft_consensus.cc:740] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.496284 19246 raft_consensus.cc:493] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.496567 19243 consensus_queue.cc:260] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.497005 19249 raft_consensus.cc:2804] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.497695 19249 raft_consensus.cc:493] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.498193 19249 raft_consensus.cc:3060] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.499397 19243 ts_tablet_manager.cc:1434] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260709 13:07:39.496865 19246 raft_consensus.cc:515] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.507103 19243 tablet_bootstrap.cc:492] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c: Bootstrap starting.
I20260709 13:07:39.508571 19242 tablet_bootstrap.cc:654] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.509771 19246 leader_election.cc:290] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:39.518764 19243 tablet_bootstrap.cc:654] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.525933 19244 tablet_bootstrap.cc:492] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf: No bootstrap required, opened a new log
I20260709 13:07:39.526566 19244 ts_tablet_manager.cc:1403] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent bootstrapping tablet: real 0.058s	user 0.028s	sys 0.027s
I20260709 13:07:39.527572 19251 raft_consensus.cc:493] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.528429 19251 raft_consensus.cc:515] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } }
I20260709 13:07:39.530143 19244 raft_consensus.cc:359] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.531064 19244 raft_consensus.cc:385] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.531383 19244 raft_consensus.cc:740] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.532359 19244 consensus_queue.cc:260] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.538549 19249 raft_consensus.cc:515] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.542860 19053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c504c2e2f0b49b4aa1b80600f93b37b" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
W20260709 13:07:39.544737 19094 leader_election.cc:343] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.545236 19271 raft_consensus.cc:493] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.545924 19271 raft_consensus.cc:515] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.554096 19251 leader_election.cc:290] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:39.556535 19053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "375340f82f454f609b1dc1f347042929" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" is_pre_election: true
W20260709 13:07:39.558661 19094 leader_election.cc:343] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.549485 19244 ts_tablet_manager.cc:1434] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.005s
I20260709 13:07:39.560554 19244 tablet_bootstrap.cc:492] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf: Bootstrap starting.
I20260709 13:07:39.563386 19202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c504c2e2f0b49b4aa1b80600f93b37b" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:39.564293 19202 raft_consensus.cc:3060] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.574647 19271 leader_election.cc:290] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:39.575263 19128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" is_pre_election: true
I20260709 13:07:39.576211 19128 raft_consensus.cc:2468] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4c532a3d5a243c285f2c23fdfd914f8 in term 0.
I20260709 13:07:39.578445 19017 leader_election.cc:304] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:39.581141 19271 raft_consensus.cc:493] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.581545 19201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "375340f82f454f609b1dc1f347042929" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c" is_pre_election: true
I20260709 13:07:39.583549 19201 raft_consensus.cc:2468] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 0.
I20260709 13:07:39.581984 19053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7816d5b785b0476f9243429fc021d806" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" is_pre_election: true
I20260709 13:07:39.585366 19053 raft_consensus.cc:2468] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 0.
I20260709 13:07:39.586226 19244 tablet_bootstrap.cc:654] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.587276 19168 leader_election.cc:304] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:39.588528 19251 raft_consensus.cc:2804] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.589010 19251 raft_consensus.cc:493] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.589452 19251 raft_consensus.cc:3060] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.591826 19246 raft_consensus.cc:2804] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.583483 19271 raft_consensus.cc:515] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.592280 19246 raft_consensus.cc:493] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.592734 19246 raft_consensus.cc:3060] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.595522 19053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" is_pre_election: true
W20260709 13:07:39.597491 19094 leader_election.cc:343] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.556631 19249 leader_election.cc:290] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:39.599211 19271 leader_election.cc:290] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:39.590173 19092 leader_election.cc:304] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:39.601935 19201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c" is_pre_election: true
I20260709 13:07:39.602771 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c" is_pre_election: true
W20260709 13:07:39.604305 19092 leader_election.cc:343] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.605926 19092 leader_election.cc:304] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:39.607666 19249 raft_consensus.cc:2749] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:39.604859 19271 raft_consensus.cc:2804] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.604905 19251 raft_consensus.cc:515] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } }
I20260709 13:07:39.605190 19243 tablet_bootstrap.cc:492] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c: No bootstrap required, opened a new log
I20260709 13:07:39.602222 19246 raft_consensus.cc:515] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.608675 19271 raft_consensus.cc:493] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.609236 19243 ts_tablet_manager.cc:1403] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c: Time spent bootstrapping tablet: real 0.109s	user 0.016s	sys 0.056s
I20260709 13:07:39.610438 19242 tablet_bootstrap.cc:492] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8: No bootstrap required, opened a new log
I20260709 13:07:39.610561 19246 leader_election.cc:290] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 election: Requested vote from peers 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:39.611534 19242 ts_tablet_manager.cc:1403] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.130s	user 0.014s	sys 0.008s
I20260709 13:07:39.612957 19251 leader_election.cc:290] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:39.614296 19243 raft_consensus.cc:359] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.614954 19202 raft_consensus.cc:2468] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 1.
I20260709 13:07:39.615248 19243 raft_consensus.cc:385] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.615072 19053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7816d5b785b0476f9243429fc021d806" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:39.615870 19243 raft_consensus.cc:740] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.616245 19053 raft_consensus.cc:3060] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.616484 19092 leader_election.cc:304] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:39.617054 19243 consensus_queue.cc:260] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.618095 19249 raft_consensus.cc:2804] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.619642 19249 raft_consensus.cc:697] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Becoming Leader. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:39.620232 19243 ts_tablet_manager.cc:1434] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 13:07:39.620641 19249 consensus_queue.cc:237] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.622797 19246 raft_consensus.cc:493] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.623391 19246 raft_consensus.cc:515] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.625973 19128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7816d5b785b0476f9243429fc021d806" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" is_pre_election: true
I20260709 13:07:39.630088 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
W20260709 13:07:39.632758 19017 leader_election.cc:343] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.610020 19271 raft_consensus.cc:3060] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.633497 19127 raft_consensus.cc:3060] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.628767 19246 leader_election.cc:290] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
W20260709 13:07:39.630084 19166 leader_election.cc:343] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.644820 19242 raft_consensus.cc:359] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.648759 19126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8743904619452f81b5552cadd9f491" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" is_pre_election: true
I20260709 13:07:39.632249 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:39.629390 19128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7816d5b785b0476f9243429fc021d806" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:39.631145 19243 tablet_bootstrap.cc:492] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c: Bootstrap starting.
I20260709 13:07:39.650701 19202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8743904619452f81b5552cadd9f491" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c" is_pre_election: true
W20260709 13:07:39.652552 19166 leader_election.cc:343] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.653080 19244 tablet_bootstrap.cc:492] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf: No bootstrap required, opened a new log
I20260709 13:07:39.653067 19202 raft_consensus.cc:2468] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4c532a3d5a243c285f2c23fdfd914f8 in term 0.
I20260709 13:07:39.653940 19244 ts_tablet_manager.cc:1403] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent bootstrapping tablet: real 0.094s	user 0.033s	sys 0.028s
I20260709 13:07:39.652196 19053 raft_consensus.cc:2468] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 1.
I20260709 13:07:39.655941 19168 leader_election.cc:304] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 14f4e3e7668c4c7998f4c46e7ab56edf
W20260709 13:07:39.651674 19017 leader_election.cc:343] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.658023 19244 raft_consensus.cc:359] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.658469 19271 raft_consensus.cc:515] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.659058 19244 raft_consensus.cc:385] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.659281 19251 raft_consensus.cc:2804] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.659499 19244 raft_consensus.cc:740] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.660579 19244 consensus_queue.cc:260] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.646319 19242 raft_consensus.cc:385] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.661676 19242 raft_consensus.cc:740] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.662551 19242 consensus_queue.cc:260] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.663645 19127 raft_consensus.cc:2468] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4c532a3d5a243c285f2c23fdfd914f8 in term 1.
I20260709 13:07:39.667706 19244 ts_tablet_manager.cc:1434] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.005s
I20260709 13:07:39.668324 19053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "375340f82f454f609b1dc1f347042929" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:39.669106 19244 tablet_bootstrap.cc:492] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf: Bootstrap starting.
I20260709 13:07:39.669875 19243 tablet_bootstrap.cc:654] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:39.670111 19094 leader_election.cc:343] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:39.659296 19017 leader_election.cc:343] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.675984 19242 ts_tablet_manager.cc:1434] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.064s	user 0.011s	sys 0.000s
I20260709 13:07:39.675383 18950 catalog_manager.cc:5697] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf reported cstate change: term changed from 0 to 1, leader changed from <none> to 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130). New cstate: current_term: 1 leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:39.677858 19242 tablet_bootstrap.cc:492] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:39.678977 19271 leader_election.cc:290] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:39.680490 19017 leader_election.cc:304] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 14f4e3e7668c4c7998f4c46e7ab56edf
I20260709 13:07:39.685321 19246 raft_consensus.cc:2804] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.685802 19246 raft_consensus.cc:493] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.686172 19246 raft_consensus.cc:3060] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.686582 19243 tablet_bootstrap.cc:492] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c: No bootstrap required, opened a new log
I20260709 13:07:39.686882 19244 tablet_bootstrap.cc:654] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.687077 19242 tablet_bootstrap.cc:654] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.687253 19243 ts_tablet_manager.cc:1403] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c: Time spent bootstrapping tablet: real 0.056s	user 0.020s	sys 0.004s
I20260709 13:07:39.690925 19243 raft_consensus.cc:359] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.691744 19243 raft_consensus.cc:385] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.692042 19243 raft_consensus.cc:740] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.692828 19243 consensus_queue.cc:260] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.696574 19282 raft_consensus.cc:493] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.697353 19282 raft_consensus.cc:515] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.698114 19243 ts_tablet_manager.cc:1434] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:07:39.704166 19242 tablet_bootstrap.cc:492] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8: No bootstrap required, opened a new log
I20260709 13:07:39.704926 19242 ts_tablet_manager.cc:1403] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.007s
I20260709 13:07:39.705087 19053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280ee5e3334b4646b9baf8ea4c11b8e9" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" is_pre_election: true
I20260709 13:07:39.707511 19053 raft_consensus.cc:2468] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 0.
I20260709 13:07:39.708899 19168 leader_election.cc:304] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:39.709597 19244 tablet_bootstrap.cc:492] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf: No bootstrap required, opened a new log
I20260709 13:07:39.709725 19243 tablet_bootstrap.cc:492] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c: Bootstrap starting.
I20260709 13:07:39.712342 19242 raft_consensus.cc:359] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.713326 19242 raft_consensus.cc:385] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.713663 19242 raft_consensus.cc:740] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.714123 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280ee5e3334b4646b9baf8ea4c11b8e9" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" is_pre_election: true
I20260709 13:07:39.714942 19127 raft_consensus.cc:2468] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 0.
I20260709 13:07:39.710486 19244 ts_tablet_manager.cc:1403] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.007s
I20260709 13:07:39.717453 19282 leader_election.cc:290] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:39.717641 19246 raft_consensus.cc:515] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.718566 19282 raft_consensus.cc:2804] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.719023 19244 raft_consensus.cc:359] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } }
I20260709 13:07:39.720089 19244 raft_consensus.cc:385] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.706453 19017 leader_election.cc:304] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 07cd48e27e034bdda1af81e80009363c
I20260709 13:07:39.720337 19243 tablet_bootstrap.cc:654] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.720448 19244 raft_consensus.cc:740] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.714587 19242 consensus_queue.cc:260] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.721791 19244 consensus_queue.cc:260] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } }
I20260709 13:07:39.719026 19282 raft_consensus.cc:493] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.725782 19282 raft_consensus.cc:3060] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.728820 19283 raft_consensus.cc:493] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.729982 19283 raft_consensus.cc:515] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.725150 19244 ts_tablet_manager.cc:1434] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:07:39.736545 19244 tablet_bootstrap.cc:492] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf: Bootstrap starting.
I20260709 13:07:39.738271 19285 raft_consensus.cc:2804] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.742733 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" is_pre_election: true
I20260709 13:07:39.748658 19282 raft_consensus.cc:515] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.754310 19244 tablet_bootstrap.cc:654] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.757510 19251 raft_consensus.cc:697] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Becoming Leader. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:39.759380 19251 consensus_queue.cc:237] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } }
I20260709 13:07:39.761765 19246 leader_election.cc:290] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 election: Requested vote from peers 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:39.762197 19053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280ee5e3334b4646b9baf8ea4c11b8e9" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:39.763667 19271 raft_consensus.cc:493] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.734432 19283 leader_election.cc:290] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:39.767992 19053 raft_consensus.cc:3060] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.765058 19282 leader_election.cc:290] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
W20260709 13:07:39.775969 19017 leader_election.cc:343] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.773828 19271 raft_consensus.cc:515] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.779260 19202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c" is_pre_election: true
I20260709 13:07:39.767273 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280ee5e3334b4646b9baf8ea4c11b8e9" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:39.780125 19202 raft_consensus.cc:2468] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4c532a3d5a243c285f2c23fdfd914f8 in term 0.
I20260709 13:07:39.780263 19127 raft_consensus.cc:3060] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.781925 19202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "375340f82f454f609b1dc1f347042929" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:39.782297 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05278d681f5f466e9cfc401e8f7e191d" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" is_pre_election: true
I20260709 13:07:39.782847 19202 raft_consensus.cc:3060] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.783250 19052 raft_consensus.cc:2468] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 0.
I20260709 13:07:39.780328 19271 leader_election.cc:290] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:39.784884 19094 leader_election.cc:304] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:39.785966 19271 raft_consensus.cc:2804] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.786477 19271 raft_consensus.cc:493] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.786967 19271 raft_consensus.cc:3060] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.791325 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05278d681f5f466e9cfc401e8f7e191d" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c" is_pre_election: true
W20260709 13:07:39.793098 19092 leader_election.cc:343] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.794762 19017 leader_election.cc:304] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 14f4e3e7668c4c7998f4c46e7ab56edf
I20260709 13:07:39.795042 19128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8743904619452f81b5552cadd9f491" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:39.798280 19128 raft_consensus.cc:3060] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.798990 19286 raft_consensus.cc:2804] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.799715 19286 raft_consensus.cc:493] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.796873 19243 tablet_bootstrap.cc:492] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c: No bootstrap required, opened a new log
I20260709 13:07:39.797022 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8743904619452f81b5552cadd9f491" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:39.801618 19243 ts_tablet_manager.cc:1403] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c: Time spent bootstrapping tablet: real 0.093s	user 0.026s	sys 0.034s
I20260709 13:07:39.802347 19200 raft_consensus.cc:3060] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.812486 19242 ts_tablet_manager.cc:1434] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.107s	user 0.036s	sys 0.017s
I20260709 13:07:39.800514 19285 raft_consensus.cc:697] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 LEADER]: Becoming Leader. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Running, Role: LEADER
I20260709 13:07:39.801702 19286 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.823954 19053 raft_consensus.cc:2468] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 1.
I20260709 13:07:39.826875 19168 leader_election.cc:304] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:39.828070 19282 raft_consensus.cc:2804] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.828660 19282 raft_consensus.cc:697] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Becoming Leader. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:39.829627 19282 consensus_queue.cc:237] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.831111 19202 raft_consensus.cc:2468] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 1.
I20260709 13:07:39.814540 19242 tablet_bootstrap.cc:492] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:39.834669 19243 raft_consensus.cc:359] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.836738 19127 raft_consensus.cc:2468] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 1.
I20260709 13:07:39.836880 19285 consensus_queue.cc:237] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.840597 19128 raft_consensus.cc:2468] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4c532a3d5a243c285f2c23fdfd914f8 in term 1.
I20260709 13:07:39.842394 19017 leader_election.cc:304] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:39.836431 19243 raft_consensus.cc:385] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.844225 19092 leader_election.cc:304] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:39.847072 19243 raft_consensus.cc:740] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.845495 19288 raft_consensus.cc:2804] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.848048 19249 raft_consensus.cc:2804] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.846415 19242 tablet_bootstrap.cc:654] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.848024 19243 consensus_queue.cc:260] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.849110 19288 raft_consensus.cc:697] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 LEADER]: Becoming Leader. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Running, Role: LEADER
I20260709 13:07:39.849143 19249 raft_consensus.cc:697] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Becoming Leader. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:39.856876 19249 consensus_queue.cc:237] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.851419 19268 raft_consensus.cc:493] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.853216 19200 raft_consensus.cc:2468] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4c532a3d5a243c285f2c23fdfd914f8 in term 1.
I20260709 13:07:39.864866 19268 raft_consensus.cc:515] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } }
I20260709 13:07:39.866099 19244 tablet_bootstrap.cc:492] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf: No bootstrap required, opened a new log
I20260709 13:07:39.866813 19244 ts_tablet_manager.cc:1403] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent bootstrapping tablet: real 0.131s	user 0.024s	sys 0.028s
I20260709 13:07:39.847837 19271 raft_consensus.cc:515] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.872509 19243 ts_tablet_manager.cc:1434] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c: Time spent starting tablet: real 0.070s	user 0.008s	sys 0.003s
I20260709 13:07:39.873214 19268 leader_election.cc:290] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:39.873911 19243 tablet_bootstrap.cc:492] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c: Bootstrap starting.
I20260709 13:07:39.875193 19244 raft_consensus.cc:359] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.876047 19244 raft_consensus.cc:385] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.876320 19244 raft_consensus.cc:740] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.876708 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05278d681f5f466e9cfc401e8f7e191d" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:39.877578 19052 raft_consensus.cc:3060] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.877223 19244 consensus_queue.cc:260] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.876693 19053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7816d5b785b0476f9243429fc021d806" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" is_pre_election: true
I20260709 13:07:39.881778 19053 raft_consensus.cc:2393] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 14f4e3e7668c4c7998f4c46e7ab56edf in current term 1: Already voted for candidate 07cd48e27e034bdda1af81e80009363c in this term.
I20260709 13:07:39.883265 19288 consensus_queue.cc:237] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.884313 18948 catalog_manager.cc:5697] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c reported cstate change: term changed from 0 to 1, leader changed from <none> to 07cd48e27e034bdda1af81e80009363c (127.7.89.131). New cstate: current_term: 1 leader_uuid: "07cd48e27e034bdda1af81e80009363c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:39.886242 19286 raft_consensus.cc:515] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.888005 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7816d5b785b0476f9243429fc021d806" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c" is_pre_election: true
I20260709 13:07:39.893208 19128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:39.894824 19202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:39.895787 19202 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.898738 19243 tablet_bootstrap.cc:654] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.903314 19271 leader_election.cc:290] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:39.906956 19092 leader_election.cc:304] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8
W20260709 13:07:39.895087 19017 leader_election.cc:343] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.909555 19271 raft_consensus.cc:3060] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.914004 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05278d681f5f466e9cfc401e8f7e191d" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:39.915283 19200 raft_consensus.cc:3060] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.916666 19052 raft_consensus.cc:2468] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 1.
I20260709 13:07:39.917630 19202 raft_consensus.cc:2468] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4c532a3d5a243c285f2c23fdfd914f8 in term 1.
I20260709 13:07:39.918377 19094 leader_election.cc:304] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:39.919958 19244 ts_tablet_manager.cc:1434] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent starting tablet: real 0.053s	user 0.010s	sys 0.009s
I20260709 13:07:39.920037 19017 leader_election.cc:304] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 14f4e3e7668c4c7998f4c46e7ab56edf
I20260709 13:07:39.921209 19249 raft_consensus.cc:2804] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.921446 19244 tablet_bootstrap.cc:492] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf: Bootstrap starting.
I20260709 13:07:39.922079 19249 raft_consensus.cc:697] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Becoming Leader. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:39.923184 19249 consensus_queue.cc:237] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.925252 19288 raft_consensus.cc:2804] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.926077 19288 raft_consensus.cc:697] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 LEADER]: Becoming Leader. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Running, Role: LEADER
I20260709 13:07:39.927079 19288 consensus_queue.cc:237] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.911186 19286 leader_election.cc:290] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 election: Requested vote from peers 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:39.933334 19244 tablet_bootstrap.cc:654] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.936065 19243 tablet_bootstrap.cc:492] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c: No bootstrap required, opened a new log
I20260709 13:07:39.936720 19243 ts_tablet_manager.cc:1403] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c: Time spent bootstrapping tablet: real 0.063s	user 0.022s	sys 0.013s
I20260709 13:07:39.939862 19242 tablet_bootstrap.cc:492] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8: No bootstrap required, opened a new log
I20260709 13:07:39.942523 19271 raft_consensus.cc:2749] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:39.944448 19243 raft_consensus.cc:359] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.948642 19243 raft_consensus.cc:385] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.948977 19243 raft_consensus.cc:740] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.950657 19243 consensus_queue.cc:260] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.946453 19200 raft_consensus.cc:2468] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 1.
I20260709 13:07:39.951086 19242 ts_tablet_manager.cc:1403] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.137s	user 0.009s	sys 0.010s
I20260709 13:07:39.955996 19243 ts_tablet_manager.cc:1434] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 13:07:39.956594 18949 catalog_manager.cc:5697] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:39.957562 19243 tablet_bootstrap.cc:492] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c: Bootstrap starting.
I20260709 13:07:39.960040 19242 raft_consensus.cc:359] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.960940 19242 raft_consensus.cc:385] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.961180 19244 tablet_bootstrap.cc:492] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf: No bootstrap required, opened a new log
I20260709 13:07:39.961831 19244 ts_tablet_manager.cc:1403] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.015s
I20260709 13:07:39.961269 19242 raft_consensus.cc:740] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.965996 19242 consensus_queue.cc:260] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.973070 19243 tablet_bootstrap.cc:654] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.976894 19244 raft_consensus.cc:359] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.978144 19244 raft_consensus.cc:385] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.978570 19244 raft_consensus.cc:740] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.979518 19244 consensus_queue.cc:260] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.981000 19242 ts_tablet_manager.cc:1434] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.002s
I20260709 13:07:39.981431 19244 ts_tablet_manager.cc:1434] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.001s
I20260709 13:07:39.982384 19242 tablet_bootstrap.cc:492] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:39.985430 19243 tablet_bootstrap.cc:492] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c: No bootstrap required, opened a new log
I20260709 13:07:39.986254 19243 ts_tablet_manager.cc:1403] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.005s
I20260709 13:07:39.989223 18948 catalog_manager.cc:5697] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c reported cstate change: term changed from 0 to 1, leader changed from <none> to 07cd48e27e034bdda1af81e80009363c (127.7.89.131). New cstate: current_term: 1 leader_uuid: "07cd48e27e034bdda1af81e80009363c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:39.990185 19243 raft_consensus.cc:359] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.991107 19243 raft_consensus.cc:385] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.991467 19243 raft_consensus.cc:740] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.992403 19243 consensus_queue.cc:260] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:39.995384 19243 ts_tablet_manager.cc:1434] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:07:39.997081 19242 tablet_bootstrap.cc:654] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.003427 19242 tablet_bootstrap.cc:492] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8: No bootstrap required, opened a new log
I20260709 13:07:40.003970 19242 ts_tablet_manager.cc:1403] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.001s
I20260709 13:07:40.005532 18950 catalog_manager.cc:5697] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf reported cstate change: term changed from 0 to 1, leader changed from <none> to 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130). New cstate: current_term: 1 leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.006642 19242 raft_consensus.cc:359] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.007503 19242 raft_consensus.cc:385] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.007807 19242 raft_consensus.cc:740] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.008539 19242 consensus_queue.cc:260] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.008303 18950 catalog_manager.cc:5697] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf reported cstate change: term changed from 0 to 1, leader changed from <none> to 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130). New cstate: current_term: 1 leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.010960 19242 ts_tablet_manager.cc:1434] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:07:40.012092 19242 tablet_bootstrap.cc:492] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:40.018448 19242 tablet_bootstrap.cc:654] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.021836 18949 catalog_manager.cc:5697] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.023635 19242 tablet_bootstrap.cc:492] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8: No bootstrap required, opened a new log
I20260709 13:07:40.024353 19242 ts_tablet_manager.cc:1403] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:07:40.023978 18949 catalog_manager.cc:5697] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.028065 19242 raft_consensus.cc:359] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.028941 19242 raft_consensus.cc:385] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.029240 19242 raft_consensus.cc:740] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.029994 19242 consensus_queue.cc:260] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.031972 19242 ts_tablet_manager.cc:1434] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:07:40.033078 19242 tablet_bootstrap.cc:492] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:40.038300 19242 tablet_bootstrap.cc:654] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.043350 19242 tablet_bootstrap.cc:492] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8: No bootstrap required, opened a new log
I20260709 13:07:40.043843 19242 ts_tablet_manager.cc:1403] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:07:40.046200 19242 raft_consensus.cc:359] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.047006 19242 raft_consensus.cc:385] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.047246 19242 raft_consensus.cc:740] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.047856 19242 consensus_queue.cc:260] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.049424 19242 ts_tablet_manager.cc:1434] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:40.056311 19251 raft_consensus.cc:493] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.057071 19251 raft_consensus.cc:515] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.059520 19251 leader_election.cc:290] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:40.060643 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" is_pre_election: true
I20260709 13:07:40.060829 19128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" is_pre_election: true
I20260709 13:07:40.061825 19128 raft_consensus.cc:2393] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07cd48e27e034bdda1af81e80009363c in current term 1: Already voted for candidate f4c532a3d5a243c285f2c23fdfd914f8 in this term.
I20260709 13:07:40.062922 19166 leader_election.cc:304] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c; no voters: 14f4e3e7668c4c7998f4c46e7ab56edf, f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:40.063763 19251 raft_consensus.cc:3060] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.069141 19282 raft_consensus.cc:493] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.069933 19282 raft_consensus.cc:515] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.071336 19251 raft_consensus.cc:2749] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.072603 19282 leader_election.cc:290] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:40.073647 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f04a57ced394d7db3587eb966026f3a" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" is_pre_election: true
I20260709 13:07:40.074035 19128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f04a57ced394d7db3587eb966026f3a" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" is_pre_election: true
I20260709 13:07:40.074563 19052 raft_consensus.cc:2468] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 0.
I20260709 13:07:40.074824 19128 raft_consensus.cc:2468] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 0.
I20260709 13:07:40.075158 19288 raft_consensus.cc:493] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.075704 19288 raft_consensus.cc:515] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.076300 19168 leader_election.cc:304] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:40.077349 19282 raft_consensus.cc:2804] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:40.077732 19282 raft_consensus.cc:493] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.078078 19288 leader_election.cc:290] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:40.078078 19282 raft_consensus.cc:3060] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.079113 19128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f04a57ced394d7db3587eb966026f3a" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" is_pre_election: true
I20260709 13:07:40.079869 19128 raft_consensus.cc:2468] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4c532a3d5a243c285f2c23fdfd914f8 in term 0.
I20260709 13:07:40.079946 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f04a57ced394d7db3587eb966026f3a" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c" is_pre_election: true
I20260709 13:07:40.081100 19017 leader_election.cc:304] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:40.081991 19288 raft_consensus.cc:2804] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:40.082346 19288 raft_consensus.cc:493] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.082697 19288 raft_consensus.cc:3060] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.086210 19282 raft_consensus.cc:515] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.087672 19200 raft_consensus.cc:2393] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f4c532a3d5a243c285f2c23fdfd914f8 in current term 1: Already voted for candidate 07cd48e27e034bdda1af81e80009363c in this term.
I20260709 13:07:40.090229 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f04a57ced394d7db3587eb966026f3a" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:40.090297 19288 raft_consensus.cc:515] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.090477 19128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f04a57ced394d7db3587eb966026f3a" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:40.090727 19282 leader_election.cc:290] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:40.091740 19128 raft_consensus.cc:3060] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.091966 19052 raft_consensus.cc:2393] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 07cd48e27e034bdda1af81e80009363c in current term 1: Already voted for candidate f4c532a3d5a243c285f2c23fdfd914f8 in this term.
I20260709 13:07:40.095892 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f04a57ced394d7db3587eb966026f3a" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:40.095129 19288 leader_election.cc:290] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 election: Requested vote from peers 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:40.097545 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f04a57ced394d7db3587eb966026f3a" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:40.098475 19200 raft_consensus.cc:2393] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f4c532a3d5a243c285f2c23fdfd914f8 in current term 1: Already voted for candidate 07cd48e27e034bdda1af81e80009363c in this term.
I20260709 13:07:40.100364 19017 leader_election.cc:304] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f4c532a3d5a243c285f2c23fdfd914f8; no voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf
I20260709 13:07:40.100926 19128 raft_consensus.cc:2468] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 1.
I20260709 13:07:40.101433 19288 raft_consensus.cc:2749] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.102226 19166 leader_election.cc:304] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:40.103091 19282 raft_consensus.cc:2804] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:40.103562 19282 raft_consensus.cc:697] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Becoming Leader. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:40.104477 19282 consensus_queue.cc:237] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.112972 18950 catalog_manager.cc:5697] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c reported cstate change: term changed from 0 to 1, leader changed from <none> to 07cd48e27e034bdda1af81e80009363c (127.7.89.131). New cstate: current_term: 1 leader_uuid: "07cd48e27e034bdda1af81e80009363c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:40.135551 19249 consensus_queue.cc:1048] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:40.148614 19052 raft_consensus.cc:3060] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.160703 19271 consensus_queue.cc:1048] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.215515 19282 consensus_queue.cc:1048] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:40.220923 19282 raft_consensus.cc:493] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.221570 19282 raft_consensus.cc:515] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.226151 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" is_pre_election: true
I20260709 13:07:40.226565 19282 leader_election.cc:290] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:40.228641 19052 raft_consensus.cc:2468] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 0.
I20260709 13:07:40.228462 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" is_pre_election: true
I20260709 13:07:40.232277 19168 leader_election.cc:304] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:40.232383 19282 consensus_queue.cc:1048] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.233937 19277 raft_consensus.cc:2804] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:40.234012 19127 raft_consensus.cc:2468] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 0.
I20260709 13:07:40.234414 19277 raft_consensus.cc:493] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.234899 19277 raft_consensus.cc:3060] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.246439 19277 raft_consensus.cc:515] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.249787 19277 leader_election.cc:290] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:40.249998 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:40.250861 19127 raft_consensus.cc:3060] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.250835 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:40.252477 19052 raft_consensus.cc:3060] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.259667 19127 raft_consensus.cc:2468] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 1.
I20260709 13:07:40.261302 19052 raft_consensus.cc:2468] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 1.
I20260709 13:07:40.261297 19166 leader_election.cc:304] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: 
I20260709 13:07:40.262737 19277 raft_consensus.cc:2804] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:40.263391 19277 raft_consensus.cc:697] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Becoming Leader. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:40.264240 19277 consensus_queue.cc:237] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.272686 18950 catalog_manager.cc:5697] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c reported cstate change: term changed from 0 to 1, leader changed from <none> to 07cd48e27e034bdda1af81e80009363c (127.7.89.131). New cstate: current_term: 1 leader_uuid: "07cd48e27e034bdda1af81e80009363c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:40.283294 19288 consensus_queue.cc:1048] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.291060 19271 consensus_queue.cc:1048] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.301501 19285 consensus_queue.cc:1048] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.318256 19053 raft_consensus.cc:3060] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.319176 19282 consensus_queue.cc:1048] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.330400 19249 raft_consensus.cc:493] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.331069 19249 raft_consensus.cc:515] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.335424 19249 leader_election.cc:290] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:40.335533 19202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c" is_pre_election: true
I20260709 13:07:40.336578 19202 raft_consensus.cc:2393] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 14f4e3e7668c4c7998f4c46e7ab56edf in current term 1: Already voted for candidate f4c532a3d5a243c285f2c23fdfd914f8 in this term.
I20260709 13:07:40.340050 19051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" is_pre_election: true
I20260709 13:07:40.341351 19249 consensus_queue.cc:1048] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.341709 19282 consensus_queue.cc:1048] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.341982 19094 leader_election.cc:304] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:40.343096 19271 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.351682 19286 raft_consensus.cc:493] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.352474 19286 raft_consensus.cc:515] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:40.355756 19126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "375340f82f454f609b1dc1f347042929" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" is_pre_election: true
I20260709 13:07:40.358129 19285 consensus_queue.cc:1048] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.359191 19249 consensus_queue.cc:1048] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:40.361714 19271 raft_consensus.cc:2749] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.365648 19202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "375340f82f454f609b1dc1f347042929" candidate_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c" is_pre_election: true
I20260709 13:07:40.358494 19286 leader_election.cc:290] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:40.377894 19017 leader_election.cc:304] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f4c532a3d5a243c285f2c23fdfd914f8; no voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf
I20260709 13:07:40.378928 19286 raft_consensus.cc:2749] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:40.388835 19249 consensus_queue.cc:1048] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.394338  7526 ts_manager.cc:295] Set tserver state for f4c532a3d5a243c285f2c23fdfd914f8 to MAINTENANCE_MODE
I20260709 13:07:40.395443  7526 tablet_server.cc:179] TabletServer@127.7.89.129:0 shutting down...
I20260709 13:07:40.414045 19286 consensus_queue.cc:1048] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.433174 19286 consensus_queue.cc:1048] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.444521 19286 consensus_queue.cc:1048] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.506597  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:40.507845  7526 tablet_replica.cc:333] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8: stopping tablet replica
I20260709 13:07:40.508699  7526 raft_consensus.cc:2243] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.509267  7526 raft_consensus.cc:2272] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.512180  7526 tablet_replica.cc:333] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8: stopping tablet replica
I20260709 13:07:40.512884  7526 raft_consensus.cc:2243] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.513433  7526 raft_consensus.cc:2272] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.516350  7526 tablet_replica.cc:333] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8: stopping tablet replica
I20260709 13:07:40.517011  7526 raft_consensus.cc:2243] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:40.517932  7526 raft_consensus.cc:2272] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.520607  7526 tablet_replica.cc:333] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8: stopping tablet replica
I20260709 13:07:40.521345  7526 raft_consensus.cc:2243] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.521806  7526 raft_consensus.cc:2272] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.524394  7526 tablet_replica.cc:333] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8: stopping tablet replica
I20260709 13:07:40.525112  7526 raft_consensus.cc:2243] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.525595  7526 raft_consensus.cc:2272] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.528442  7526 tablet_replica.cc:333] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8: stopping tablet replica
I20260709 13:07:40.529146  7526 raft_consensus.cc:2243] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.529654  7526 raft_consensus.cc:2272] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.532559  7526 tablet_replica.cc:333] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8: stopping tablet replica
I20260709 13:07:40.533246  7526 raft_consensus.cc:2243] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.533735  7526 raft_consensus.cc:2272] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.536146  7526 tablet_replica.cc:333] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8: stopping tablet replica
I20260709 13:07:40.536839  7526 raft_consensus.cc:2243] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:40.537839  7526 raft_consensus.cc:2272] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.540639  7526 tablet_replica.cc:333] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8: stopping tablet replica
I20260709 13:07:40.541289  7526 raft_consensus.cc:2243] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:40.542346  7526 raft_consensus.cc:2272] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.545029  7526 tablet_replica.cc:333] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8: stopping tablet replica
I20260709 13:07:40.545637  7526 raft_consensus.cc:2243] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.546151  7526 raft_consensus.cc:2272] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.581501  7526 tablet_server.cc:196] TabletServer@127.7.89.129:0 shutdown complete.
W20260709 13:07:40.659687 19168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111) [suppressed 231 similar messages]
W20260709 13:07:40.663621 19168 consensus_peers.cc:597] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:40.672603 19282 consensus_queue.cc:1048] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:40.688493 19168 consensus_peers.cc:597] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:40.711971 19094 consensus_peers.cc:597] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:40.762357 19094 consensus_peers.cc:597] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:40.769035 19277 consensus_queue.cc:1048] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:07:40.785605 19168 consensus_peers.cc:597] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:40.816124 19168 consensus_peers.cc:597] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:40.818765 19094 consensus_peers.cc:597] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:41.467995 18999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:41.799170 19249 raft_consensus.cc:493] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f4c532a3d5a243c285f2c23fdfd914f8)
I20260709 13:07:41.799897 19249 raft_consensus.cc:515] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:41.801885 19249 leader_election.cc:290] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:41.803186 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c" is_pre_election: true
W20260709 13:07:41.805788 19094 leader_election.cc:336] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111)
I20260709 13:07:41.806077 19276 raft_consensus.cc:493] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f4c532a3d5a243c285f2c23fdfd914f8)
I20260709 13:07:41.806272 19094 leader_election.cc:304] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:41.806740 19276 raft_consensus.cc:515] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:41.807145 19249 raft_consensus.cc:2749] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:41.808768 19276 leader_election.cc:290] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:41.810034 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" is_pre_election: true
I20260709 13:07:41.810823 19127 raft_consensus.cc:2468] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 1.
I20260709 13:07:41.812042 19166 leader_election.cc:304] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: 
W20260709 13:07:41.812707 19168 leader_election.cc:336] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111)
I20260709 13:07:41.812759 19276 raft_consensus.cc:2804] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:41.813208 19276 raft_consensus.cc:493] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Starting leader election (detected failure of leader f4c532a3d5a243c285f2c23fdfd914f8)
I20260709 13:07:41.813512 19276 raft_consensus.cc:3060] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:41.819904 19276 raft_consensus.cc:515] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:41.821846 19276 leader_election.cc:290] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:41.823177 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:41.823982 19127 raft_consensus.cc:3060] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:41.826325 19168 leader_election.cc:336] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111)
I20260709 13:07:41.831306 19127 raft_consensus.cc:2468] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 2.
I20260709 13:07:41.832592 19166 leader_election.cc:304] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:41.833371 19276 raft_consensus.cc:2804] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:41.834405 19276 raft_consensus.cc:697] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 2 LEADER]: Becoming Leader. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:41.835237 19276 consensus_queue.cc:237] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:41.843834 18947 catalog_manager.cc:5697] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c reported cstate change: term changed from 1 to 2, leader changed from f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129) to 07cd48e27e034bdda1af81e80009363c (127.7.89.131). New cstate: current_term: 2 leader_uuid: "07cd48e27e034bdda1af81e80009363c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.914418 19249 raft_consensus.cc:493] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f4c532a3d5a243c285f2c23fdfd914f8)
I20260709 13:07:41.915150 19249 raft_consensus.cc:515] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:41.917075 19249 leader_election.cc:290] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:41.918344 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8743904619452f81b5552cadd9f491" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c" is_pre_election: true
I20260709 13:07:41.918867 19276 raft_consensus.cc:493] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f4c532a3d5a243c285f2c23fdfd914f8)
I20260709 13:07:41.919449 19276 raft_consensus.cc:515] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:41.920285 19200 raft_consensus.cc:2468] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 1.
W20260709 13:07:41.921316 19094 leader_election.cc:336] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111)
I20260709 13:07:41.921681 19276 leader_election.cc:290] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:41.922955 19092 leader_election.cc:304] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:41.923081 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8743904619452f81b5552cadd9f491" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" is_pre_election: true
I20260709 13:07:41.923872 19249 raft_consensus.cc:2804] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:41.924242 19249 raft_consensus.cc:493] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Starting leader election (detected failure of leader f4c532a3d5a243c285f2c23fdfd914f8)
I20260709 13:07:41.924554 19249 raft_consensus.cc:3060] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:41.926483 19168 leader_election.cc:336] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111)
I20260709 13:07:41.931324 19249 raft_consensus.cc:515] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:41.932534 19127 raft_consensus.cc:2393] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07cd48e27e034bdda1af81e80009363c in current term 2: Already voted for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in this term.
I20260709 13:07:41.933542 19249 leader_election.cc:290] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:41.934573 19166 leader_election.cc:304] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c; no voters: 14f4e3e7668c4c7998f4c46e7ab56edf, f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:41.935019 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b8743904619452f81b5552cadd9f491" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:41.935683 19200 raft_consensus.cc:3060] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:41.937754 19094 leader_election.cc:336] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111)
I20260709 13:07:41.942392 19200 raft_consensus.cc:2468] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 2.
I20260709 13:07:41.943259 19276 raft_consensus.cc:2749] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:41.944149 19092 leader_election.cc:304] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:41.945130 19249 raft_consensus.cc:2804] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:41.946527 19249 raft_consensus.cc:697] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Becoming Leader. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:41.947419 19249 consensus_queue.cc:237] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:41.956486 18947 catalog_manager.cc:5697] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf reported cstate change: term changed from 1 to 2, leader changed from f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129) to 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130). New cstate: current_term: 2 leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:41.992467 19276 raft_consensus.cc:493] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f4c532a3d5a243c285f2c23fdfd914f8)
I20260709 13:07:41.993038 19276 raft_consensus.cc:515] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:41.995261 19276 leader_election.cc:290] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:41.996479 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" is_pre_election: true
W20260709 13:07:41.999401 19168 leader_election.cc:336] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111)
I20260709 13:07:41.999845 19168 leader_election.cc:304] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c; no voters: 14f4e3e7668c4c7998f4c46e7ab56edf, f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:41.999977 19249 raft_consensus.cc:493] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f4c532a3d5a243c285f2c23fdfd914f8)
I20260709 13:07:42.000686 19276 raft_consensus.cc:2749] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:42.000572 19249 raft_consensus.cc:515] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:42.002708 19249 leader_election.cc:290] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:42.003939 19202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c" is_pre_election: true
I20260709 13:07:42.004663 19202 raft_consensus.cc:2468] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 1.
I20260709 13:07:42.005805 19092 leader_election.cc:304] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: 
I20260709 13:07:42.006536 19249 raft_consensus.cc:2804] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:42.006976 19249 raft_consensus.cc:493] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Starting leader election (detected failure of leader f4c532a3d5a243c285f2c23fdfd914f8)
I20260709 13:07:42.007364 19249 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:42.008070 19094 leader_election.cc:336] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111)
I20260709 13:07:42.014061 19249 raft_consensus.cc:515] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:42.016357 19249 leader_election.cc:290] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:42.017731 19202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:42.018635 19202 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:42.021168 19094 leader_election.cc:336] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111)
I20260709 13:07:42.025811 19202 raft_consensus.cc:2468] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 2.
I20260709 13:07:42.027348 19092 leader_election.cc:304] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: f4c532a3d5a243c285f2c23fdfd914f8
I20260709 13:07:42.028319 19249 raft_consensus.cc:2804] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:42.028831 19249 raft_consensus.cc:697] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Becoming Leader. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:42.029755 19249 consensus_queue.cc:237] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:42.038528 18947 catalog_manager.cc:5697] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf reported cstate change: term changed from 1 to 2, leader changed from f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129) to 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130). New cstate: current_term: 2 leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:42.354419 19168 consensus_peers.cc:597] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:42.426056 19202 raft_consensus.cc:1275] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:42.427639 19249 consensus_queue.cc:1048] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:42.434461 19127 raft_consensus.cc:1275] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:42.436597 19276 consensus_queue.cc:1048] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:42.456508 19094 consensus_peers.cc:597] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:42.469161 18999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:07:42.523756 19094 consensus_peers.cc:597] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:42.551443 19202 raft_consensus.cc:1275] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:42.553253 19348 consensus_queue.cc:1048] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:42.976058 19094 consensus_peers.cc:597] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:42.980567 19094 consensus_peers.cc:597] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:43.081902 19168 consensus_peers.cc:597] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:43.133049 19168 consensus_peers.cc:597] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:43.137727 19094 consensus_peers.cc:597] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:43.154237 19168 consensus_peers.cc:597] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:43.419414 19168 consensus_peers.cc:597] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:43.470582 18999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:07:44.472043 18999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:07:44.639060 19168 consensus_peers.cc:597] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:44.766264 19094 consensus_peers.cc:597] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:44.990171 19094 consensus_peers.cc:597] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:45.155668 19094 consensus_peers.cc:597] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:45.473273 18999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:07:45.587098 19094 consensus_peers.cc:597] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:45.604605  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:45.612865 19353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:45.614089 19354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:45.615319 19356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:45.618424 19168 consensus_peers.cc:597] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:45.619442  7526 server_base.cc:1061] running on GCE node
I20260709 13:07:45.620988  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:45.621245  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:45.621488  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602465621466 us; error 0 us; skew 500 ppm
I20260709 13:07:45.622112  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:45.625646  7526 webserver.cc:533] Webserver started at http://127.7.89.129:42915/ using document root <none> and password file <none>
I20260709 13:07:45.626211  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:45.626428  7526 fs_manager.cc:365] Using existing metadata directory in first data directory
W20260709 13:07:45.626528 19168 consensus_peers.cc:597] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:45.630992  7526 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:07:45.636029 19361 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:45.637070  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.001s
I20260709 13:07:45.637452  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-0-root
uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
format_stamp: "Formatted at 2026-07-09 13:07:38 on dist-test-slave-h0rz"
I20260709 13:07:45.637763  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602061986899-7526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:45.652380  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:45.653810  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:45.655607  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:45.660053 19368 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:07:45.662983 19094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111) [suppressed 104 similar messages]
W20260709 13:07:45.698666 19168 consensus_peers.cc:597] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:45.729231  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:07:45.729864  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.071s	user 0.003s	sys 0.000s
I20260709 13:07:45.730291  7526 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
W20260709 13:07:45.734122 19094 consensus_peers.cc:597] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:45.737047 19368 tablet_bootstrap.cc:492] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:45.756742 19368 tablet_bootstrap.cc:492] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:45.757932 19368 tablet_bootstrap.cc:492] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap complete.
I20260709 13:07:45.758702 19368 ts_tablet_manager.cc:1403] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.004s
I20260709 13:07:45.761582 19368 raft_consensus.cc:359] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:45.762384 19368 raft_consensus.cc:740] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.763265 19368 consensus_queue.cc:260] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:45.766435 19368 ts_tablet_manager.cc:1434] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:45.767532 19368 tablet_bootstrap.cc:492] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:45.785758 19368 tablet_bootstrap.cc:492] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:45.787006 19368 tablet_bootstrap.cc:492] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap complete.
I20260709 13:07:45.787786 19368 ts_tablet_manager.cc:1403] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260709 13:07:45.790578 19368 raft_consensus.cc:359] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } }
I20260709 13:07:45.791379 19368 raft_consensus.cc:740] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.792151 19368 consensus_queue.cc:260] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } }
I20260709 13:07:45.794312 19368 ts_tablet_manager.cc:1434] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 13:07:45.795291 19368 tablet_bootstrap.cc:492] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:45.806071  7526 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:07:45.806553  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.076s	user 0.069s	sys 0.003s
I20260709 13:07:45.811172 19368 tablet_bootstrap.cc:492] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:45.812316 19368 tablet_bootstrap.cc:492] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap complete.
I20260709 13:07:45.813169 19368 ts_tablet_manager.cc:1403] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:07:45.816358 19368 raft_consensus.cc:359] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:45.817495 19368 raft_consensus.cc:740] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.818821 19368 consensus_queue.cc:260] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:45.821771 19368 ts_tablet_manager.cc:1434] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 13:07:45.823097 19368 tablet_bootstrap.cc:492] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:45.840538 19368 tablet_bootstrap.cc:492] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:45.841588 19368 tablet_bootstrap.cc:492] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap complete.
I20260709 13:07:45.842381 19368 ts_tablet_manager.cc:1403] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 13:07:45.845041 19368 raft_consensus.cc:359] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:45.845883 19368 raft_consensus.cc:740] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.846829 19368 consensus_queue.cc:260] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:45.848587 19368 ts_tablet_manager.cc:1434] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:45.849400 19368 tablet_bootstrap.cc:492] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:45.862757 19368 tablet_bootstrap.cc:492] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:45.863828 19368 tablet_bootstrap.cc:492] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap complete.
I20260709 13:07:45.864468 19368 ts_tablet_manager.cc:1403] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.004s
I20260709 13:07:45.866539 19368 raft_consensus.cc:359] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:45.867255 19368 raft_consensus.cc:740] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.867995 19368 consensus_queue.cc:260] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:45.869728 19368 ts_tablet_manager.cc:1434] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:07:45.870559 19368 tablet_bootstrap.cc:492] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:45.875660  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.129:33445
I20260709 13:07:45.875762 19430 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.129:33445 every 8 connection(s)
I20260709 13:07:45.881088  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:45.903831 19368 tablet_bootstrap.cc:492] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:45.905047 19368 tablet_bootstrap.cc:492] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap complete.
I20260709 13:07:45.906004 19368 ts_tablet_manager.cc:1403] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.036s	user 0.020s	sys 0.008s
I20260709 13:07:45.909196 19368 raft_consensus.cc:359] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:45.911404 19368 raft_consensus.cc:740] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.912328 19368 consensus_queue.cc:260] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:45.914803 19368 ts_tablet_manager.cc:1434] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:45.916179 19368 tablet_bootstrap.cc:492] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:45.921087 19431 heartbeater.cc:344] Connected to a master server at 127.7.89.190:43061
I20260709 13:07:45.921716 19431 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:45.922842 19431 heartbeater.cc:507] Master 127.7.89.190:43061 requested a full tablet report, sending...
I20260709 13:07:45.925772 19126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "375340f82f454f609b1dc1f347042929"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
mode: GRACEFUL
new_leader_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:35992
I20260709 13:07:45.926753 19126 raft_consensus.cc:606] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Received request to transfer leadership to TS 07cd48e27e034bdda1af81e80009363c
I20260709 13:07:45.933324 18949 ts_manager.cc:194] Re-registered known tserver with Master: f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445)
I20260709 13:07:45.941605 19368 tablet_bootstrap.cc:492] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:45.943316 19368 tablet_bootstrap.cc:492] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap complete.
I20260709 13:07:45.944365 19368 ts_tablet_manager.cc:1403] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.008s
I20260709 13:07:45.945137 18949 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42764
I20260709 13:07:45.948232 19368 raft_consensus.cc:359] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:45.948987 19368 raft_consensus.cc:740] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.949505 19431 heartbeater.cc:499] Master 127.7.89.190:43061 was elected leader, sending a full tablet report...
I20260709 13:07:45.949735 19368 consensus_queue.cc:260] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:45.952804 19368 ts_tablet_manager.cc:1434] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.001s
I20260709 13:07:45.953899 19368 tablet_bootstrap.cc:492] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:45.962659 19202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
mode: GRACEFUL
new_leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
 from {username='slave'} at 127.0.0.1:50076
I20260709 13:07:45.963577 19202 raft_consensus.cc:606] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Received request to transfer leadership to TS 14f4e3e7668c4c7998f4c46e7ab56edf
I20260709 13:07:45.967033 19126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "897c954cea374c7bbfb017af4c503055"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
mode: GRACEFUL
new_leader_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:35992
I20260709 13:07:45.967796 19126 raft_consensus.cc:606] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Received request to transfer leadership to TS 07cd48e27e034bdda1af81e80009363c
I20260709 13:07:45.969281  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:45.969951  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:45.970048 19368 tablet_bootstrap.cc:492] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:45.970820  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:45.971387 19368 tablet_bootstrap.cc:492] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap complete.
I20260709 13:07:45.972308 19368 ts_tablet_manager.cc:1403] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 13:07:45.975176 19368 raft_consensus.cc:359] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:45.976003 19368 raft_consensus.cc:740] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:45.976770 19368 consensus_queue.cc:260] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:45.979085 19368 ts_tablet_manager.cc:1434] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:07:45.980154 19368 tablet_bootstrap.cc:492] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:45.998742 19368 tablet_bootstrap.cc:492] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:45.999953 19368 tablet_bootstrap.cc:492] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap complete.
I20260709 13:07:46.000880 19368 ts_tablet_manager.cc:1403] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.004s
I20260709 13:07:46.004231 19368 raft_consensus.cc:359] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:46.005179 19368 raft_consensus.cc:740] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:46.006208 19368 consensus_queue.cc:260] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:46.009188 19368 ts_tablet_manager.cc:1434] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:46.010269 19368 tablet_bootstrap.cc:492] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap starting.
I20260709 13:07:46.011476 19352 raft_consensus.cc:993] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c: : Instructing follower 14f4e3e7668c4c7998f4c46e7ab56edf to start an election
I20260709 13:07:46.012204 19352 raft_consensus.cc:1081] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Signalling peer 14f4e3e7668c4c7998f4c46e7ab56edf to start an election
I20260709 13:07:46.014446 19126 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
 from {username='slave'} at 127.0.0.1:37092
I20260709 13:07:46.015259 19126 raft_consensus.cc:493] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:46.015702 19126 raft_consensus.cc:3060] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.022665 19126 raft_consensus.cc:515] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:46.026456 19202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:46.027227 19202 raft_consensus.cc:3055] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:46.027726 19202 raft_consensus.cc:740] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:46.028450 19368 tablet_bootstrap.cc:492] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:46.027076 19126 leader_election.cc:290] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:46.028766 19202 consensus_queue.cc:260] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:46.029521 19368 tablet_bootstrap.cc:492] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8: Bootstrap complete.
I20260709 13:07:46.030153 19202 raft_consensus.cc:3060] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.030333 19368 ts_tablet_manager.cc:1403] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:07:46.032517 19368 raft_consensus.cc:359] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:46.033228 19368 raft_consensus.cc:740] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c532a3d5a243c285f2c23fdfd914f8, State: Initialized, Role: FOLLOWER
I20260709 13:07:46.033924 19368 consensus_queue.cc:260] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:46.036123 19368 ts_tablet_manager.cc:1434] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 13:07:46.038981 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:46.039710 19405 raft_consensus.cc:3060] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.040236 19446 raft_consensus.cc:993] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf: : Instructing follower 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:46.040928 19446 raft_consensus.cc:1081] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Signalling peer 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:46.041057 19202 raft_consensus.cc:2468] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 2.
I20260709 13:07:46.042655 19092 leader_election.cc:304] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: 
I20260709 13:07:46.043068 19200 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "375340f82f454f609b1dc1f347042929"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:07:46.043759 19200 raft_consensus.cc:493] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:46.043890 19446 raft_consensus.cc:2804] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:46.044195 19200 raft_consensus.cc:3060] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.044468 19446 raft_consensus.cc:697] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Becoming Leader. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:46.045442 19446 consensus_queue.cc:237] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:46.051070 19405 raft_consensus.cc:2468] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 2.
I20260709 13:07:46.057116 19200 raft_consensus.cc:515] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:46.062191 19126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "375340f82f454f609b1dc1f347042929" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:46.063030 19126 raft_consensus.cc:3055] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:46.063411 19126 raft_consensus.cc:740] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:46.062534 18949 catalog_manager.cc:5697] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf reported cstate change: term changed from 1 to 2, leader changed from 07cd48e27e034bdda1af81e80009363c (127.7.89.131) to 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130). New cstate: current_term: 2 leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:46.064378 19126 consensus_queue.cc:260] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:46.065706 19126 raft_consensus.cc:3060] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.067373 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "375340f82f454f609b1dc1f347042929" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:46.068076 19405 raft_consensus.cc:3060] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.070796 19200 leader_election.cc:290] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:46.076193 19126 raft_consensus.cc:2468] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 2.
I20260709 13:07:46.077289 19405 raft_consensus.cc:2468] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 2.
I20260709 13:07:46.078306 19166 leader_election.cc:304] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: 
I20260709 13:07:46.079609 19352 raft_consensus.cc:2804] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:46.080211 19352 raft_consensus.cc:697] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 2 LEADER]: Becoming Leader. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:46.080642 19404 raft_consensus.cc:3060] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.081197 19352 consensus_queue.cc:237] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:46.093158 18949 catalog_manager.cc:5697] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c reported cstate change: term changed from 1 to 2, leader changed from 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130) to 07cd48e27e034bdda1af81e80009363c (127.7.89.131). New cstate: current_term: 2 leader_uuid: "07cd48e27e034bdda1af81e80009363c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:46.137343 19404 raft_consensus.cc:3060] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.254443 19404 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:46.436282 19404 raft_consensus.cc:1275] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:46.437788 19446 consensus_queue.cc:1048] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:46.447625 19200 raft_consensus.cc:1275] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:46.449429 19446 consensus_queue.cc:1048] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:46.474735 18999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:46.481087 19404 raft_consensus.cc:1275] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:46.482903 19349 consensus_queue.cc:1048] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:46.494185 19126 raft_consensus.cc:1275] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:46.496819 19462 consensus_queue.cc:1048] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:46.512111 19249 raft_consensus.cc:993] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf: : Instructing follower 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:46.512730 19446 raft_consensus.cc:1081] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Signalling peer 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:46.514544 19200 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "897c954cea374c7bbfb017af4c503055"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:07:46.515223 19200 raft_consensus.cc:493] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:46.515617 19200 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:46.523464 19200 raft_consensus.cc:515] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:46.525815 19200 leader_election.cc:290] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 3 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:46.526737 19404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:46.527091 19126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:46.527673 19404 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:46.527776 19126 raft_consensus.cc:3055] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:46.528474 19126 raft_consensus.cc:740] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:46.529613 19126 consensus_queue.cc:260] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:46.530818 19126 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:46.535612 19404 raft_consensus.cc:2468] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 3.
I20260709 13:07:46.537137 19168 leader_election.cc:304] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:46.537993 19462 raft_consensus.cc:2804] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:46.538658 19462 raft_consensus.cc:697] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 3 LEADER]: Becoming Leader. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:46.538535 19126 raft_consensus.cc:2468] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 3.
I20260709 13:07:46.540419 19462 consensus_queue.cc:237] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:46.548628 18949 catalog_manager.cc:5697] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c reported cstate change: term changed from 2 to 3, leader changed from 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130) to 07cd48e27e034bdda1af81e80009363c (127.7.89.131). New cstate: current_term: 3 leader_uuid: "07cd48e27e034bdda1af81e80009363c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:46.981516  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:46.990401 19126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05278d681f5f466e9cfc401e8f7e191d"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
mode: GRACEFUL
new_leader_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:35992
I20260709 13:07:46.991197 19126 raft_consensus.cc:606] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Received request to transfer leadership to TS 07cd48e27e034bdda1af81e80009363c
I20260709 13:07:46.994369 19200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f04a57ced394d7db3587eb966026f3a"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
mode: GRACEFUL
new_leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
 from {username='slave'} at 127.0.0.1:50076
I20260709 13:07:46.995007 19200 raft_consensus.cc:606] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Received request to transfer leadership to TS 14f4e3e7668c4c7998f4c46e7ab56edf
I20260709 13:07:46.998136 19200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "897c954cea374c7bbfb017af4c503055"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
mode: GRACEFUL
new_leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
 from {username='slave'} at 127.0.0.1:50076
I20260709 13:07:46.998775 19404 raft_consensus.cc:1275] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:46.998787 19200 raft_consensus.cc:606] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 3 LEADER]: Received request to transfer leadership to TS 14f4e3e7668c4c7998f4c46e7ab56edf
I20260709 13:07:47.000506  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:47.000687 19462 consensus_queue.cc:1048] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:47.001546  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:47.002437  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:47.014932 19126 raft_consensus.cc:1275] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:47.016980 19460 consensus_queue.cc:1048] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:47.026228 19352 raft_consensus.cc:993] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c: : Instructing follower 14f4e3e7668c4c7998f4c46e7ab56edf to start an election
I20260709 13:07:47.027447 19460 raft_consensus.cc:1081] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 3 LEADER]: Signalling peer 14f4e3e7668c4c7998f4c46e7ab56edf to start an election
I20260709 13:07:47.029410 19126 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "897c954cea374c7bbfb017af4c503055"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
 from {username='slave'} at 127.0.0.1:37092
I20260709 13:07:47.030170 19126 raft_consensus.cc:493] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:47.030673 19126 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:47.037343 19126 raft_consensus.cc:515] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:47.039562 19126 leader_election.cc:290] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 4 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:47.041034 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:47.042079 19200 raft_consensus.cc:3055] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:47.040977 19404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:47.042484 19200 raft_consensus.cc:740] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:47.043622 19200 consensus_queue.cc:260] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:47.042836 19404 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:47.045086 19200 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:47.055078 19200 raft_consensus.cc:2468] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 4.
I20260709 13:07:47.055078 19404 raft_consensus.cc:2468] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 4.
I20260709 13:07:47.057263 19092 leader_election.cc:304] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, 14f4e3e7668c4c7998f4c46e7ab56edf; no voters: 
I20260709 13:07:47.058333 19249 raft_consensus.cc:2804] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:47.058969 19249 raft_consensus.cc:697] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 LEADER]: Becoming Leader. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:47.059847 19249 consensus_queue.cc:237] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:47.067162 18947 catalog_manager.cc:5697] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf reported cstate change: term changed from 3 to 4, leader changed from 07cd48e27e034bdda1af81e80009363c (127.7.89.131) to 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130). New cstate: current_term: 4 leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:47.072955 19446 raft_consensus.cc:993] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf: : Instructing follower 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:47.073666 19446 raft_consensus.cc:1081] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Signalling peer 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:47.075848 19200 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05278d681f5f466e9cfc401e8f7e191d"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:07:47.076493 19200 raft_consensus.cc:493] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:47.076834 19200 raft_consensus.cc:3060] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:47.084650 19200 raft_consensus.cc:515] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:47.086459 19200 leader_election.cc:290] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:47.087714 19126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05278d681f5f466e9cfc401e8f7e191d" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:47.087751 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05278d681f5f466e9cfc401e8f7e191d" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:47.088451 19126 raft_consensus.cc:3055] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:47.088587 19405 raft_consensus.cc:3060] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:47.088783 19126 raft_consensus.cc:740] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:47.089754 19126 consensus_queue.cc:260] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:47.090941 19126 raft_consensus.cc:3060] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:47.095429 19405 raft_consensus.cc:2468] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 2.
I20260709 13:07:47.096992 19168 leader_election.cc:304] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:47.097582 19126 raft_consensus.cc:2468] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 2.
I20260709 13:07:47.097776 19460 raft_consensus.cc:2804] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:47.098534 19460 raft_consensus.cc:697] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [term 2 LEADER]: Becoming Leader. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:47.099538 19460 consensus_queue.cc:237] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:47.115496 19462 raft_consensus.cc:993] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c: : Instructing follower 14f4e3e7668c4c7998f4c46e7ab56edf to start an election
I20260709 13:07:47.116114 19462 raft_consensus.cc:1081] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Signalling peer 14f4e3e7668c4c7998f4c46e7ab56edf to start an election
I20260709 13:07:47.118019 19126 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5f04a57ced394d7db3587eb966026f3a"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
 from {username='slave'} at 127.0.0.1:37092
I20260709 13:07:47.118675 19126 raft_consensus.cc:493] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:47.119069 19126 raft_consensus.cc:3060] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:47.121236 18947 catalog_manager.cc:5697] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c reported cstate change: term changed from 1 to 2, leader changed from 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130) to 07cd48e27e034bdda1af81e80009363c (127.7.89.131). New cstate: current_term: 2 leader_uuid: "07cd48e27e034bdda1af81e80009363c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:47.127755 19126 raft_consensus.cc:515] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:47.130369 19126 leader_election.cc:290] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:47.131709 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f04a57ced394d7db3587eb966026f3a" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:47.132495 19200 raft_consensus.cc:3055] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:47.132463 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f04a57ced394d7db3587eb966026f3a" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:47.133203 19200 raft_consensus.cc:740] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:47.133419 19405 raft_consensus.cc:3060] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:47.134114 19200 consensus_queue.cc:260] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:47.137993 19200 raft_consensus.cc:3060] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:47.142297 19405 raft_consensus.cc:2468] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 2.
I20260709 13:07:47.143792 19094 leader_election.cc:304] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:47.144603 19249 raft_consensus.cc:2804] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:47.145023 19249 raft_consensus.cc:697] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Becoming Leader. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:47.145840 19200 raft_consensus.cc:2468] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 2.
I20260709 13:07:47.145769 19249 consensus_queue.cc:237] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:47.154017 18947 catalog_manager.cc:5697] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf reported cstate change: term changed from 1 to 2, leader changed from 07cd48e27e034bdda1af81e80009363c (127.7.89.131) to 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130). New cstate: current_term: 2 leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:47.475971 18999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:47.491508 19405 raft_consensus.cc:1275] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 4 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:47.493446 19249 consensus_queue.cc:1048] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:47.502640 19200 raft_consensus.cc:1275] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 4 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:47.505110 19446 consensus_queue.cc:1048] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:47.526427 19126 raft_consensus.cc:1275] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:47.528124 19475 consensus_queue.cc:1048] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:47.539754 19405 raft_consensus.cc:1275] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:47.541657 19405 raft_consensus.cc:1275] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:47.541597 19446 consensus_queue.cc:1048] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:47.543690 19462 consensus_queue.cc:1048] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:47.558521 19200 raft_consensus.cc:1275] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:47.560727 19465 consensus_queue.cc:1048] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:48.012794  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:48.023708 19200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "375340f82f454f609b1dc1f347042929"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
mode: GRACEFUL
new_leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
 from {username='slave'} at 127.0.0.1:50076
I20260709 13:07:48.024761 19200 raft_consensus.cc:606] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 2 LEADER]: Received request to transfer leadership to TS 14f4e3e7668c4c7998f4c46e7ab56edf
I20260709 13:07:48.029160 19126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
mode: GRACEFUL
new_leader_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:35992
I20260709 13:07:48.029855 19126 raft_consensus.cc:606] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Received request to transfer leadership to TS 07cd48e27e034bdda1af81e80009363c
I20260709 13:07:48.033106 19126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "897c954cea374c7bbfb017af4c503055"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
mode: GRACEFUL
new_leader_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:35992
I20260709 13:07:48.033668 19126 raft_consensus.cc:606] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 LEADER]: Received request to transfer leadership to TS 07cd48e27e034bdda1af81e80009363c
I20260709 13:07:48.034977  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:48.035600  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:48.036415  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:48.056334 19484 raft_consensus.cc:993] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c: : Instructing follower 14f4e3e7668c4c7998f4c46e7ab56edf to start an election
I20260709 13:07:48.056994 19484 raft_consensus.cc:1081] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 2 LEADER]: Signalling peer 14f4e3e7668c4c7998f4c46e7ab56edf to start an election
I20260709 13:07:48.059038 19126 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "375340f82f454f609b1dc1f347042929"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
 from {username='slave'} at 127.0.0.1:37092
I20260709 13:07:48.059656 19126 raft_consensus.cc:493] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:48.060050 19126 raft_consensus.cc:3060] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:48.071041 19126 raft_consensus.cc:515] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:48.073334 19126 leader_election.cc:290] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 3 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:48.074748 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "375340f82f454f609b1dc1f347042929" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:48.074748 19404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "375340f82f454f609b1dc1f347042929" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:48.075994 19200 raft_consensus.cc:3055] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:48.076035 19404 raft_consensus.cc:3060] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:48.076557 19200 raft_consensus.cc:740] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:48.077989 19200 consensus_queue.cc:260] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:48.080096 19200 raft_consensus.cc:3060] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:48.089217 19404 raft_consensus.cc:2468] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 3.
I20260709 13:07:48.091162 19094 leader_election.cc:304] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:48.092283 19465 raft_consensus.cc:2804] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:48.092412 19200 raft_consensus.cc:2468] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 3.
I20260709 13:07:48.092823 19465 raft_consensus.cc:697] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 LEADER]: Becoming Leader. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:48.094012 19465 consensus_queue.cc:237] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:48.102995 18950 catalog_manager.cc:5697] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf reported cstate change: term changed from 2 to 3, leader changed from 07cd48e27e034bdda1af81e80009363c (127.7.89.131) to 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130). New cstate: current_term: 3 leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:48.307118 19249 raft_consensus.cc:993] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf: : Instructing follower 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:48.307726 19465 raft_consensus.cc:1081] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 LEADER]: Signalling peer 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:48.309338 19200 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "897c954cea374c7bbfb017af4c503055"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:07:48.309789 19200 raft_consensus.cc:493] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:48.310040 19200 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:48.316434 19200 raft_consensus.cc:515] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:48.318245 19200 leader_election.cc:290] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 5 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:48.319238 19404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:48.320094 19404 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:48.319895 19126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897c954cea374c7bbfb017af4c503055" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:48.320827 19126 raft_consensus.cc:3055] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:48.321199 19126 raft_consensus.cc:740] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:48.322198 19126 consensus_queue.cc:260] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:48.323853 19126 raft_consensus.cc:3060] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:48.330662 19404 raft_consensus.cc:2468] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 5.
I20260709 13:07:48.332386 19168 leader_election.cc:304] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:48.333402 19489 raft_consensus.cc:2804] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:48.333954 19489 raft_consensus.cc:697] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 5 LEADER]: Becoming Leader. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:48.334028 19126 raft_consensus.cc:2468] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 5.
I20260709 13:07:48.335255 19489 consensus_queue.cc:237] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:48.343478 18950 catalog_manager.cc:5697] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c reported cstate change: term changed from 4 to 5, leader changed from 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130) to 07cd48e27e034bdda1af81e80009363c (127.7.89.131). New cstate: current_term: 5 leader_uuid: "07cd48e27e034bdda1af81e80009363c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:48.477402 18999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:48.482378 19249 raft_consensus.cc:993] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf: : Instructing follower 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:48.483070 19465 raft_consensus.cc:1081] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Signalling peer 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:48.485266 19200 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:07:48.485867 19200 raft_consensus.cc:493] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:48.486269 19200 raft_consensus.cc:3060] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:48.492416 19200 raft_consensus.cc:515] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:48.494098 19200 leader_election.cc:290] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 3 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:48.495378 19404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:48.495453 19126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:48.496208 19404 raft_consensus.cc:3060] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:48.496270 19126 raft_consensus.cc:3055] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:48.496805 19126 raft_consensus.cc:740] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:48.497781 19126 consensus_queue.cc:260] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:48.499320 19126 raft_consensus.cc:3060] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:48.503672 19404 raft_consensus.cc:2468] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 3.
I20260709 13:07:48.505666 19168 leader_election.cc:304] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:48.506546 19489 raft_consensus.cc:2804] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:48.507091 19489 raft_consensus.cc:697] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 3 LEADER]: Becoming Leader. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:48.507891 19126 raft_consensus.cc:2468] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 3.
I20260709 13:07:48.508452 19489 consensus_queue.cc:237] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:48.516073 18950 catalog_manager.cc:5697] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c reported cstate change: term changed from 2 to 3, leader changed from 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130) to 07cd48e27e034bdda1af81e80009363c (127.7.89.131). New cstate: current_term: 3 leader_uuid: "07cd48e27e034bdda1af81e80009363c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:48.596756 19200 raft_consensus.cc:1275] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:48.598222 19249 consensus_queue.cc:1048] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:48.609974 19404 raft_consensus.cc:1275] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:48.611783 19465 consensus_queue.cc:1048] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:48.767601 19126 raft_consensus.cc:1275] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 5 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:07:48.769040 19489 consensus_queue.cc:1048] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:48.777395 19405 raft_consensus.cc:1275] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 5 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:07:48.779495 19489 consensus_queue.cc:1048] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:49.039703 19405 raft_consensus.cc:1275] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:49.041465 19489 consensus_queue.cc:1048] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:49.046989  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:49.056685 19126 raft_consensus.cc:1275] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:49.059461 19501 consensus_queue.cc:1048] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:49.066129 19200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "280ee5e3334b4646b9baf8ea4c11b8e9"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
mode: GRACEFUL
new_leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
 from {username='slave'} at 127.0.0.1:50076
I20260709 13:07:49.066963 19200 raft_consensus.cc:606] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Received request to transfer leadership to TS 14f4e3e7668c4c7998f4c46e7ab56edf
I20260709 13:07:49.070737 19126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c504c2e2f0b49b4aa1b80600f93b37b"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
mode: GRACEFUL
new_leader_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:35992
I20260709 13:07:49.071403 19126 raft_consensus.cc:606] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Received request to transfer leadership to TS 07cd48e27e034bdda1af81e80009363c
I20260709 13:07:49.075176 19200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
mode: GRACEFUL
new_leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
 from {username='slave'} at 127.0.0.1:50076
I20260709 13:07:49.075842 19200 raft_consensus.cc:606] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 2 LEADER]: Received request to transfer leadership to TS 14f4e3e7668c4c7998f4c46e7ab56edf
I20260709 13:07:49.077311  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:49.077941  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:49.078706  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:49.121279 19462 raft_consensus.cc:993] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c: : Instructing follower 14f4e3e7668c4c7998f4c46e7ab56edf to start an election
I20260709 13:07:49.122005 19501 raft_consensus.cc:1081] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 2 LEADER]: Signalling peer 14f4e3e7668c4c7998f4c46e7ab56edf to start an election
I20260709 13:07:49.124008 19126 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
 from {username='slave'} at 127.0.0.1:37092
I20260709 13:07:49.124574 19126 raft_consensus.cc:493] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:49.124933 19126 raft_consensus.cc:3060] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:49.131227 19126 raft_consensus.cc:515] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:49.133034 19126 leader_election.cc:290] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 3 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:49.134310 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:49.134187 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:49.135082 19200 raft_consensus.cc:3055] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:49.135170 19405 raft_consensus.cc:3060] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:49.135402 19200 raft_consensus.cc:740] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:49.136495 19200 consensus_queue.cc:260] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:49.137660 19200 raft_consensus.cc:3060] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:49.144577 19405 raft_consensus.cc:2468] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 3.
I20260709 13:07:49.144713 19200 raft_consensus.cc:2468] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 3.
I20260709 13:07:49.146238 19094 leader_election.cc:304] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:49.147187 19465 raft_consensus.cc:2804] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:49.147648 19465 raft_consensus.cc:697] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 LEADER]: Becoming Leader. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:49.148738 19465 consensus_queue.cc:237] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:49.158308 18950 catalog_manager.cc:5697] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf reported cstate change: term changed from 2 to 3, leader changed from 07cd48e27e034bdda1af81e80009363c (127.7.89.131) to 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130). New cstate: current_term: 3 leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:49.274098 19493 raft_consensus.cc:993] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf: : Instructing follower 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:49.274688 19465 raft_consensus.cc:1081] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Signalling peer 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:49.276628 19200 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c504c2e2f0b49b4aa1b80600f93b37b"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:07:49.277185 19200 raft_consensus.cc:493] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:49.277534 19200 raft_consensus.cc:3060] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:49.278787 19462 raft_consensus.cc:993] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c: : Instructing follower 14f4e3e7668c4c7998f4c46e7ab56edf to start an election
I20260709 13:07:49.279428 19462 raft_consensus.cc:1081] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Signalling peer 14f4e3e7668c4c7998f4c46e7ab56edf to start an election
I20260709 13:07:49.281183 19126 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "280ee5e3334b4646b9baf8ea4c11b8e9"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
 from {username='slave'} at 127.0.0.1:37092
I20260709 13:07:49.281749 19126 raft_consensus.cc:493] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:49.282184 19126 raft_consensus.cc:3060] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:49.287345 19200 raft_consensus.cc:515] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:49.289484 19200 leader_election.cc:290] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:49.290445 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c504c2e2f0b49b4aa1b80600f93b37b" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:49.290832 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c504c2e2f0b49b4aa1b80600f93b37b" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:49.291445 19405 raft_consensus.cc:3060] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:49.291612 19127 raft_consensus.cc:3055] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:49.291399 19126 raft_consensus.cc:515] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:49.292291 19127 raft_consensus.cc:740] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:49.293934 19127 consensus_queue.cc:260] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:49.294786 19126 leader_election.cc:290] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:49.295178 19404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280ee5e3334b4646b9baf8ea4c11b8e9" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:49.295961 19127 raft_consensus.cc:3060] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:49.296074 19404 raft_consensus.cc:3060] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:49.299947 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280ee5e3334b4646b9baf8ea4c11b8e9" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:49.300946 19200 raft_consensus.cc:3055] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:49.301360 19200 raft_consensus.cc:740] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:49.301715 19405 raft_consensus.cc:2468] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 2.
I20260709 13:07:49.302537 19200 consensus_queue.cc:260] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:49.303601 19168 leader_election.cc:304] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:49.304298 19200 raft_consensus.cc:3060] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:49.304741 19462 raft_consensus.cc:2804] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:49.305290 19462 raft_consensus.cc:697] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [term 2 LEADER]: Becoming Leader. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:49.306468 19462 consensus_queue.cc:237] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:49.307427 19127 raft_consensus.cc:2468] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 2.
I20260709 13:07:49.306430 19404 raft_consensus.cc:2468] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 2.
I20260709 13:07:49.330375 19094 leader_election.cc:304] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:49.331112 19200 raft_consensus.cc:2468] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 2.
I20260709 13:07:49.331768 19493 raft_consensus.cc:2804] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:49.332429 19493 raft_consensus.cc:697] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Becoming Leader. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:49.333621 19493 consensus_queue.cc:237] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:49.335600 18950 catalog_manager.cc:5697] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c reported cstate change: term changed from 1 to 2, leader changed from 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130) to 07cd48e27e034bdda1af81e80009363c (127.7.89.131). New cstate: current_term: 2 leader_uuid: "07cd48e27e034bdda1af81e80009363c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:49.343698 18947 catalog_manager.cc:5697] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf reported cstate change: term changed from 1 to 2, leader changed from 07cd48e27e034bdda1af81e80009363c (127.7.89.131) to 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130). New cstate: current_term: 2 leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:49.478670 18999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:49.541778 19404 raft_consensus.cc:1275] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:49.543427 19493 consensus_queue.cc:1048] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:49.555282 19200 raft_consensus.cc:1275] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:49.557823 19465 consensus_queue.cc:1048] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:49.691892 19404 raft_consensus.cc:1275] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:49.693351 19462 consensus_queue.cc:1048] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:49.703928 19127 raft_consensus.cc:1275] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:49.707337 19489 consensus_queue.cc:1048] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:49.846325 19404 raft_consensus.cc:1275] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:49.847774 19465 consensus_queue.cc:1048] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:49.855876 19200 raft_consensus.cc:1275] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:49.858541 19249 consensus_queue.cc:1048] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:50.086479  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:50.094527 19127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "280ee5e3334b4646b9baf8ea4c11b8e9"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
mode: GRACEFUL
new_leader_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:35992
I20260709 13:07:50.095220 19127 raft_consensus.cc:606] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Received request to transfer leadership to TS 07cd48e27e034bdda1af81e80009363c
I20260709 13:07:50.098091 19127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
mode: GRACEFUL
new_leader_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:35992
I20260709 13:07:50.098673 19127 raft_consensus.cc:606] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 LEADER]: Received request to transfer leadership to TS 07cd48e27e034bdda1af81e80009363c
I20260709 13:07:50.101173 19200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
mode: GRACEFUL
new_leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
 from {username='slave'} at 127.0.0.1:50076
I20260709 13:07:50.101668 19200 raft_consensus.cc:606] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 3 LEADER]: Received request to transfer leadership to TS 14f4e3e7668c4c7998f4c46e7ab56edf
I20260709 13:07:50.103009  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:50.103585  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:50.104238  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:50.113767 19465 raft_consensus.cc:993] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf: : Instructing follower 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:50.114439 19249 raft_consensus.cc:1081] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 LEADER]: Signalling peer 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:50.116259 19200 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:07:50.116873 19200 raft_consensus.cc:493] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:50.117288 19200 raft_consensus.cc:3060] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:50.123189 19200 raft_consensus.cc:515] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:50.125195 19200 leader_election.cc:290] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 4 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:50.126339 19404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:50.126360 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d7f014f1dfc4d1680a708a717ee0eb5" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:50.127174 19404 raft_consensus.cc:3060] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:50.127377 19127 raft_consensus.cc:3055] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:50.127802 19127 raft_consensus.cc:740] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:50.128669 19127 consensus_queue.cc:260] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:50.129979 19127 raft_consensus.cc:3060] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:50.133971 19404 raft_consensus.cc:2468] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 4.
I20260709 13:07:50.135634 19168 leader_election.cc:304] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:50.136575 19489 raft_consensus.cc:2804] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:50.137113 19489 raft_consensus.cc:697] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 4 LEADER]: Becoming Leader. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:50.138172 19489 consensus_queue.cc:237] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:50.138819 19127 raft_consensus.cc:2468] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 4.
I20260709 13:07:50.146699 18950 catalog_manager.cc:5697] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c reported cstate change: term changed from 3 to 4, leader changed from 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130) to 07cd48e27e034bdda1af81e80009363c (127.7.89.131). New cstate: current_term: 4 leader_uuid: "07cd48e27e034bdda1af81e80009363c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:50.409947 19465 raft_consensus.cc:993] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf: : Instructing follower 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:50.410557 19493 raft_consensus.cc:1081] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Signalling peer 07cd48e27e034bdda1af81e80009363c to start an election
I20260709 13:07:50.412519 19200 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "280ee5e3334b4646b9baf8ea4c11b8e9"
dest_uuid: "07cd48e27e034bdda1af81e80009363c"
 from {username='slave'} at 127.0.0.1:59752
I20260709 13:07:50.413091 19200 raft_consensus.cc:493] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:50.413460 19200 raft_consensus.cc:3060] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:50.420133 19200 raft_consensus.cc:515] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:50.421933 19200 leader_election.cc:290] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 3 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415)
I20260709 13:07:50.423267 19404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280ee5e3334b4646b9baf8ea4c11b8e9" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:50.423267 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280ee5e3334b4646b9baf8ea4c11b8e9" candidate_uuid: "07cd48e27e034bdda1af81e80009363c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
I20260709 13:07:50.424091 19404 raft_consensus.cc:3060] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:50.424403 19127 raft_consensus.cc:3055] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:50.424759 19127 raft_consensus.cc:740] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:50.425666 19127 consensus_queue.cc:260] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:50.427137 19127 raft_consensus.cc:3060] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:50.431349 19404 raft_consensus.cc:2468] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 3.
I20260709 13:07:50.432726 19168 leader_election.cc:304] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07cd48e27e034bdda1af81e80009363c, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:50.433418 19127 raft_consensus.cc:2468] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07cd48e27e034bdda1af81e80009363c in term 3.
I20260709 13:07:50.433736 19519 raft_consensus.cc:2804] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:50.434357 19519 raft_consensus.cc:697] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 3 LEADER]: Becoming Leader. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:50.435228 19519 consensus_queue.cc:237] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:50.442291 18950 catalog_manager.cc:5697] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c reported cstate change: term changed from 2 to 3, leader changed from 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130) to 07cd48e27e034bdda1af81e80009363c (127.7.89.131). New cstate: current_term: 3 leader_uuid: "07cd48e27e034bdda1af81e80009363c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:50.479830 18999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:50.606279 19519 raft_consensus.cc:993] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c: : Instructing follower 14f4e3e7668c4c7998f4c46e7ab56edf to start an election
I20260709 13:07:50.607183 19489 raft_consensus.cc:1081] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 3 LEADER]: Signalling peer 14f4e3e7668c4c7998f4c46e7ab56edf to start an election
I20260709 13:07:50.608999 19127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b"
dest_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf"
 from {username='slave'} at 127.0.0.1:37092
I20260709 13:07:50.609515 19127 raft_consensus.cc:493] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:50.609804 19127 raft_consensus.cc:3060] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:50.615788 19127 raft_consensus.cc:515] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:50.617457 19127 leader_election.cc:290] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 4 election: Requested vote from peers f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445), 07cd48e27e034bdda1af81e80009363c (127.7.89.131:44617)
I20260709 13:07:50.618558 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "07cd48e27e034bdda1af81e80009363c"
I20260709 13:07:50.618549 19404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b9646c958c4e8c9e6133698a77d51b" candidate_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f4c532a3d5a243c285f2c23fdfd914f8"
I20260709 13:07:50.619392 19200 raft_consensus.cc:3055] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:50.619501 19404 raft_consensus.cc:3060] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:50.619692 19200 raft_consensus.cc:740] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 07cd48e27e034bdda1af81e80009363c, State: Running, Role: LEADER
I20260709 13:07:50.620458 19200 consensus_queue.cc:260] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:50.621457 19200 raft_consensus.cc:3060] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:50.625712 19404 raft_consensus.cc:2468] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 4.
I20260709 13:07:50.627030 19200 raft_consensus.cc:2468] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14f4e3e7668c4c7998f4c46e7ab56edf in term 4.
I20260709 13:07:50.627041 19094 leader_election.cc:304] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14f4e3e7668c4c7998f4c46e7ab56edf, f4c532a3d5a243c285f2c23fdfd914f8; no voters: 
I20260709 13:07:50.628134 19493 raft_consensus.cc:2804] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:50.628659 19493 raft_consensus.cc:697] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 LEADER]: Becoming Leader. State: Replica: 14f4e3e7668c4c7998f4c46e7ab56edf, State: Running, Role: LEADER
I20260709 13:07:50.629539 19493 consensus_queue.cc:237] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } }
I20260709 13:07:50.636982 19127 raft_consensus.cc:1275] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:50.637989 18950 catalog_manager.cc:5697] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf reported cstate change: term changed from 3 to 4, leader changed from 07cd48e27e034bdda1af81e80009363c (127.7.89.131) to 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130). New cstate: current_term: 4 leader_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:50.638600 19489 consensus_queue.cc:1048] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:50.654479 19404 raft_consensus.cc:1275] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 4 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:50.656049 19519 consensus_queue.cc:1048] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:50.816761 19404 raft_consensus.cc:1275] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:50.818218 19519 consensus_queue.cc:1048] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:50.828251 19127 raft_consensus.cc:1275] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Refusing update from remote peer 07cd48e27e034bdda1af81e80009363c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:50.830334 19501 consensus_queue.cc:1048] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f4e3e7668c4c7998f4c46e7ab56edf" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:51.110808  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:51.137704 19405 raft_consensus.cc:1275] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 4 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:51.139596 19493 consensus_queue.cc:1048] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4c532a3d5a243c285f2c23fdfd914f8" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 33445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:51.145999  7526 tablet_server.cc:179] TabletServer@127.7.89.129:33445 shutting down...
W20260709 13:07:51.155162 19094 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
I20260709 13:07:51.158138 19200 raft_consensus.cc:1275] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 4 FOLLOWER]: Refusing update from remote peer 14f4e3e7668c4c7998f4c46e7ab56edf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:51.160303 19531 consensus_queue.cc:1048] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cd48e27e034bdda1af81e80009363c" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:07:51.163710 19094 consensus_peers.cc:597] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:51.194217 19168 consensus_peers.cc:597] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:51.202841  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:51.203879  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:51.205006  7526 raft_consensus.cc:2243] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 5 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:51.205338 19168 consensus_peers.cc:597] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:51.205811  7526 raft_consensus.cc:2272] T 897c954cea374c7bbfb017af4c503055 P f4c532a3d5a243c285f2c23fdfd914f8 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.209861  7526 tablet_replica.cc:333] stopping tablet replica
W20260709 13:07:51.210206 19094 consensus_peers.cc:597] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:51.210825  7526 raft_consensus.cc:2243] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.211513  7526 raft_consensus.cc:2272] T 9b8743904619452f81b5552cadd9f491 P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.214469  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:51.215395  7526 raft_consensus.cc:2243] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.215955  7526 raft_consensus.cc:2272] T 375340f82f454f609b1dc1f347042929 P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.218592  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:51.219319  7526 raft_consensus.cc:2243] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.219841  7526 raft_consensus.cc:2272] T 280ee5e3334b4646b9baf8ea4c11b8e9 P f4c532a3d5a243c285f2c23fdfd914f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.223939  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:51.224980  7526 raft_consensus.cc:2243] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.225651  7526 pending_rounds.cc:70] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8: Trying to abort 1 pending ops.
I20260709 13:07:51.225955  7526 pending_rounds.cc:77] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305635719706640384 op_type: NO_OP noop_request { }
I20260709 13:07:51.226400  7526 raft_consensus.cc:2889] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:51.226888  7526 raft_consensus.cc:2272] T 75b9646c958c4e8c9e6133698a77d51b P f4c532a3d5a243c285f2c23fdfd914f8 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:51.228037 19168 consensus_peers.cc:597] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:51.230945  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:51.231760  7526 raft_consensus.cc:2243] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.232331  7526 raft_consensus.cc:2272] T 4c504c2e2f0b49b4aa1b80600f93b37b P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.235208  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:51.235767  7526 raft_consensus.cc:2243] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.236215  7526 raft_consensus.cc:2272] T 6d7f014f1dfc4d1680a708a717ee0eb5 P f4c532a3d5a243c285f2c23fdfd914f8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.239336  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:51.240072  7526 raft_consensus.cc:2243] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.240599  7526 raft_consensus.cc:2272] T 5f04a57ced394d7db3587eb966026f3a P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.243322  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:51.243850  7526 raft_consensus.cc:2243] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.244287  7526 raft_consensus.cc:2272] T 7816d5b785b0476f9243429fc021d806 P f4c532a3d5a243c285f2c23fdfd914f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.247171  7526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:51.247720  7526 raft_consensus.cc:2243] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.248188  7526 raft_consensus.cc:2272] T 05278d681f5f466e9cfc401e8f7e191d P f4c532a3d5a243c285f2c23fdfd914f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.283905  7526 tablet_server.cc:196] TabletServer@127.7.89.129:33445 shutdown complete.
W20260709 13:07:51.302488 19094 consensus_peers.cc:597] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:51.305132 19168 consensus_peers.cc:597] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c -> Peer f4c532a3d5a243c285f2c23fdfd914f8 (127.7.89.129:33445): Couldn't send request to peer f4c532a3d5a243c285f2c23fdfd914f8. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:33445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:51.321585  7526 tablet_server.cc:179] TabletServer@127.7.89.130:0 shutting down...
I20260709 13:07:51.354444  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:51.355450  7526 tablet_replica.cc:333] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf: stopping tablet replica
I20260709 13:07:51.356417  7526 raft_consensus.cc:2243] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.357018  7526 raft_consensus.cc:2272] T 897c954cea374c7bbfb017af4c503055 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.360146  7526 tablet_replica.cc:333] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf: stopping tablet replica
I20260709 13:07:51.361416  7526 raft_consensus.cc:2243] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.362322  7526 raft_consensus.cc:2272] T 7816d5b785b0476f9243429fc021d806 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.365734  7526 tablet_replica.cc:333] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf: stopping tablet replica
I20260709 13:07:51.366560  7526 raft_consensus.cc:2243] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.367771  7526 raft_consensus.cc:2272] T 9b8743904619452f81b5552cadd9f491 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.370414  7526 tablet_replica.cc:333] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf: stopping tablet replica
I20260709 13:07:51.371099  7526 raft_consensus.cc:2243] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.371688  7526 raft_consensus.cc:2272] T 05278d681f5f466e9cfc401e8f7e191d P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.374476  7526 tablet_replica.cc:333] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf: stopping tablet replica
I20260709 13:07:51.375236  7526 raft_consensus.cc:2243] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.376238  7526 raft_consensus.cc:2272] T 75b9646c958c4e8c9e6133698a77d51b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.379544  7526 tablet_replica.cc:333] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf: stopping tablet replica
I20260709 13:07:51.380247  7526 raft_consensus.cc:2243] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.380782  7526 raft_consensus.cc:2272] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.384128  7526 tablet_replica.cc:333] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf: stopping tablet replica
I20260709 13:07:51.384878  7526 raft_consensus.cc:2243] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.385397  7526 raft_consensus.cc:2272] T 4c504c2e2f0b49b4aa1b80600f93b37b P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.387971  7526 tablet_replica.cc:333] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf: stopping tablet replica
I20260709 13:07:51.388620  7526 raft_consensus.cc:2243] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.389611  7526 raft_consensus.cc:2272] T 5f04a57ced394d7db3587eb966026f3a P 14f4e3e7668c4c7998f4c46e7ab56edf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.392330  7526 tablet_replica.cc:333] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf: stopping tablet replica
I20260709 13:07:51.393006  7526 raft_consensus.cc:2243] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.393499  7526 raft_consensus.cc:2272] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.396665  7526 tablet_replica.cc:333] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf: stopping tablet replica
I20260709 13:07:51.397284  7526 raft_consensus.cc:2243] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.398205  7526 raft_consensus.cc:2272] T 375340f82f454f609b1dc1f347042929 P 14f4e3e7668c4c7998f4c46e7ab56edf [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:51.432929 19166 consensus_peers.cc:597] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c -> Peer 14f4e3e7668c4c7998f4c46e7ab56edf (127.7.89.130:33415): Couldn't send request to peer 14f4e3e7668c4c7998f4c46e7ab56edf. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:51.435341  7526 tablet_server.cc:196] TabletServer@127.7.89.130:0 shutdown complete.
I20260709 13:07:51.470836  7526 tablet_server.cc:179] TabletServer@127.7.89.131:0 shutting down...
W20260709 13:07:51.482187 18999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:51.505023  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:51.506157  7526 tablet_replica.cc:333] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c: stopping tablet replica
I20260709 13:07:51.507077  7526 raft_consensus.cc:2243] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.508308  7526 raft_consensus.cc:2272] T 05278d681f5f466e9cfc401e8f7e191d P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.511116  7526 tablet_replica.cc:333] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c: stopping tablet replica
I20260709 13:07:51.512007  7526 raft_consensus.cc:2243] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.512634  7526 raft_consensus.cc:2272] T 5f04a57ced394d7db3587eb966026f3a P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.515551  7526 tablet_replica.cc:333] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c: stopping tablet replica
I20260709 13:07:51.516296  7526 raft_consensus.cc:2243] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.517299  7526 raft_consensus.cc:2272] T 6d7f014f1dfc4d1680a708a717ee0eb5 P 07cd48e27e034bdda1af81e80009363c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.520762  7526 tablet_replica.cc:333] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c: stopping tablet replica
I20260709 13:07:51.521592  7526 raft_consensus.cc:2243] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.522863  7526 raft_consensus.cc:2272] T 897c954cea374c7bbfb017af4c503055 P 07cd48e27e034bdda1af81e80009363c [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.525950  7526 tablet_replica.cc:333] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c: stopping tablet replica
I20260709 13:07:51.526816  7526 raft_consensus.cc:2243] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.527403  7526 raft_consensus.cc:2272] T 375340f82f454f609b1dc1f347042929 P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.530364  7526 tablet_replica.cc:333] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c: stopping tablet replica
I20260709 13:07:51.531332  7526 raft_consensus.cc:2243] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.531957  7526 raft_consensus.cc:2272] T 75b9646c958c4e8c9e6133698a77d51b P 07cd48e27e034bdda1af81e80009363c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.535617  7526 tablet_replica.cc:333] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c: stopping tablet replica
I20260709 13:07:51.536417  7526 raft_consensus.cc:2243] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.537463  7526 raft_consensus.cc:2272] T 4c504c2e2f0b49b4aa1b80600f93b37b P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.540416  7526 tablet_replica.cc:333] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c: stopping tablet replica
I20260709 13:07:51.541253  7526 raft_consensus.cc:2243] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.542363  7526 raft_consensus.cc:2272] T 280ee5e3334b4646b9baf8ea4c11b8e9 P 07cd48e27e034bdda1af81e80009363c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.545838  7526 tablet_replica.cc:333] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c: stopping tablet replica
I20260709 13:07:51.546550  7526 raft_consensus.cc:2243] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:51.547397  7526 raft_consensus.cc:2272] T 9b8743904619452f81b5552cadd9f491 P 07cd48e27e034bdda1af81e80009363c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.551023  7526 tablet_replica.cc:333] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c: stopping tablet replica
I20260709 13:07:51.551891  7526 raft_consensus.cc:2243] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.553037  7526 raft_consensus.cc:2272] T 7816d5b785b0476f9243429fc021d806 P 07cd48e27e034bdda1af81e80009363c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.591189  7526 tablet_server.cc:196] TabletServer@127.7.89.131:0 shutdown complete.
I20260709 13:07:51.622231  7526 master.cc:562] Master@127.7.89.190:43061 shutting down...
I20260709 13:07:51.654340  7526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.655189  7526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.655614  7526 tablet_replica.cc:333] T 00000000000000000000000000000000 P c997592ab6304092a0cba7c630ef9b2d: stopping tablet replica
I20260709 13:07:51.680967  7526 master.cc:584] Master@127.7.89.190:43061 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13533 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:07:51.731777  7526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.89.190:34969
I20260709 13:07:51.733155  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:51.739472 19539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:51.739521 19538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:51.741216 19541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:51.742306  7526 server_base.cc:1061] running on GCE node
I20260709 13:07:51.743592  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:51.743779  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:51.743979  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602471743941 us; error 0 us; skew 500 ppm
I20260709 13:07:51.744581  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:51.747735  7526 webserver.cc:533] Webserver started at http://127.7.89.190:36949/ using document root <none> and password file <none>
I20260709 13:07:51.748323  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:51.748512  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:51.748852  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:51.750154  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/master-0-root/instance:
uuid: "5d223354cef4409e8f6b63379c8cd7fb"
format_stamp: "Formatted at 2026-07-09 13:07:51 on dist-test-slave-h0rz"
I20260709 13:07:51.755734  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.002s
I20260709 13:07:51.759711 19546 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:51.760612  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:07:51.760952  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/master-0-root
uuid: "5d223354cef4409e8f6b63379c8cd7fb"
format_stamp: "Formatted at 2026-07-09 13:07:51 on dist-test-slave-h0rz"
I20260709 13:07:51.761243  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:51.785609  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:51.787189  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:51.832041  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.190:34969
I20260709 13:07:51.832149 19597 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.190:34969 every 8 connection(s)
I20260709 13:07:51.836469 19598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:51.850258 19598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb: Bootstrap starting.
I20260709 13:07:51.856034 19598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.861088 19598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb: No bootstrap required, opened a new log
I20260709 13:07:51.863847 19598 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d223354cef4409e8f6b63379c8cd7fb" member_type: VOTER }
I20260709 13:07:51.864326 19598 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.864530 19598 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d223354cef4409e8f6b63379c8cd7fb, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.865175 19598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d223354cef4409e8f6b63379c8cd7fb" member_type: VOTER }
I20260709 13:07:51.865688 19598 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:51.865907 19598 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:51.866135 19598 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.872236 19598 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d223354cef4409e8f6b63379c8cd7fb" member_type: VOTER }
I20260709 13:07:51.872845 19598 leader_election.cc:304] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d223354cef4409e8f6b63379c8cd7fb; no voters: 
I20260709 13:07:51.874076 19598 leader_election.cc:290] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:51.874698 19601 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:51.876315 19601 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [term 1 LEADER]: Becoming Leader. State: Replica: 5d223354cef4409e8f6b63379c8cd7fb, State: Running, Role: LEADER
I20260709 13:07:51.877100 19601 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d223354cef4409e8f6b63379c8cd7fb" member_type: VOTER }
I20260709 13:07:51.877434 19598 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:51.883261 19603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d223354cef4409e8f6b63379c8cd7fb. Latest consensus state: current_term: 1 leader_uuid: "5d223354cef4409e8f6b63379c8cd7fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d223354cef4409e8f6b63379c8cd7fb" member_type: VOTER } }
I20260709 13:07:51.883363 19602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d223354cef4409e8f6b63379c8cd7fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d223354cef4409e8f6b63379c8cd7fb" member_type: VOTER } }
I20260709 13:07:51.884307 19603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:51.884397 19602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:51.886657 19609 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:51.892925 19609 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:51.896133  7526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:51.904275 19609 catalog_manager.cc:1383] Generated new cluster ID: 0ef8f3bde0374615acc907ffd0569c10
I20260709 13:07:51.904642 19609 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:51.936095 19609 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:51.937542 19609 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:51.951802 19609 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb: Generated new TSK 0
I20260709 13:07:51.952742 19609 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:51.963478  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:51.971894 19619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:51.973176 19620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:51.976182 19622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:51.976464  7526 server_base.cc:1061] running on GCE node
I20260709 13:07:51.977870  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:51.978107  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:51.978240  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602471978227 us; error 0 us; skew 500 ppm
I20260709 13:07:51.978916  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:51.981930  7526 webserver.cc:533] Webserver started at http://127.7.89.129:46181/ using document root <none> and password file <none>
I20260709 13:07:51.982591  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:51.982873  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:51.983204  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:51.984627  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-0-root/instance:
uuid: "fe6a23bd84de411380327ea182db7081"
format_stamp: "Formatted at 2026-07-09 13:07:51 on dist-test-slave-h0rz"
I20260709 13:07:51.990393  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 13:07:51.994738 19627 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:51.995694  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:07:51.996099  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-0-root
uuid: "fe6a23bd84de411380327ea182db7081"
format_stamp: "Formatted at 2026-07-09 13:07:51 on dist-test-slave-h0rz"
I20260709 13:07:51.996429  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:52.007885  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:52.009303  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:52.011205  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:52.013916  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:52.014205  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.014497  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:52.014735  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:07:52.066031  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.129:44905
I20260709 13:07:52.066115 19689 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.129:44905 every 8 connection(s)
I20260709 13:07:52.072705  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:52.081365 19694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:52.084712 19695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:52.091837 19690 heartbeater.cc:344] Connected to a master server at 127.7.89.190:34969
I20260709 13:07:52.092235  7526 server_base.cc:1061] running on GCE node
W20260709 13:07:52.091979 19697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:52.092650 19690 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:52.093691  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:07:52.093699 19690 heartbeater.cc:507] Master 127.7.89.190:34969 requested a full tablet report, sending...
W20260709 13:07:52.093997  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:52.094338  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602472094315 us; error 0 us; skew 500 ppm
I20260709 13:07:52.095088  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:52.096766 19563 ts_manager.cc:194] Registered new tserver with Master: fe6a23bd84de411380327ea182db7081 (127.7.89.129:44905)
I20260709 13:07:52.098742  7526 webserver.cc:533] Webserver started at http://127.7.89.130:35037/ using document root <none> and password file <none>
I20260709 13:07:52.099447  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:52.099319 19563 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53260
I20260709 13:07:52.099772  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:52.100137  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:52.101840  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-1-root/instance:
uuid: "53ab9ab583da48de99064faf5a7b95f3"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-h0rz"
I20260709 13:07:52.107416  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:07:52.111701 19702 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.112820  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:07:52.113231  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-1-root
uuid: "53ab9ab583da48de99064faf5a7b95f3"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-h0rz"
I20260709 13:07:52.113616  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:52.146143  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:52.147792  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:52.149616  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:52.152325  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:52.152643  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.152913  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:52.153115  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.202126  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.130:44805
I20260709 13:07:52.202247 19764 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.130:44805 every 8 connection(s)
I20260709 13:07:52.208139  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:52.217293 19768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:52.220680 19769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:52.222360 19771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:52.224378 19765 heartbeater.cc:344] Connected to a master server at 127.7.89.190:34969
I20260709 13:07:52.224938 19765 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:52.225139  7526 server_base.cc:1061] running on GCE node
I20260709 13:07:52.225929 19765 heartbeater.cc:507] Master 127.7.89.190:34969 requested a full tablet report, sending...
I20260709 13:07:52.226774  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:52.227236  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:52.227538  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602472227515 us; error 0 us; skew 500 ppm
I20260709 13:07:52.228698  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:52.229550 19563 ts_manager.cc:194] Registered new tserver with Master: 53ab9ab583da48de99064faf5a7b95f3 (127.7.89.130:44805)
I20260709 13:07:52.231989 19563 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53272
I20260709 13:07:52.235060  7526 webserver.cc:533] Webserver started at http://127.7.89.131:42389/ using document root <none> and password file <none>
I20260709 13:07:52.236061  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:52.236375  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:52.236793  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:52.238860  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-2-root/instance:
uuid: "9f7901d190834b2da3910b213a7a8281"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-h0rz"
I20260709 13:07:52.245050  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:07:52.249599 19776 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.250931  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:07:52.251332  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-2-root
uuid: "9f7901d190834b2da3910b213a7a8281"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-h0rz"
I20260709 13:07:52.251658  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602061986899-7526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:52.267693  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:52.269169  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:52.270681  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:52.273293  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:52.273581  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.273854  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:52.274047  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.324863  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.131:46163
I20260709 13:07:52.324963 19838 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.131:46163 every 8 connection(s)
I20260709 13:07:52.341984 19839 heartbeater.cc:344] Connected to a master server at 127.7.89.190:34969
I20260709 13:07:52.342506 19839 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:52.343502 19839 heartbeater.cc:507] Master 127.7.89.190:34969 requested a full tablet report, sending...
I20260709 13:07:52.345799 19563 ts_manager.cc:194] Registered new tserver with Master: 9f7901d190834b2da3910b213a7a8281 (127.7.89.131:46163)
I20260709 13:07:52.346487  7526 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017663397s
I20260709 13:07:52.347525 19563 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53274
I20260709 13:07:52.379817 19563 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53286:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:52.382416 19563 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:52.447283 19655 tablet_service.cc:1511] Processing CreateTablet for tablet 7ce825bbc71842a28b9746894f170c85 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=33bb942b10bc45baaddc63ad76dac9e5]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:52.447678 19654 tablet_service.cc:1511] Processing CreateTablet for tablet d15b991955494e1081504f24331e65ed (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=33bb942b10bc45baaddc63ad76dac9e5]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:52.449151 19655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ce825bbc71842a28b9746894f170c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.449939 19654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d15b991955494e1081504f24331e65ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.455897 19729 tablet_service.cc:1511] Processing CreateTablet for tablet d15b991955494e1081504f24331e65ed (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=33bb942b10bc45baaddc63ad76dac9e5]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:52.457722 19729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d15b991955494e1081504f24331e65ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.463310 19730 tablet_service.cc:1511] Processing CreateTablet for tablet 7ce825bbc71842a28b9746894f170c85 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=33bb942b10bc45baaddc63ad76dac9e5]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:52.465042 19730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ce825bbc71842a28b9746894f170c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.475397 19803 tablet_service.cc:1511] Processing CreateTablet for tablet d15b991955494e1081504f24331e65ed (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=33bb942b10bc45baaddc63ad76dac9e5]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:52.477138 19803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d15b991955494e1081504f24331e65ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.475396 19804 tablet_service.cc:1511] Processing CreateTablet for tablet 7ce825bbc71842a28b9746894f170c85 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=33bb942b10bc45baaddc63ad76dac9e5]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:52.481705 19804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ce825bbc71842a28b9746894f170c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.494220 19854 tablet_bootstrap.cc:492] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081: Bootstrap starting.
I20260709 13:07:52.497922 19855 tablet_bootstrap.cc:492] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3: Bootstrap starting.
I20260709 13:07:52.504504 19854 tablet_bootstrap.cc:654] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.508085 19855 tablet_bootstrap.cc:654] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.517831 19856 tablet_bootstrap.cc:492] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281: Bootstrap starting.
I20260709 13:07:52.518837 19854 tablet_bootstrap.cc:492] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081: No bootstrap required, opened a new log
I20260709 13:07:52.519542 19854 ts_tablet_manager.cc:1403] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.013s
I20260709 13:07:52.521136 19855 tablet_bootstrap.cc:492] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3: No bootstrap required, opened a new log
I20260709 13:07:52.521764 19855 ts_tablet_manager.cc:1403] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.004s
I20260709 13:07:52.522193 19854 raft_consensus.cc:359] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.522949 19854 raft_consensus.cc:385] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.523214 19854 raft_consensus.cc:740] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6a23bd84de411380327ea182db7081, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.523977 19854 consensus_queue.cc:260] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.525266 19855 raft_consensus.cc:359] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } }
I20260709 13:07:52.526108 19855 raft_consensus.cc:385] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.526510 19856 tablet_bootstrap.cc:654] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.526438 19855 raft_consensus.cc:740] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ab9ab583da48de99064faf5a7b95f3, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.527640 19690 heartbeater.cc:499] Master 127.7.89.190:34969 was elected leader, sending a full tablet report...
I20260709 13:07:52.527941 19854 ts_tablet_manager.cc:1434] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:07:52.527753 19855 consensus_queue.cc:260] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } }
I20260709 13:07:52.529253 19854 tablet_bootstrap.cc:492] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081: Bootstrap starting.
I20260709 13:07:52.540535 19855 ts_tablet_manager.cc:1434] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.007s
I20260709 13:07:52.540676 19854 tablet_bootstrap.cc:654] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.541612 19765 heartbeater.cc:499] Master 127.7.89.190:34969 was elected leader, sending a full tablet report...
I20260709 13:07:52.542215 19855 tablet_bootstrap.cc:492] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3: Bootstrap starting.
I20260709 13:07:52.541150 19856 tablet_bootstrap.cc:492] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281: No bootstrap required, opened a new log
I20260709 13:07:52.544598 19856 ts_tablet_manager.cc:1403] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.004s
I20260709 13:07:52.548377 19856 raft_consensus.cc:359] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } }
I20260709 13:07:52.549256 19856 raft_consensus.cc:385] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.549499 19855 tablet_bootstrap.cc:654] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.549597 19856 raft_consensus.cc:740] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f7901d190834b2da3910b213a7a8281, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.550808 19856 consensus_queue.cc:260] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } }
I20260709 13:07:52.553570 19839 heartbeater.cc:499] Master 127.7.89.190:34969 was elected leader, sending a full tablet report...
I20260709 13:07:52.554189 19856 ts_tablet_manager.cc:1434] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:07:52.557080 19856 tablet_bootstrap.cc:492] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281: Bootstrap starting.
I20260709 13:07:52.561622 19854 tablet_bootstrap.cc:492] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081: No bootstrap required, opened a new log
I20260709 13:07:52.562515 19854 ts_tablet_manager.cc:1403] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.006s
I20260709 13:07:52.566177 19854 raft_consensus.cc:359] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } }
I20260709 13:07:52.567605 19854 raft_consensus.cc:385] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.568079 19854 raft_consensus.cc:740] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6a23bd84de411380327ea182db7081, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.568993 19854 consensus_queue.cc:260] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } }
I20260709 13:07:52.572088 19854 ts_tablet_manager.cc:1434] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:07:52.574769 19856 tablet_bootstrap.cc:654] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.581121 19855 tablet_bootstrap.cc:492] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3: No bootstrap required, opened a new log
I20260709 13:07:52.581815 19855 ts_tablet_manager.cc:1403] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.019s
I20260709 13:07:52.583770 19856 tablet_bootstrap.cc:492] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281: No bootstrap required, opened a new log
I20260709 13:07:52.584517 19856 ts_tablet_manager.cc:1403] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.005s
I20260709 13:07:52.584972 19855 raft_consensus.cc:359] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.585688 19855 raft_consensus.cc:385] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.585947 19855 raft_consensus.cc:740] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ab9ab583da48de99064faf5a7b95f3, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.586817 19855 consensus_queue.cc:260] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.588213 19856 raft_consensus.cc:359] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.588857 19856 raft_consensus.cc:385] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.589047 19856 raft_consensus.cc:740] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f7901d190834b2da3910b213a7a8281, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.589205 19855 ts_tablet_manager.cc:1434] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:07:52.589828 19856 consensus_queue.cc:260] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.592273 19856 ts_tablet_manager.cc:1434] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:07:52.701776 19862 raft_consensus.cc:493] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.702721 19862 raft_consensus.cc:515] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } }
I20260709 13:07:52.705104 19862 leader_election.cc:290] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe6a23bd84de411380327ea182db7081 (127.7.89.129:44905), 53ab9ab583da48de99064faf5a7b95f3 (127.7.89.130:44805)
I20260709 13:07:52.708388 19860 raft_consensus.cc:493] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.709152 19860 raft_consensus.cc:515] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.712484 19860 leader_election.cc:290] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe6a23bd84de411380327ea182db7081 (127.7.89.129:44905), 9f7901d190834b2da3910b213a7a8281 (127.7.89.131:46163)
I20260709 13:07:52.720849 19665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ce825bbc71842a28b9746894f170c85" candidate_uuid: "9f7901d190834b2da3910b213a7a8281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6a23bd84de411380327ea182db7081" is_pre_election: true
I20260709 13:07:52.722003 19665 raft_consensus.cc:2468] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f7901d190834b2da3910b213a7a8281 in term 0.
I20260709 13:07:52.723950 19777 leader_election.cc:304] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f7901d190834b2da3910b213a7a8281, fe6a23bd84de411380327ea182db7081; no voters: 
I20260709 13:07:52.725057 19862 raft_consensus.cc:2804] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:52.725493 19862 raft_consensus.cc:493] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.725862 19862 raft_consensus.cc:3060] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.727110 19740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ce825bbc71842a28b9746894f170c85" candidate_uuid: "9f7901d190834b2da3910b213a7a8281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ab9ab583da48de99064faf5a7b95f3" is_pre_election: true
I20260709 13:07:52.728016 19740 raft_consensus.cc:2468] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f7901d190834b2da3910b213a7a8281 in term 0.
I20260709 13:07:52.728703 19814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15b991955494e1081504f24331e65ed" candidate_uuid: "53ab9ab583da48de99064faf5a7b95f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f7901d190834b2da3910b213a7a8281" is_pre_election: true
I20260709 13:07:52.729562 19814 raft_consensus.cc:2468] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ab9ab583da48de99064faf5a7b95f3 in term 0.
I20260709 13:07:52.730484 19665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15b991955494e1081504f24331e65ed" candidate_uuid: "53ab9ab583da48de99064faf5a7b95f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6a23bd84de411380327ea182db7081" is_pre_election: true
I20260709 13:07:52.731060 19703 leader_election.cc:304] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ab9ab583da48de99064faf5a7b95f3, 9f7901d190834b2da3910b213a7a8281; no voters: 
I20260709 13:07:52.731406 19665 raft_consensus.cc:2468] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ab9ab583da48de99064faf5a7b95f3 in term 0.
I20260709 13:07:52.732059 19860 raft_consensus.cc:2804] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:52.732481 19860 raft_consensus.cc:493] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.732818 19860 raft_consensus.cc:3060] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.737653 19862 raft_consensus.cc:515] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } }
I20260709 13:07:52.740501 19862 leader_election.cc:290] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [CANDIDATE]: Term 1 election: Requested vote from peers fe6a23bd84de411380327ea182db7081 (127.7.89.129:44905), 53ab9ab583da48de99064faf5a7b95f3 (127.7.89.130:44805)
I20260709 13:07:52.741487 19665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ce825bbc71842a28b9746894f170c85" candidate_uuid: "9f7901d190834b2da3910b213a7a8281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6a23bd84de411380327ea182db7081"
I20260709 13:07:52.741818 19740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ce825bbc71842a28b9746894f170c85" candidate_uuid: "9f7901d190834b2da3910b213a7a8281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ab9ab583da48de99064faf5a7b95f3"
I20260709 13:07:52.742167 19665 raft_consensus.cc:3060] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.742548 19740 raft_consensus.cc:3060] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.743105 19860 raft_consensus.cc:515] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.745734 19860 leader_election.cc:290] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [CANDIDATE]: Term 1 election: Requested vote from peers fe6a23bd84de411380327ea182db7081 (127.7.89.129:44905), 9f7901d190834b2da3910b213a7a8281 (127.7.89.131:46163)
I20260709 13:07:52.746922 19664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15b991955494e1081504f24331e65ed" candidate_uuid: "53ab9ab583da48de99064faf5a7b95f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6a23bd84de411380327ea182db7081"
I20260709 13:07:52.747593 19814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d15b991955494e1081504f24331e65ed" candidate_uuid: "53ab9ab583da48de99064faf5a7b95f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f7901d190834b2da3910b213a7a8281"
I20260709 13:07:52.747856 19664 raft_consensus.cc:3060] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.748273 19814 raft_consensus.cc:3060] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.753198 19740 raft_consensus.cc:2468] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f7901d190834b2da3910b213a7a8281 in term 1.
I20260709 13:07:52.755085 19778 leader_election.cc:304] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ab9ab583da48de99064faf5a7b95f3, 9f7901d190834b2da3910b213a7a8281; no voters: 
I20260709 13:07:52.755059 19665 raft_consensus.cc:2468] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f7901d190834b2da3910b213a7a8281 in term 1.
I20260709 13:07:52.756374 19862 raft_consensus.cc:2804] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:52.757555 19664 raft_consensus.cc:2468] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ab9ab583da48de99064faf5a7b95f3 in term 1.
I20260709 13:07:52.758278 19862 raft_consensus.cc:697] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 1 LEADER]: Becoming Leader. State: Replica: 9f7901d190834b2da3910b213a7a8281, State: Running, Role: LEADER
I20260709 13:07:52.759636 19814 raft_consensus.cc:2468] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ab9ab583da48de99064faf5a7b95f3 in term 1.
I20260709 13:07:52.759397 19703 leader_election.cc:304] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ab9ab583da48de99064faf5a7b95f3, fe6a23bd84de411380327ea182db7081; no voters: 
I20260709 13:07:52.759508 19862 consensus_queue.cc:237] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } }
I20260709 13:07:52.760736 19860 raft_consensus.cc:2804] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:52.762194 19860 raft_consensus.cc:697] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [term 1 LEADER]: Becoming Leader. State: Replica: 53ab9ab583da48de99064faf5a7b95f3, State: Running, Role: LEADER
I20260709 13:07:52.763343 19860 consensus_queue.cc:237] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.772712 19562 catalog_manager.cc:5697] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f7901d190834b2da3910b213a7a8281 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "9f7901d190834b2da3910b213a7a8281" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:52.772805 19563 catalog_manager.cc:5697] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ab9ab583da48de99064faf5a7b95f3 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "53ab9ab583da48de99064faf5a7b95f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:52.846952 19562 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53294:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:52.849387 19562 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:52.876333 19654 tablet_service.cc:1511] Processing CreateTablet for tablet 842164261a2348e7975904f808d4a128 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=0577d6e52a574ba48cc1152107510b08]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:52.877909 19654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 842164261a2348e7975904f808d4a128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.878160 19730 tablet_service.cc:1511] Processing CreateTablet for tablet 842164261a2348e7975904f808d4a128 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=0577d6e52a574ba48cc1152107510b08]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:52.879729 19730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 842164261a2348e7975904f808d4a128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.880931 19804 tablet_service.cc:1511] Processing CreateTablet for tablet 842164261a2348e7975904f808d4a128 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=0577d6e52a574ba48cc1152107510b08]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:52.881426 19655 tablet_service.cc:1511] Processing CreateTablet for tablet 359387f574b74510954bee5608679e12 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=0577d6e52a574ba48cc1152107510b08]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:52.882481 19804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 842164261a2348e7975904f808d4a128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.882763 19655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 359387f574b74510954bee5608679e12. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:52.895574 19611 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:52.899250 19854 tablet_bootstrap.cc:492] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081: Bootstrap starting.
I20260709 13:07:52.900328 19855 tablet_bootstrap.cc:492] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3: Bootstrap starting.
I20260709 13:07:52.904997 19729 tablet_service.cc:1511] Processing CreateTablet for tablet 359387f574b74510954bee5608679e12 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=0577d6e52a574ba48cc1152107510b08]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:52.906932 19729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 359387f574b74510954bee5608679e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.908910 19803 tablet_service.cc:1511] Processing CreateTablet for tablet 359387f574b74510954bee5608679e12 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=0577d6e52a574ba48cc1152107510b08]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:52.910490 19803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 359387f574b74510954bee5608679e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.912725 19854 tablet_bootstrap.cc:654] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.913435 19855 tablet_bootstrap.cc:654] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.924804 19856 tablet_bootstrap.cc:492] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281: Bootstrap starting.
I20260709 13:07:52.933679 19855 tablet_bootstrap.cc:492] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3: No bootstrap required, opened a new log
I20260709 13:07:52.934537 19855 ts_tablet_manager.cc:1403] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.004s
I20260709 13:07:52.936410 19856 tablet_bootstrap.cc:654] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.937881 19855 raft_consensus.cc:359] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.938582 19855 raft_consensus.cc:385] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.938899 19855 raft_consensus.cc:740] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ab9ab583da48de99064faf5a7b95f3, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.939154 19854 tablet_bootstrap.cc:492] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081: No bootstrap required, opened a new log
I20260709 13:07:52.939845 19854 ts_tablet_manager.cc:1403] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.000s
I20260709 13:07:52.939743 19855 consensus_queue.cc:260] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.942468 19855 ts_tablet_manager.cc:1434] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:07:52.943771 19855 tablet_bootstrap.cc:492] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3: Bootstrap starting.
I20260709 13:07:52.943943 19856 tablet_bootstrap.cc:492] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281: No bootstrap required, opened a new log
I20260709 13:07:52.944600 19856 ts_tablet_manager.cc:1403] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.000s
I20260709 13:07:52.946888 19854 raft_consensus.cc:359] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.947863 19854 raft_consensus.cc:385] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.948210 19854 raft_consensus.cc:740] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6a23bd84de411380327ea182db7081, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.948112 19856 raft_consensus.cc:359] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.949116 19856 raft_consensus.cc:385] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.949443 19856 raft_consensus.cc:740] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f7901d190834b2da3910b213a7a8281, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.949482 19854 consensus_queue.cc:260] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.950590 19856 consensus_queue.cc:260] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.952702 19854 ts_tablet_manager.cc:1434] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:07:52.953713 19856 ts_tablet_manager.cc:1434] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 13:07:52.955096 19856 tablet_bootstrap.cc:492] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281: Bootstrap starting.
I20260709 13:07:52.955288 19854 tablet_bootstrap.cc:492] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081: Bootstrap starting.
I20260709 13:07:52.958773 19855 tablet_bootstrap.cc:654] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.962904 19856 tablet_bootstrap.cc:654] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.963212 19854 tablet_bootstrap.cc:654] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.975741 19854 tablet_bootstrap.cc:492] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081: No bootstrap required, opened a new log
I20260709 13:07:52.976612 19854 ts_tablet_manager.cc:1403] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.002s
I20260709 13:07:52.983857 19854 raft_consensus.cc:359] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.985191 19854 raft_consensus.cc:385] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.985649 19854 raft_consensus.cc:740] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6a23bd84de411380327ea182db7081, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.987780 19854 consensus_queue.cc:260] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.990294 19856 tablet_bootstrap.cc:492] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281: No bootstrap required, opened a new log
I20260709 13:07:52.990674 19854 ts_tablet_manager.cc:1434] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 13:07:52.991096 19856 ts_tablet_manager.cc:1403] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.020s
I20260709 13:07:52.995003 19856 raft_consensus.cc:359] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.995939 19856 raft_consensus.cc:385] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.996297 19856 raft_consensus.cc:740] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f7901d190834b2da3910b213a7a8281, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.997257 19856 consensus_queue.cc:260] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:52.998485 19855 tablet_bootstrap.cc:492] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3: No bootstrap required, opened a new log
I20260709 13:07:52.999191 19855 ts_tablet_manager.cc:1403] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3: Time spent bootstrapping tablet: real 0.056s	user 0.033s	sys 0.010s
I20260709 13:07:53.002081 19856 ts_tablet_manager.cc:1434] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.005s
I20260709 13:07:53.003366 19855 raft_consensus.cc:359] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:53.004364 19855 raft_consensus.cc:385] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:53.004704 19855 raft_consensus.cc:740] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ab9ab583da48de99064faf5a7b95f3, State: Initialized, Role: FOLLOWER
I20260709 13:07:53.005662 19855 consensus_queue.cc:260] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:53.008380 19855 ts_tablet_manager.cc:1434] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:07:53.077324 19860 raft_consensus.cc:493] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.078140 19860 raft_consensus.cc:515] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:53.080345 19860 leader_election.cc:290] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe6a23bd84de411380327ea182db7081 (127.7.89.129:44905), 9f7901d190834b2da3910b213a7a8281 (127.7.89.131:46163)
I20260709 13:07:53.081478 19814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "842164261a2348e7975904f808d4a128" candidate_uuid: "53ab9ab583da48de99064faf5a7b95f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f7901d190834b2da3910b213a7a8281" is_pre_election: true
I20260709 13:07:53.081349 19664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "842164261a2348e7975904f808d4a128" candidate_uuid: "53ab9ab583da48de99064faf5a7b95f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6a23bd84de411380327ea182db7081" is_pre_election: true
I20260709 13:07:53.082419 19814 raft_consensus.cc:2468] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ab9ab583da48de99064faf5a7b95f3 in term 0.
I20260709 13:07:53.082450 19664 raft_consensus.cc:2468] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ab9ab583da48de99064faf5a7b95f3 in term 0.
I20260709 13:07:53.083873 19703 leader_election.cc:304] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ab9ab583da48de99064faf5a7b95f3, 9f7901d190834b2da3910b213a7a8281; no voters: 
I20260709 13:07:53.084751 19860 raft_consensus.cc:2804] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:53.085086 19860 raft_consensus.cc:493] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.085388 19860 raft_consensus.cc:3060] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.092509 19860 raft_consensus.cc:515] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:53.094470 19860 leader_election.cc:290] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [CANDIDATE]: Term 1 election: Requested vote from peers fe6a23bd84de411380327ea182db7081 (127.7.89.129:44905), 9f7901d190834b2da3910b213a7a8281 (127.7.89.131:46163)
I20260709 13:07:53.095543 19664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "842164261a2348e7975904f808d4a128" candidate_uuid: "53ab9ab583da48de99064faf5a7b95f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6a23bd84de411380327ea182db7081"
I20260709 13:07:53.095832 19814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "842164261a2348e7975904f808d4a128" candidate_uuid: "53ab9ab583da48de99064faf5a7b95f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f7901d190834b2da3910b213a7a8281"
I20260709 13:07:53.096362 19664 raft_consensus.cc:3060] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.096446 19814 raft_consensus.cc:3060] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.105284 19664 raft_consensus.cc:2468] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ab9ab583da48de99064faf5a7b95f3 in term 1.
I20260709 13:07:53.105284 19814 raft_consensus.cc:2468] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ab9ab583da48de99064faf5a7b95f3 in term 1.
I20260709 13:07:53.107014 19703 leader_election.cc:304] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ab9ab583da48de99064faf5a7b95f3, fe6a23bd84de411380327ea182db7081; no voters: 
I20260709 13:07:53.107786 19860 raft_consensus.cc:2804] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:53.108258 19860 raft_consensus.cc:697] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 LEADER]: Becoming Leader. State: Replica: 53ab9ab583da48de99064faf5a7b95f3, State: Running, Role: LEADER
I20260709 13:07:53.109135 19860 consensus_queue.cc:237] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:53.116988 19559 catalog_manager.cc:5697] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ab9ab583da48de99064faf5a7b95f3 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "53ab9ab583da48de99064faf5a7b95f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:53.247386 19862 consensus_queue.cc:1048] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.264267 19887 consensus_queue.cc:1048] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:53.277737 19860 consensus_queue.cc:1048] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.293812 19870 consensus_queue.cc:1048] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:53.323086 19870 raft_consensus.cc:493] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.323865 19870 raft_consensus.cc:515] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:53.326248 19870 leader_election.cc:290] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe6a23bd84de411380327ea182db7081 (127.7.89.129:44905), 9f7901d190834b2da3910b213a7a8281 (127.7.89.131:46163)
I20260709 13:07:53.327574 19814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "359387f574b74510954bee5608679e12" candidate_uuid: "53ab9ab583da48de99064faf5a7b95f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f7901d190834b2da3910b213a7a8281" is_pre_election: true
I20260709 13:07:53.327425 19664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "359387f574b74510954bee5608679e12" candidate_uuid: "53ab9ab583da48de99064faf5a7b95f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6a23bd84de411380327ea182db7081" is_pre_election: true
I20260709 13:07:53.328466 19814 raft_consensus.cc:2468] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ab9ab583da48de99064faf5a7b95f3 in term 0.
I20260709 13:07:53.328466 19664 raft_consensus.cc:2468] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ab9ab583da48de99064faf5a7b95f3 in term 0.
I20260709 13:07:53.329941 19703 leader_election.cc:304] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ab9ab583da48de99064faf5a7b95f3, fe6a23bd84de411380327ea182db7081; no voters: 
I20260709 13:07:53.330772 19870 raft_consensus.cc:2804] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:53.331130 19870 raft_consensus.cc:493] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.331398 19870 raft_consensus.cc:3060] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.339710 19870 raft_consensus.cc:515] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:53.342038 19870 leader_election.cc:290] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [CANDIDATE]: Term 1 election: Requested vote from peers fe6a23bd84de411380327ea182db7081 (127.7.89.129:44905), 9f7901d190834b2da3910b213a7a8281 (127.7.89.131:46163)
I20260709 13:07:53.343258 19814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "359387f574b74510954bee5608679e12" candidate_uuid: "53ab9ab583da48de99064faf5a7b95f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f7901d190834b2da3910b213a7a8281"
I20260709 13:07:53.343122 19664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "359387f574b74510954bee5608679e12" candidate_uuid: "53ab9ab583da48de99064faf5a7b95f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6a23bd84de411380327ea182db7081"
I20260709 13:07:53.344229 19814 raft_consensus.cc:3060] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.344310 19664 raft_consensus.cc:3060] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.351099 19814 raft_consensus.cc:2468] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ab9ab583da48de99064faf5a7b95f3 in term 1.
I20260709 13:07:53.351099 19664 raft_consensus.cc:2468] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ab9ab583da48de99064faf5a7b95f3 in term 1.
I20260709 13:07:53.352691 19703 leader_election.cc:304] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ab9ab583da48de99064faf5a7b95f3, 9f7901d190834b2da3910b213a7a8281; no voters: 
I20260709 13:07:53.353629 19870 raft_consensus.cc:2804] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:53.354322 19870 raft_consensus.cc:697] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 LEADER]: Becoming Leader. State: Replica: 53ab9ab583da48de99064faf5a7b95f3, State: Running, Role: LEADER
I20260709 13:07:53.355146 19870 consensus_queue.cc:237] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:53.363924 19559 catalog_manager.cc:5697] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ab9ab583da48de99064faf5a7b95f3 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "53ab9ab583da48de99064faf5a7b95f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:53.414460 19814 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ce825bbc71842a28b9746894f170c85"
dest_uuid: "9f7901d190834b2da3910b213a7a8281"
mode: GRACEFUL
new_leader_uuid: "fe6a23bd84de411380327ea182db7081"
 from {username='slave'} at 127.0.0.1:43728
I20260709 13:07:53.415238 19814 raft_consensus.cc:606] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 1 LEADER]: Received request to transfer leadership to TS fe6a23bd84de411380327ea182db7081
I20260709 13:07:53.435091 19740 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "842164261a2348e7975904f808d4a128"
dest_uuid: "53ab9ab583da48de99064faf5a7b95f3"
mode: GRACEFUL
new_leader_uuid: "fe6a23bd84de411380327ea182db7081"
 from {username='slave'} at 127.0.0.1:45324
I20260709 13:07:53.435722 19740 raft_consensus.cc:606] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 LEADER]: Received request to transfer leadership to TS fe6a23bd84de411380327ea182db7081
I20260709 13:07:53.441507 19740 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "359387f574b74510954bee5608679e12"
dest_uuid: "53ab9ab583da48de99064faf5a7b95f3"
mode: GRACEFUL
new_leader_uuid: "fe6a23bd84de411380327ea182db7081"
 from {username='slave'} at 127.0.0.1:45324
I20260709 13:07:53.442072 19740 raft_consensus.cc:606] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 LEADER]: Received request to transfer leadership to TS fe6a23bd84de411380327ea182db7081
I20260709 13:07:53.491606 19870 consensus_queue.cc:1048] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:53.500662 19860 raft_consensus.cc:993] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3: : Instructing follower fe6a23bd84de411380327ea182db7081 to start an election
I20260709 13:07:53.501317 19860 raft_consensus.cc:1081] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 LEADER]: Signalling peer fe6a23bd84de411380327ea182db7081 to start an election
I20260709 13:07:53.503774 19664 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "842164261a2348e7975904f808d4a128"
dest_uuid: "fe6a23bd84de411380327ea182db7081"
 from {username='slave'} at 127.0.0.1:55580
I20260709 13:07:53.504894 19664 raft_consensus.cc:493] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:53.505607 19664 raft_consensus.cc:3060] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.513094 19860 consensus_queue.cc:1048] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:53.515590 19664 raft_consensus.cc:515] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:53.518569 19665 raft_consensus.cc:1240] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 2 FOLLOWER]: Rejecting Update request from peer 53ab9ab583da48de99064faf5a7b95f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:53.523972 19870 consensus_queue.cc:1059] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:53.525271 19892 raft_consensus.cc:3055] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:53.525735 19892 raft_consensus.cc:740] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53ab9ab583da48de99064faf5a7b95f3, State: Running, Role: LEADER
I20260709 13:07:53.527174 19892 consensus_queue.cc:260] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:53.528685 19892 raft_consensus.cc:3060] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.530483 19664 leader_election.cc:290] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [CANDIDATE]: Term 2 election: Requested vote from peers 53ab9ab583da48de99064faf5a7b95f3 (127.7.89.130:44805), 9f7901d190834b2da3910b213a7a8281 (127.7.89.131:46163)
I20260709 13:07:53.535315 19740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "842164261a2348e7975904f808d4a128" candidate_uuid: "fe6a23bd84de411380327ea182db7081" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53ab9ab583da48de99064faf5a7b95f3"
I20260709 13:07:53.551206 19814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "842164261a2348e7975904f808d4a128" candidate_uuid: "fe6a23bd84de411380327ea182db7081" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f7901d190834b2da3910b213a7a8281"
I20260709 13:07:53.552079 19814 raft_consensus.cc:3060] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.552201 19740 raft_consensus.cc:2468] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe6a23bd84de411380327ea182db7081 in term 2.
I20260709 13:07:53.555488 19629 leader_election.cc:304] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ab9ab583da48de99064faf5a7b95f3, fe6a23bd84de411380327ea182db7081; no voters: 
I20260709 13:07:53.556638 19859 raft_consensus.cc:2804] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:53.559126 19859 raft_consensus.cc:697] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 2 LEADER]: Becoming Leader. State: Replica: fe6a23bd84de411380327ea182db7081, State: Running, Role: LEADER
I20260709 13:07:53.560140 19859 consensus_queue.cc:237] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:53.560683 19814 raft_consensus.cc:2468] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe6a23bd84de411380327ea182db7081 in term 2.
I20260709 13:07:53.571996 19560 catalog_manager.cc:5697] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 reported cstate change: term changed from 1 to 2, leader changed from 53ab9ab583da48de99064faf5a7b95f3 (127.7.89.130) to fe6a23bd84de411380327ea182db7081 (127.7.89.129). New cstate: current_term: 2 leader_uuid: "fe6a23bd84de411380327ea182db7081" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:53.826718 19892 consensus_queue.cc:1048] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:53.835557 19860 raft_consensus.cc:993] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3: : Instructing follower fe6a23bd84de411380327ea182db7081 to start an election
I20260709 13:07:53.836427 19860 raft_consensus.cc:1081] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 LEADER]: Signalling peer fe6a23bd84de411380327ea182db7081 to start an election
I20260709 13:07:53.841435 19664 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "359387f574b74510954bee5608679e12"
dest_uuid: "fe6a23bd84de411380327ea182db7081"
 from {username='slave'} at 127.0.0.1:55580
I20260709 13:07:53.842247 19664 raft_consensus.cc:493] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:53.842717 19664 raft_consensus.cc:3060] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.844208 19870 consensus_queue.cc:1048] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:53.856858 19887 raft_consensus.cc:993] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281: : Instructing follower fe6a23bd84de411380327ea182db7081 to start an election
I20260709 13:07:53.857383 19887 raft_consensus.cc:1081] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 1 LEADER]: Signalling peer fe6a23bd84de411380327ea182db7081 to start an election
I20260709 13:07:53.858071 19664 raft_consensus.cc:515] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:53.859449 19665 raft_consensus.cc:1240] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 2 FOLLOWER]: Rejecting Update request from peer 53ab9ab583da48de99064faf5a7b95f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:53.860404 19663 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ce825bbc71842a28b9746894f170c85"
dest_uuid: "fe6a23bd84de411380327ea182db7081"
 from {username='slave'} at 127.0.0.1:55576
I20260709 13:07:53.861213 19663 raft_consensus.cc:493] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:53.861603 19663 raft_consensus.cc:3060] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.861954 19740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "359387f574b74510954bee5608679e12" candidate_uuid: "fe6a23bd84de411380327ea182db7081" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53ab9ab583da48de99064faf5a7b95f3"
I20260709 13:07:53.862394 19814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "359387f574b74510954bee5608679e12" candidate_uuid: "fe6a23bd84de411380327ea182db7081" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f7901d190834b2da3910b213a7a8281"
I20260709 13:07:53.862855 19740 raft_consensus.cc:3055] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:53.863317 19740 raft_consensus.cc:740] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53ab9ab583da48de99064faf5a7b95f3, State: Running, Role: LEADER
I20260709 13:07:53.863487 19860 consensus_queue.cc:1059] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:53.863759 19664 leader_election.cc:290] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [CANDIDATE]: Term 2 election: Requested vote from peers 53ab9ab583da48de99064faf5a7b95f3 (127.7.89.130:44805), 9f7901d190834b2da3910b213a7a8281 (127.7.89.131:46163)
I20260709 13:07:53.863365 19814 raft_consensus.cc:3060] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.864991 19740 consensus_queue.cc:260] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:53.866323 19740 raft_consensus.cc:3060] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.870950 19663 raft_consensus.cc:515] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } }
I20260709 13:07:53.874449 19813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ce825bbc71842a28b9746894f170c85" candidate_uuid: "fe6a23bd84de411380327ea182db7081" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f7901d190834b2da3910b213a7a8281"
I20260709 13:07:53.874279 19663 leader_election.cc:290] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [CANDIDATE]: Term 2 election: Requested vote from peers 9f7901d190834b2da3910b213a7a8281 (127.7.89.131:46163), 53ab9ab583da48de99064faf5a7b95f3 (127.7.89.130:44805)
I20260709 13:07:53.874737 19739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ce825bbc71842a28b9746894f170c85" candidate_uuid: "fe6a23bd84de411380327ea182db7081" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53ab9ab583da48de99064faf5a7b95f3"
I20260709 13:07:53.875226 19740 raft_consensus.cc:2468] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe6a23bd84de411380327ea182db7081 in term 2.
I20260709 13:07:53.875713 19813 raft_consensus.cc:3055] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:53.875831 19739 raft_consensus.cc:3060] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.876215 19813 raft_consensus.cc:740] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9f7901d190834b2da3910b213a7a8281, State: Running, Role: LEADER
I20260709 13:07:53.877061 19629 leader_election.cc:304] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ab9ab583da48de99064faf5a7b95f3, fe6a23bd84de411380327ea182db7081; no voters: 
W20260709 13:07:53.876204 19906 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:07:53.873915 19814 raft_consensus.cc:2468] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe6a23bd84de411380327ea182db7081 in term 2.
I20260709 13:07:53.878360 19859 raft_consensus.cc:2804] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:53.879005 19859 raft_consensus.cc:697] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 2 LEADER]: Becoming Leader. State: Replica: fe6a23bd84de411380327ea182db7081, State: Running, Role: LEADER
I20260709 13:07:53.877295 19813 consensus_queue.cc:260] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } }
I20260709 13:07:53.881860 19813 raft_consensus.cc:3060] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:53.880092 19859 consensus_queue.cc:237] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:53.886097 19739 raft_consensus.cc:2468] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe6a23bd84de411380327ea182db7081 in term 2.
I20260709 13:07:53.887773 19629 leader_election.cc:304] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ab9ab583da48de99064faf5a7b95f3, fe6a23bd84de411380327ea182db7081; no voters: 
I20260709 13:07:53.890846 19902 raft_consensus.cc:2804] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:53.891776 19559 catalog_manager.cc:5697] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 reported cstate change: term changed from 1 to 2, leader changed from 53ab9ab583da48de99064faf5a7b95f3 (127.7.89.130) to fe6a23bd84de411380327ea182db7081 (127.7.89.129). New cstate: current_term: 2 leader_uuid: "fe6a23bd84de411380327ea182db7081" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:53.895452 19813 raft_consensus.cc:2468] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe6a23bd84de411380327ea182db7081 in term 2.
I20260709 13:07:53.900131 19902 raft_consensus.cc:697] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [term 2 LEADER]: Becoming Leader. State: Replica: fe6a23bd84de411380327ea182db7081, State: Running, Role: LEADER
I20260709 13:07:53.901399 19902 consensus_queue.cc:237] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } }
I20260709 13:07:53.911983 19559 catalog_manager.cc:5697] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 reported cstate change: term changed from 1 to 2, leader changed from 9f7901d190834b2da3910b213a7a8281 (127.7.89.131) to fe6a23bd84de411380327ea182db7081 (127.7.89.129). New cstate: current_term: 2 leader_uuid: "fe6a23bd84de411380327ea182db7081" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:53.949458  7526 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:07:53.951442  7526 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:07:53.951946  7526 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:07:53.968073 19663 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "842164261a2348e7975904f808d4a128"
dest_uuid: "fe6a23bd84de411380327ea182db7081"
mode: GRACEFUL
new_leader_uuid: "9f7901d190834b2da3910b213a7a8281"
 from {username='slave'} at 127.0.0.1:55590
I20260709 13:07:53.968773 19663 raft_consensus.cc:606] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 2 LEADER]: Received request to transfer leadership to TS 9f7901d190834b2da3910b213a7a8281
I20260709 13:07:53.970395  7526 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:07:54.028578 19814 raft_consensus.cc:1275] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 2 FOLLOWER]: Refusing update from remote peer fe6a23bd84de411380327ea182db7081: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:54.030122 19902 consensus_queue.cc:1048] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:54.036211 19908 raft_consensus.cc:993] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081: : Instructing follower 9f7901d190834b2da3910b213a7a8281 to start an election
I20260709 13:07:54.036830 19859 raft_consensus.cc:1081] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 2 LEADER]: Signalling peer 9f7901d190834b2da3910b213a7a8281 to start an election
I20260709 13:07:54.038960 19813 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "842164261a2348e7975904f808d4a128"
dest_uuid: "9f7901d190834b2da3910b213a7a8281"
 from {username='slave'} at 127.0.0.1:43744
I20260709 13:07:54.042502 19813 raft_consensus.cc:493] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:54.043092 19813 raft_consensus.cc:3060] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:54.043536 19739 raft_consensus.cc:1275] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 2 FOLLOWER]: Refusing update from remote peer fe6a23bd84de411380327ea182db7081: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:54.047624 19859 consensus_queue.cc:1048] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:54.056679 19813 raft_consensus.cc:515] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:54.065059 19813 leader_election.cc:290] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [CANDIDATE]: Term 3 election: Requested vote from peers fe6a23bd84de411380327ea182db7081 (127.7.89.129:44905), 53ab9ab583da48de99064faf5a7b95f3 (127.7.89.130:44805)
I20260709 13:07:54.065964 19663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "842164261a2348e7975904f808d4a128" candidate_uuid: "9f7901d190834b2da3910b213a7a8281" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fe6a23bd84de411380327ea182db7081"
I20260709 13:07:54.066169 19739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "842164261a2348e7975904f808d4a128" candidate_uuid: "9f7901d190834b2da3910b213a7a8281" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "53ab9ab583da48de99064faf5a7b95f3"
I20260709 13:07:54.066805 19663 raft_consensus.cc:3055] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:54.066884 19739 raft_consensus.cc:3060] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:54.067250 19663 raft_consensus.cc:740] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fe6a23bd84de411380327ea182db7081, State: Running, Role: LEADER
I20260709 13:07:54.068243 19663 consensus_queue.cc:260] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:54.069849 19663 raft_consensus.cc:3060] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:54.074703 19739 raft_consensus.cc:2468] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f7901d190834b2da3910b213a7a8281 in term 3.
I20260709 13:07:54.076480 19778 leader_election.cc:304] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ab9ab583da48de99064faf5a7b95f3, 9f7901d190834b2da3910b213a7a8281; no voters: 
I20260709 13:07:54.077006 19663 raft_consensus.cc:2468] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f7901d190834b2da3910b213a7a8281 in term 3.
I20260709 13:07:54.077565 19887 raft_consensus.cc:2804] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:54.078187 19887 raft_consensus.cc:697] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 3 LEADER]: Becoming Leader. State: Replica: 9f7901d190834b2da3910b213a7a8281, State: Running, Role: LEADER
I20260709 13:07:54.079265 19887 consensus_queue.cc:237] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } }
I20260709 13:07:54.088080 19560 catalog_manager.cc:5697] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 reported cstate change: term changed from 2 to 3, leader changed from fe6a23bd84de411380327ea182db7081 (127.7.89.129) to 9f7901d190834b2da3910b213a7a8281 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "9f7901d190834b2da3910b213a7a8281" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe6a23bd84de411380327ea182db7081" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 44905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ab9ab583da48de99064faf5a7b95f3" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f7901d190834b2da3910b213a7a8281" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46163 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:54.137148  7526 tablet_server.cc:179] TabletServer@127.7.89.129:0 shutting down...
I20260709 13:07:54.163074  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:54.164112  7526 tablet_replica.cc:333] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081: stopping tablet replica
I20260709 13:07:54.164886  7526 raft_consensus.cc:2243] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:54.165740  7526 pending_rounds.cc:70] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081: Trying to abort 2 pending ops.
I20260709 13:07:54.165978  7526 pending_rounds.cc:77] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305635733031219200 op_type: NO_OP noop_request { }
I20260709 13:07:54.166301  7526 raft_consensus.cc:2889] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:54.166574  7526 pending_rounds.cc:77] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635730870480896 op_type: NO_OP noop_request { }
I20260709 13:07:54.166898  7526 raft_consensus.cc:2889] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:54.167235  7526 raft_consensus.cc:2272] T 359387f574b74510954bee5608679e12 P fe6a23bd84de411380327ea182db7081 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.170100  7526 tablet_replica.cc:333] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081: stopping tablet replica
I20260709 13:07:54.170925  7526 raft_consensus.cc:2243] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:54.171495  7526 raft_consensus.cc:2272] T 842164261a2348e7975904f808d4a128 P fe6a23bd84de411380327ea182db7081 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.174350  7526 tablet_replica.cc:333] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081: stopping tablet replica
I20260709 13:07:54.175153  7526 raft_consensus.cc:2243] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:54.175966  7526 pending_rounds.cc:70] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081: Trying to abort 1 pending ops.
I20260709 13:07:54.176141  7526 pending_rounds.cc:77] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305635733111717888 op_type: NO_OP noop_request { }
I20260709 13:07:54.176512  7526 raft_consensus.cc:2889] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:54.176808  7526 raft_consensus.cc:2272] T 7ce825bbc71842a28b9746894f170c85 P fe6a23bd84de411380327ea182db7081 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.179811  7526 tablet_replica.cc:333] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081: stopping tablet replica
I20260709 13:07:54.180451  7526 raft_consensus.cc:2243] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:54.180990  7526 raft_consensus.cc:2272] T d15b991955494e1081504f24331e65ed P fe6a23bd84de411380327ea182db7081 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.207242  7526 tablet_server.cc:196] TabletServer@127.7.89.129:0 shutdown complete.
I20260709 13:07:54.223904  7526 tablet_server.cc:179] TabletServer@127.7.89.130:0 shutting down...
I20260709 13:07:54.248961  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:54.249814  7526 tablet_replica.cc:333] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3: stopping tablet replica
I20260709 13:07:54.250526  7526 raft_consensus.cc:2243] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:54.251183  7526 raft_consensus.cc:2272] T 359387f574b74510954bee5608679e12 P 53ab9ab583da48de99064faf5a7b95f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.253952  7526 tablet_replica.cc:333] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3: stopping tablet replica
I20260709 13:07:54.254715  7526 raft_consensus.cc:2243] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:54.255295  7526 raft_consensus.cc:2272] T 842164261a2348e7975904f808d4a128 P 53ab9ab583da48de99064faf5a7b95f3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.258165  7526 tablet_replica.cc:333] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3: stopping tablet replica
I20260709 13:07:54.259086  7526 raft_consensus.cc:2243] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:54.260190  7526 raft_consensus.cc:2272] T d15b991955494e1081504f24331e65ed P 53ab9ab583da48de99064faf5a7b95f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.262909  7526 tablet_replica.cc:333] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3: stopping tablet replica
I20260709 13:07:54.263628  7526 raft_consensus.cc:2243] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:54.264133  7526 raft_consensus.cc:2272] T 7ce825bbc71842a28b9746894f170c85 P 53ab9ab583da48de99064faf5a7b95f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.291146  7526 tablet_server.cc:196] TabletServer@127.7.89.130:0 shutdown complete.
I20260709 13:07:54.308738  7526 tablet_server.cc:179] TabletServer@127.7.89.131:0 shutting down...
I20260709 13:07:54.332338  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:54.333145  7526 tablet_replica.cc:333] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281: stopping tablet replica
I20260709 13:07:54.333927  7526 raft_consensus.cc:2243] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:54.334527  7526 raft_consensus.cc:2272] T 359387f574b74510954bee5608679e12 P 9f7901d190834b2da3910b213a7a8281 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.338277  7526 tablet_replica.cc:333] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281: stopping tablet replica
I20260709 13:07:54.339181  7526 raft_consensus.cc:2243] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:54.340086  7526 pending_rounds.cc:70] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281: Trying to abort 1 pending ops.
I20260709 13:07:54.340301  7526 pending_rounds.cc:77] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305635733840199680 op_type: NO_OP noop_request { }
I20260709 13:07:54.340633  7526 raft_consensus.cc:2889] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:54.340899  7526 raft_consensus.cc:2272] T 842164261a2348e7975904f808d4a128 P 9f7901d190834b2da3910b213a7a8281 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.343708  7526 tablet_replica.cc:333] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281: stopping tablet replica
I20260709 13:07:54.344429  7526 raft_consensus.cc:2243] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:54.344974  7526 raft_consensus.cc:2272] T d15b991955494e1081504f24331e65ed P 9f7901d190834b2da3910b213a7a8281 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.347395  7526 tablet_replica.cc:333] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281: stopping tablet replica
I20260709 13:07:54.347951  7526 raft_consensus.cc:2243] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:54.348479  7526 raft_consensus.cc:2272] T 7ce825bbc71842a28b9746894f170c85 P 9f7901d190834b2da3910b213a7a8281 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.375084  7526 tablet_server.cc:196] TabletServer@127.7.89.131:0 shutdown complete.
I20260709 13:07:54.391690  7526 master.cc:562] Master@127.7.89.190:34969 shutting down...
I20260709 13:07:54.411235  7526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:54.411939  7526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:54.412326  7526 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5d223354cef4409e8f6b63379c8cd7fb: stopping tablet replica
I20260709 13:07:54.434535  7526 master.cc:584] Master@127.7.89.190:34969 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2736 ms)
[----------] 6 tests from LeaderRebalancerTest (412287 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:07:54.468694  7526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.89.190:32917
I20260709 13:07:54.469998  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:54.476161 19913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:54.477293 19912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:54.478312  7526 server_base.cc:1061] running on GCE node
W20260709 13:07:54.478214 19915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:54.479561  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:54.479772  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:54.479928  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602474479911 us; error 0 us; skew 500 ppm
I20260709 13:07:54.480551  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:54.483174  7526 webserver.cc:533] Webserver started at http://127.7.89.190:46037/ using document root <none> and password file <none>
I20260709 13:07:54.483698  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:54.483889  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:54.484201  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:54.485414  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/master-0-root/instance:
uuid: "ecdbb3aebe6a4cdba0bf3720a8654c47"
format_stamp: "Formatted at 2026-07-09 13:07:54 on dist-test-slave-h0rz"
I20260709 13:07:54.490988  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:07:54.495326 19920 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:54.496296  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:07:54.496767  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/master-0-root
uuid: "ecdbb3aebe6a4cdba0bf3720a8654c47"
format_stamp: "Formatted at 2026-07-09 13:07:54 on dist-test-slave-h0rz"
I20260709 13:07:54.497148  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:54.510835  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:54.512282  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:54.558288  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.190:32917
I20260709 13:07:54.558401 19971 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.190:32917 every 8 connection(s)
I20260709 13:07:54.563359 19972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.578770 19972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47: Bootstrap starting.
I20260709 13:07:54.584259 19972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.589398 19972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47: No bootstrap required, opened a new log
I20260709 13:07:54.592049 19972 raft_consensus.cc:359] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdbb3aebe6a4cdba0bf3720a8654c47" member_type: VOTER }
I20260709 13:07:54.592530 19972 raft_consensus.cc:385] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:54.592821 19972 raft_consensus.cc:740] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecdbb3aebe6a4cdba0bf3720a8654c47, State: Initialized, Role: FOLLOWER
I20260709 13:07:54.593528 19972 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdbb3aebe6a4cdba0bf3720a8654c47" member_type: VOTER }
I20260709 13:07:54.594105 19972 raft_consensus.cc:399] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:54.594326 19972 raft_consensus.cc:493] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:54.594703 19972 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:54.601401 19972 raft_consensus.cc:515] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdbb3aebe6a4cdba0bf3720a8654c47" member_type: VOTER }
I20260709 13:07:54.602129 19972 leader_election.cc:304] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecdbb3aebe6a4cdba0bf3720a8654c47; no voters: 
I20260709 13:07:54.603634 19972 leader_election.cc:290] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:54.604022 19975 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:54.605568 19975 raft_consensus.cc:697] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [term 1 LEADER]: Becoming Leader. State: Replica: ecdbb3aebe6a4cdba0bf3720a8654c47, State: Running, Role: LEADER
I20260709 13:07:54.606317 19975 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdbb3aebe6a4cdba0bf3720a8654c47" member_type: VOTER }
I20260709 13:07:54.607471 19972 sys_catalog.cc:565] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:54.610872 19977 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecdbb3aebe6a4cdba0bf3720a8654c47. Latest consensus state: current_term: 1 leader_uuid: "ecdbb3aebe6a4cdba0bf3720a8654c47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdbb3aebe6a4cdba0bf3720a8654c47" member_type: VOTER } }
I20260709 13:07:54.610872 19976 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecdbb3aebe6a4cdba0bf3720a8654c47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdbb3aebe6a4cdba0bf3720a8654c47" member_type: VOTER } }
I20260709 13:07:54.611789 19977 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:54.611908 19976 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:54.619700 19981 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:54.624979 19981 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:54.626646  7526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:54.635102 19981 catalog_manager.cc:1383] Generated new cluster ID: 9ad7b8431f8a4185bba4201c9992316a
I20260709 13:07:54.635468 19981 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:54.668831 19981 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:54.670513 19981 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:54.687307 19981 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47: Generated new TSK 0
I20260709 13:07:54.688117 19981 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:54.694201  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:54.701313 19993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:54.702593 19994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:54.705001 19996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:54.705051  7526 server_base.cc:1061] running on GCE node
I20260709 13:07:54.706501  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:54.706797  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:54.706991  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602474706964 us; error 0 us; skew 500 ppm
I20260709 13:07:54.707590  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:54.710151  7526 webserver.cc:533] Webserver started at http://127.7.89.129:33457/ using document root <none> and password file <none>
I20260709 13:07:54.710716  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:54.710892  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:54.711138  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:54.712332  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-0-root/instance:
uuid: "e873e0e292554636a9f2813ef8ed92a3"
format_stamp: "Formatted at 2026-07-09 13:07:54 on dist-test-slave-h0rz"
I20260709 13:07:54.717449  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:07:54.721644 20001 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:54.722543  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:07:54.722955  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-0-root
uuid: "e873e0e292554636a9f2813ef8ed92a3"
format_stamp: "Formatted at 2026-07-09 13:07:54 on dist-test-slave-h0rz"
I20260709 13:07:54.723294  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:54.736129  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:54.737501  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:54.739207  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:54.741714  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:54.741971  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:54.742236  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:54.742417  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:54.788010  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.129:38539
I20260709 13:07:54.788120 20063 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.129:38539 every 8 connection(s)
I20260709 13:07:54.793918  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:54.801884 20068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:54.803118 20069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:54.805064 20071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:54.807253  7526 server_base.cc:1061] running on GCE node
I20260709 13:07:54.808343 20064 heartbeater.cc:344] Connected to a master server at 127.7.89.190:32917
I20260709 13:07:54.808630  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:07:54.808835 20064 heartbeater.cc:461] Registering TS with master...
W20260709 13:07:54.808957  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:54.809315  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602474809289 us; error 0 us; skew 500 ppm
I20260709 13:07:54.809914 20064 heartbeater.cc:507] Master 127.7.89.190:32917 requested a full tablet report, sending...
I20260709 13:07:54.810127  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:54.812685 19937 ts_manager.cc:194] Registered new tserver with Master: e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539)
I20260709 13:07:54.813282  7526 webserver.cc:533] Webserver started at http://127.7.89.130:38167/ using document root <none> and password file <none>
I20260709 13:07:54.813925  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:54.814162  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:54.814452  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:54.814985 19937 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52132
I20260709 13:07:54.815886  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-1-root/instance:
uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
format_stamp: "Formatted at 2026-07-09 13:07:54 on dist-test-slave-h0rz"
I20260709 13:07:54.821092  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:07:54.825670 20076 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:54.826563  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:07:54.827004  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-1-root
uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
format_stamp: "Formatted at 2026-07-09 13:07:54 on dist-test-slave-h0rz"
I20260709 13:07:54.827378  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:54.837767  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:54.839143  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:54.840812  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:54.843663  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:54.843930  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:54.844219  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:54.844410  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:54.890892  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.130:41545
I20260709 13:07:54.891023 20138 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.130:41545 every 8 connection(s)
I20260709 13:07:54.897142  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:54.907593 20143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:54.909165 20142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:54.913630  7526 server_base.cc:1061] running on GCE node
I20260709 13:07:54.913673 20139 heartbeater.cc:344] Connected to a master server at 127.7.89.190:32917
W20260709 13:07:54.914011 20145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:54.914570 20139 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:54.915351  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:54.915736  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:54.915580 20139 heartbeater.cc:507] Master 127.7.89.190:32917 requested a full tablet report, sending...
I20260709 13:07:54.916105  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602474916070 us; error 0 us; skew 500 ppm
I20260709 13:07:54.916885  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:54.918670 19937 ts_manager.cc:194] Registered new tserver with Master: 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:07:54.920364  7526 webserver.cc:533] Webserver started at http://127.7.89.131:44879/ using document root <none> and password file <none>
I20260709 13:07:54.920382 19937 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52138
I20260709 13:07:54.921131  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:54.921392  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:54.921720  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:54.923112  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-2-root/instance:
uuid: "aeccd99157ea41799d2e4c1592044899"
format_stamp: "Formatted at 2026-07-09 13:07:54 on dist-test-slave-h0rz"
I20260709 13:07:54.928332  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:07:54.932224 20150 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:54.933178  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:07:54.933531  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-2-root
uuid: "aeccd99157ea41799d2e4c1592044899"
format_stamp: "Formatted at 2026-07-09 13:07:54 on dist-test-slave-h0rz"
I20260709 13:07:54.933867  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602061986899-7526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:54.948302  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:54.949641  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:54.951364  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:54.954053  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:54.954320  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:54.954602  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:54.954856  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:07:55.003374  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.131:46661
I20260709 13:07:55.003525 20212 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.131:46661 every 8 connection(s)
I20260709 13:07:55.020017 20213 heartbeater.cc:344] Connected to a master server at 127.7.89.190:32917
I20260709 13:07:55.020609 20213 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:55.021489 20213 heartbeater.cc:507] Master 127.7.89.190:32917 requested a full tablet report, sending...
I20260709 13:07:55.024078 19937 ts_manager.cc:194] Registered new tserver with Master: aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:55.024252  7526 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016660746s
I20260709 13:07:55.025975 19937 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52150
I20260709 13:07:55.057397 19937 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52162:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:55.060460 19937 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:55.142812 20178 tablet_service.cc:1511] Processing CreateTablet for tablet 91501c5709394cedbd76b094e9d8044b (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:55.145026 20178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91501c5709394cedbd76b094e9d8044b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.152483 20177 tablet_service.cc:1511] Processing CreateTablet for tablet ba6459261f244c929626958cdd5124c8 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:55.155083 20177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba6459261f244c929626958cdd5124c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.165153 20176 tablet_service.cc:1511] Processing CreateTablet for tablet 6392bb50c7e748a9b4fd371a6e59853e (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:55.167085 20176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6392bb50c7e748a9b4fd371a6e59853e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.169880 20175 tablet_service.cc:1511] Processing CreateTablet for tablet a6e77f8f8ac4417ca238c2951bdf9ce5 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:55.171757 20175 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6e77f8f8ac4417ca238c2951bdf9ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.181391 20174 tablet_service.cc:1511] Processing CreateTablet for tablet 7d36135d0e0d425a83c860422de70367 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:55.183912 20103 tablet_service.cc:1511] Processing CreateTablet for tablet ba6459261f244c929626958cdd5124c8 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:55.184463 20100 tablet_service.cc:1511] Processing CreateTablet for tablet 7d36135d0e0d425a83c860422de70367 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:55.185761 20103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba6459261f244c929626958cdd5124c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.186175 20174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d36135d0e0d425a83c860422de70367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.187943 20102 tablet_service.cc:1511] Processing CreateTablet for tablet 6392bb50c7e748a9b4fd371a6e59853e (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:55.186899 20100 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d36135d0e0d425a83c860422de70367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.182204 20104 tablet_service.cc:1511] Processing CreateTablet for tablet 91501c5709394cedbd76b094e9d8044b (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:55.190267 20099 tablet_service.cc:1511] Processing CreateTablet for tablet a96c1cd6fac4472492f90a5d96208c66 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:55.191566 20104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91501c5709394cedbd76b094e9d8044b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.190330 20101 tablet_service.cc:1511] Processing CreateTablet for tablet a6e77f8f8ac4417ca238c2951bdf9ce5 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:55.193234 20101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6e77f8f8ac4417ca238c2951bdf9ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.195086 20102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6392bb50c7e748a9b4fd371a6e59853e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.196403 20173 tablet_service.cc:1511] Processing CreateTablet for tablet a96c1cd6fac4472492f90a5d96208c66 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:55.198278 20173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a96c1cd6fac4472492f90a5d96208c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.199542 20099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a96c1cd6fac4472492f90a5d96208c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.210681 20098 tablet_service.cc:1511] Processing CreateTablet for tablet fdb29224078847b6b7344b8a4baada34 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:55.212486 20098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb29224078847b6b7344b8a4baada34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.213572 20029 tablet_service.cc:1511] Processing CreateTablet for tablet 91501c5709394cedbd76b094e9d8044b (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:55.215323 20029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91501c5709394cedbd76b094e9d8044b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.215881 20172 tablet_service.cc:1511] Processing CreateTablet for tablet fdb29224078847b6b7344b8a4baada34 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:55.217602 20172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb29224078847b6b7344b8a4baada34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.220021 20097 tablet_service.cc:1511] Processing CreateTablet for tablet 2647c183b98c49c2ae36c3441b258086 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:55.221813 20097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2647c183b98c49c2ae36c3441b258086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.223939 20171 tablet_service.cc:1511] Processing CreateTablet for tablet 2647c183b98c49c2ae36c3441b258086 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:55.227021 20028 tablet_service.cc:1511] Processing CreateTablet for tablet ba6459261f244c929626958cdd5124c8 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:55.227471 20096 tablet_service.cc:1511] Processing CreateTablet for tablet 21c499bdb1fe4703ad7bafda1314b22b (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:55.228819 20096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21c499bdb1fe4703ad7bafda1314b22b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.225971 20171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2647c183b98c49c2ae36c3441b258086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.230763 20027 tablet_service.cc:1511] Processing CreateTablet for tablet 6392bb50c7e748a9b4fd371a6e59853e (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:55.233439 20026 tablet_service.cc:1511] Processing CreateTablet for tablet a6e77f8f8ac4417ca238c2951bdf9ce5 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:55.228819 20028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba6459261f244c929626958cdd5124c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.234304 20170 tablet_service.cc:1511] Processing CreateTablet for tablet 21c499bdb1fe4703ad7bafda1314b22b (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:55.236070 20170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21c499bdb1fe4703ad7bafda1314b22b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.236358 20025 tablet_service.cc:1511] Processing CreateTablet for tablet 7d36135d0e0d425a83c860422de70367 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:55.237998 20025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d36135d0e0d425a83c860422de70367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.240017 20024 tablet_service.cc:1511] Processing CreateTablet for tablet a96c1cd6fac4472492f90a5d96208c66 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:55.241739 20024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a96c1cd6fac4472492f90a5d96208c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.243235 20027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6392bb50c7e748a9b4fd371a6e59853e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.248410 20023 tablet_service.cc:1511] Processing CreateTablet for tablet fdb29224078847b6b7344b8a4baada34 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:55.248823 20022 tablet_service.cc:1511] Processing CreateTablet for tablet 2647c183b98c49c2ae36c3441b258086 (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:55.250782 20026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6e77f8f8ac4417ca238c2951bdf9ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.251540 20021 tablet_service.cc:1511] Processing CreateTablet for tablet 21c499bdb1fe4703ad7bafda1314b22b (DEFAULT_TABLE table=test-workload [id=a58c22b1bba14136b5c1b8c24b46d670]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:55.253135 20021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21c499bdb1fe4703ad7bafda1314b22b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.253903 20022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2647c183b98c49c2ae36c3441b258086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.259125 20023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdb29224078847b6b7344b8a4baada34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.272012 20228 tablet_bootstrap.cc:492] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:07:55.346029 20228 tablet_bootstrap.cc:654] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.383894 20229 tablet_bootstrap.cc:492] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:07:55.396687 20229 tablet_bootstrap.cc:654] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.409063 20229 tablet_bootstrap.cc:492] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:07:55.409773 20229 ts_tablet_manager.cc:1403] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.009s
I20260709 13:07:55.413074 20229 raft_consensus.cc:359] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.413946 20229 raft_consensus.cc:385] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.414294 20229 raft_consensus.cc:740] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.416185 20229 consensus_queue.cc:260] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.420130 20228 tablet_bootstrap.cc:492] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:07:55.420819 20228 ts_tablet_manager.cc:1403] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.149s	user 0.009s	sys 0.004s
I20260709 13:07:55.424058 20230 tablet_bootstrap.cc:492] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:07:55.424242 20228 raft_consensus.cc:359] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } } peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } }
I20260709 13:07:55.425375 20228 raft_consensus.cc:385] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.425719 20228 raft_consensus.cc:740] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.426371 20064 heartbeater.cc:499] Master 127.7.89.190:32917 was elected leader, sending a full tablet report...
I20260709 13:07:55.426756 20229 ts_tablet_manager.cc:1434] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:07:55.427538 20228 consensus_queue.cc:260] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } } peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } }
I20260709 13:07:55.428148 20229 tablet_bootstrap.cc:492] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:07:55.433440 20230 tablet_bootstrap.cc:654] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.440119 20229 tablet_bootstrap.cc:654] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.446602 20230 tablet_bootstrap.cc:492] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:07:55.447351 20230 ts_tablet_manager.cc:1403] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.004s
I20260709 13:07:55.450709 20230 raft_consensus.cc:359] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.451582 20230 raft_consensus.cc:385] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.451872 20230 raft_consensus.cc:740] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.452991 20230 consensus_queue.cc:260] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.462075 20139 heartbeater.cc:499] Master 127.7.89.190:32917 was elected leader, sending a full tablet report...
I20260709 13:07:55.463230 20230 ts_tablet_manager.cc:1434] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.016s	user 0.011s	sys 0.004s
I20260709 13:07:55.464589 20230 tablet_bootstrap.cc:492] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:07:55.471530 20230 tablet_bootstrap.cc:654] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.484277 20230 tablet_bootstrap.cc:492] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:07:55.484961 20230 ts_tablet_manager.cc:1403] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 13:07:55.485602 20228 ts_tablet_manager.cc:1434] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.064s	user 0.025s	sys 0.039s
I20260709 13:07:55.485630 20213 heartbeater.cc:499] Master 127.7.89.190:32917 was elected leader, sending a full tablet report...
I20260709 13:07:55.487093 20228 tablet_bootstrap.cc:492] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:07:55.488332 20230 raft_consensus.cc:359] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.489210 20230 raft_consensus.cc:385] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.489512 20230 raft_consensus.cc:740] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.490334 20230 consensus_queue.cc:260] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.493144 20230 ts_tablet_manager.cc:1434] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:55.494318 20230 tablet_bootstrap.cc:492] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:07:55.494395 20228 tablet_bootstrap.cc:654] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.485935 20229 tablet_bootstrap.cc:492] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:07:55.496413 20229 ts_tablet_manager.cc:1403] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.069s	user 0.025s	sys 0.030s
I20260709 13:07:55.500072 20229 raft_consensus.cc:359] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.501039 20229 raft_consensus.cc:385] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.501358 20229 raft_consensus.cc:740] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.502187 20230 tablet_bootstrap.cc:654] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.502267 20229 consensus_queue.cc:260] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.507530 20229 ts_tablet_manager.cc:1434] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:07:55.512130 20229 tablet_bootstrap.cc:492] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:07:55.520285 20228 tablet_bootstrap.cc:492] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:07:55.521129 20228 ts_tablet_manager.cc:1403] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.034s	user 0.006s	sys 0.021s
I20260709 13:07:55.521729 20229 tablet_bootstrap.cc:654] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.524463 20230 tablet_bootstrap.cc:492] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:07:55.525156 20230 ts_tablet_manager.cc:1403] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.011s
I20260709 13:07:55.528118 20228 raft_consensus.cc:359] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.528321 20230 raft_consensus.cc:359] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.529157 20228 raft_consensus.cc:385] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.529490 20230 raft_consensus.cc:385] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.529534 20228 raft_consensus.cc:740] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.530081 20230 raft_consensus.cc:740] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.530798 20228 consensus_queue.cc:260] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.531073 20230 consensus_queue.cc:260] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.533444 20228 ts_tablet_manager.cc:1434] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:07:55.533578 20230 ts_tablet_manager.cc:1434] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:07:55.534783 20228 tablet_bootstrap.cc:492] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:07:55.534804 20230 tablet_bootstrap.cc:492] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:07:55.537467 20229 tablet_bootstrap.cc:492] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:07:55.538214 20229 ts_tablet_manager.cc:1403] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.006s
I20260709 13:07:55.541776 20229 raft_consensus.cc:359] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.542943 20229 raft_consensus.cc:385] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.543367 20229 raft_consensus.cc:740] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.544445 20229 consensus_queue.cc:260] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.546902 20233 raft_consensus.cc:493] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.547753 20229 ts_tablet_manager.cc:1434] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:07:55.547487 20233 raft_consensus.cc:515] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.549295 20229 tablet_bootstrap.cc:492] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:07:55.550139 20228 tablet_bootstrap.cc:654] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.542474 20230 tablet_bootstrap.cc:654] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.558357 20229 tablet_bootstrap.cc:654] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.561040 20233 leader_election.cc:290] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:55.572882 20230 tablet_bootstrap.cc:492] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:07:55.573653 20230 ts_tablet_manager.cc:1403] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.039s	user 0.008s	sys 0.006s
I20260709 13:07:55.577198 20230 raft_consensus.cc:359] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.578048 20230 raft_consensus.cc:385] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.578325 20230 raft_consensus.cc:740] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.579701 20230 consensus_queue.cc:260] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.582166 20230 ts_tablet_manager.cc:1434] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:07:55.583424 20230 tablet_bootstrap.cc:492] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:07:55.590087 20230 tablet_bootstrap.cc:654] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.606069 20230 tablet_bootstrap.cc:492] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:07:55.607136 20230 ts_tablet_manager.cc:1403] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.008s
I20260709 13:07:55.612397 20230 raft_consensus.cc:359] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.613770 20230 raft_consensus.cc:385] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.614331 20230 raft_consensus.cc:740] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.615693 20230 consensus_queue.cc:260] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.617489 20233 raft_consensus.cc:493] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.618408 20233 raft_consensus.cc:515] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
W20260709 13:07:55.620074 19985 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:55.621636 20233 leader_election.cc:290] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:55.622153 20230 ts_tablet_manager.cc:1434] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.001s
I20260709 13:07:55.624166 20230 tablet_bootstrap.cc:492] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:07:55.639925 20230 tablet_bootstrap.cc:654] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.653673 20230 tablet_bootstrap.cc:492] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:07:55.654474 20230 ts_tablet_manager.cc:1403] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.009s
I20260709 13:07:55.656476 20228 tablet_bootstrap.cc:492] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:07:55.657186 20228 ts_tablet_manager.cc:1403] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.123s	user 0.050s	sys 0.046s
I20260709 13:07:55.658293 20230 raft_consensus.cc:359] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.659317 20230 raft_consensus.cc:385] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.659693 20230 raft_consensus.cc:740] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.660194 20229 tablet_bootstrap.cc:492] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:07:55.660952 20229 ts_tablet_manager.cc:1403] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.112s	user 0.033s	sys 0.056s
I20260709 13:07:55.661880 20233 raft_consensus.cc:493] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.662473 20233 raft_consensus.cc:515] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.664988 20233 leader_election.cc:290] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:55.665598 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d36135d0e0d425a83c860422de70367" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:07:55.660856 20230 consensus_queue.cc:260] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.666236 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96c1cd6fac4472492f90a5d96208c66" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:07:55.666699 20114 raft_consensus.cc:2468] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 0.
I20260709 13:07:55.666977 20113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96c1cd6fac4472492f90a5d96208c66" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:07:55.665642 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d36135d0e0d425a83c860422de70367" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
W20260709 13:07:55.668576 20003 leader_election.cc:343] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.668744 20005 leader_election.cc:304] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:07:55.669278 20113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb29224078847b6b7344b8a4baada34" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:07:55.665153 20229 raft_consensus.cc:359] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
W20260709 13:07:55.669893 20005 leader_election.cc:343] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.670256 20233 raft_consensus.cc:2804] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:55.669991 20113 raft_consensus.cc:2468] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 0.
I20260709 13:07:55.670672 20005 leader_election.cc:304] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e873e0e292554636a9f2813ef8ed92a3; no voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899
I20260709 13:07:55.671191 20233 raft_consensus.cc:493] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.669476 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb29224078847b6b7344b8a4baada34" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:07:55.671726 20233 raft_consensus.cc:3060] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:55.669528 20003 leader_election.cc:343] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:55.673662 20003 leader_election.cc:343] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.673309 20228 raft_consensus.cc:359] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.670076 20229 raft_consensus.cc:385] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.674829 20229 raft_consensus.cc:740] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.674284 20228 raft_consensus.cc:385] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.675758 20249 raft_consensus.cc:2749] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:55.676601 20005 leader_election.cc:304] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: aeccd99157ea41799d2e4c1592044899
I20260709 13:07:55.675868 20228 raft_consensus.cc:740] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.676343 20229 consensus_queue.cc:260] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.677946 20228 consensus_queue.cc:260] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.679133 20230 ts_tablet_manager.cc:1434] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.003s
I20260709 13:07:55.681665 20228 ts_tablet_manager.cc:1434] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 13:07:55.678310 20249 raft_consensus.cc:2804] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:55.682224 20249 raft_consensus.cc:493] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.682763 20249 raft_consensus.cc:3060] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.683022 20228 tablet_bootstrap.cc:492] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:07:55.693903 20228 tablet_bootstrap.cc:654] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.699765 20233 raft_consensus.cc:515] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.701918 20229 ts_tablet_manager.cc:1434] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.040s	user 0.011s	sys 0.008s
I20260709 13:07:55.702834 20233 leader_election.cc:290] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Requested vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:55.703655 20230 tablet_bootstrap.cc:492] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:07:55.704262 20113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d36135d0e0d425a83c860422de70367" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:07:55.705235 20113 raft_consensus.cc:3060] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.706763 20249 raft_consensus.cc:515] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.709630 20249 leader_election.cc:290] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Requested vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:55.709451 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d36135d0e0d425a83c860422de70367" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:07:55.710582 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb29224078847b6b7344b8a4baada34" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:07:55.711374 20114 raft_consensus.cc:3060] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.712846 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb29224078847b6b7344b8a4baada34" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:07:55.713819 20230 tablet_bootstrap.cc:654] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.716640 20229 tablet_bootstrap.cc:492] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
W20260709 13:07:55.719311 20003 leader_election.cc:343] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.720161 20228 tablet_bootstrap.cc:492] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
W20260709 13:07:55.720202 20003 leader_election.cc:343] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.720912 20228 ts_tablet_manager.cc:1403] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.017s
I20260709 13:07:55.723176 20114 raft_consensus.cc:2468] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 1.
I20260709 13:07:55.725142 20005 leader_election.cc:304] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: aeccd99157ea41799d2e4c1592044899
I20260709 13:07:55.726878 20249 raft_consensus.cc:2804] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:55.727805 20249 raft_consensus.cc:697] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Becoming Leader. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:07:55.726787 20228 raft_consensus.cc:359] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.729115 20249 consensus_queue.cc:237] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.730118 20228 raft_consensus.cc:385] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.731061 20228 raft_consensus.cc:740] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.732405 20228 consensus_queue.cc:260] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.733855 20113 raft_consensus.cc:2468] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 1.
I20260709 13:07:55.737918 20228 ts_tablet_manager.cc:1434] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:07:55.740610 20005 leader_election.cc:304] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: aeccd99157ea41799d2e4c1592044899
I20260709 13:07:55.741602 20230 tablet_bootstrap.cc:492] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:07:55.742307 20250 raft_consensus.cc:2804] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:55.742548 20230 ts_tablet_manager.cc:1403] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.005s
I20260709 13:07:55.742997 20250 raft_consensus.cc:697] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Becoming Leader. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:07:55.744066 20250 consensus_queue.cc:237] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.740518 20229 tablet_bootstrap.cc:654] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.750272 20237 raft_consensus.cc:493] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.751118 20237 raft_consensus.cc:515] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.755411 20237 leader_election.cc:290] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:55.749393 20228 tablet_bootstrap.cc:492] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:07:55.747695 20230 raft_consensus.cc:359] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.763872 20230 raft_consensus.cc:385] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.764293 20230 raft_consensus.cc:740] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.764736 20228 tablet_bootstrap.cc:654] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.765269 20230 consensus_queue.cc:260] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.768246 20230 ts_tablet_manager.cc:1434] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:07:55.769547 19934 catalog_manager.cc:5697] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "e873e0e292554636a9f2813ef8ed92a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:55.769542 20230 tablet_bootstrap.cc:492] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:07:55.790170 20229 tablet_bootstrap.cc:492] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:07:55.791451 20229 ts_tablet_manager.cc:1403] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.075s	user 0.003s	sys 0.015s
I20260709 13:07:55.795390 20230 tablet_bootstrap.cc:654] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.797140 20229 raft_consensus.cc:359] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.801076 20229 raft_consensus.cc:385] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.801676 20229 raft_consensus.cc:740] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.802955 20229 consensus_queue.cc:260] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.807413 20229 ts_tablet_manager.cc:1434] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.005s
I20260709 13:07:55.809348 20229 tablet_bootstrap.cc:492] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:07:55.810786 20228 tablet_bootstrap.cc:492] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:07:55.811740 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba6459261f244c929626958cdd5124c8" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
W20260709 13:07:55.814472 20079 leader_election.cc:343] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.811839 20228 ts_tablet_manager.cc:1403] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.063s	user 0.019s	sys 0.005s
I20260709 13:07:55.820586 20229 tablet_bootstrap.cc:654] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.831983 20228 raft_consensus.cc:359] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.833442 20228 raft_consensus.cc:385] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.832000 20229 tablet_bootstrap.cc:492] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:07:55.833621 20230 tablet_bootstrap.cc:492] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:07:55.833514 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba6459261f244c929626958cdd5124c8" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:07:55.834388 20230 ts_tablet_manager.cc:1403] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.065s	user 0.009s	sys 0.026s
I20260709 13:07:55.834400 20229 ts_tablet_manager.cc:1403] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.025s	user 0.003s	sys 0.020s
I20260709 13:07:55.834450 20187 raft_consensus.cc:2468] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 0.
I20260709 13:07:55.834471 20228 raft_consensus.cc:740] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.836575 20228 consensus_queue.cc:260] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.838716 20078 leader_election.cc:304] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: e873e0e292554636a9f2813ef8ed92a3
I20260709 13:07:55.838584 20229 raft_consensus.cc:359] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.839481 20228 ts_tablet_manager.cc:1434] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.004s
I20260709 13:07:55.839821 20229 raft_consensus.cc:385] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.840204 20229 raft_consensus.cc:740] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.840282 20237 raft_consensus.cc:2804] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:55.840736 20228 tablet_bootstrap.cc:492] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:07:55.841226 20229 consensus_queue.cc:260] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.841369 20230 raft_consensus.cc:359] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.842257 20230 raft_consensus.cc:385] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.842562 20230 raft_consensus.cc:740] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.843492 20230 consensus_queue.cc:260] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.844506 20229 ts_tablet_manager.cc:1434] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:07:55.845960 20229 tablet_bootstrap.cc:492] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:07:55.849328 20228 tablet_bootstrap.cc:654] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.854144 20229 tablet_bootstrap.cc:654] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.855481 20234 raft_consensus.cc:493] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.856382 20234 raft_consensus.cc:515] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.840788 20237 raft_consensus.cc:493] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.859236 20237 raft_consensus.cc:3060] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.862406 20234 leader_election.cc:290] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:07:55.903998 20264 raft_consensus.cc:493] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.905128 20264 raft_consensus.cc:515] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.910028 20228 tablet_bootstrap.cc:492] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:07:55.910722 20228 ts_tablet_manager.cc:1403] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.070s	user 0.019s	sys 0.003s
I20260709 13:07:55.909891 19937 catalog_manager.cc:5697] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "e873e0e292554636a9f2813ef8ed92a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:55.912420 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2647c183b98c49c2ae36c3441b258086" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:07:55.913214 20039 raft_consensus.cc:2468] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 0.
I20260709 13:07:55.914065 20250 raft_consensus.cc:493] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.914647 20250 raft_consensus.cc:515] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.914793 20228 raft_consensus.cc:359] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.915175 20079 leader_election.cc:304] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:07:55.915606 20228 raft_consensus.cc:385] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.917348 20228 raft_consensus.cc:740] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.916980 20250 leader_election.cc:290] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:55.918008 20113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6392bb50c7e748a9b4fd371a6e59853e" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:07:55.918735 20113 raft_consensus.cc:2468] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 0.
I20260709 13:07:55.918244 20228 consensus_queue.cc:260] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.921397 20228 ts_tablet_manager.cc:1434] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:07:55.921036 20237 raft_consensus.cc:515] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.922566 20228 tablet_bootstrap.cc:492] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:07:55.923472 20230 ts_tablet_manager.cc:1434] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.089s	user 0.034s	sys 0.024s
I20260709 13:07:55.923887 20005 leader_election.cc:304] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:07:55.925361 20250 raft_consensus.cc:2804] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:55.925752 20250 raft_consensus.cc:493] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.926098 20250 raft_consensus.cc:3060] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.928213 20230 tablet_bootstrap.cc:492] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:07:55.931376 20271 raft_consensus.cc:2804] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:55.931984 20271 raft_consensus.cc:493] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.932353 20271 raft_consensus.cc:3060] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.935926 20234 raft_consensus.cc:493] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.936440 20228 tablet_bootstrap.cc:654] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.936597 20234 raft_consensus.cc:515] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } } peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } }
I20260709 13:07:55.939666 20234 leader_election.cc:290] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:07:55.941799 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6392bb50c7e748a9b4fd371a6e59853e" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:07:55.942857 20187 raft_consensus.cc:2468] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 0.
I20260709 13:07:55.948827 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2647c183b98c49c2ae36c3441b258086" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:07:55.941838 20270 raft_consensus.cc:493] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:55.950840 20078 leader_election.cc:343] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.951427 20270 raft_consensus.cc:515] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.952193 20264 leader_election.cc:290] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:55.955549 20230 tablet_bootstrap.cc:654] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.955719 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba6459261f244c929626958cdd5124c8" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
W20260709 13:07:55.958015 20079 leader_election.cc:343] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.961283 20250 raft_consensus.cc:515] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.962715 20229 tablet_bootstrap.cc:492] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:07:55.964669 20229 ts_tablet_manager.cc:1403] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.119s	user 0.033s	sys 0.043s
I20260709 13:07:55.966661 20250 leader_election.cc:290] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Requested vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:55.966908 20113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6392bb50c7e748a9b4fd371a6e59853e" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:07:55.963337 20271 raft_consensus.cc:515] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.967634 20113 raft_consensus.cc:3060] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.968619 20229 raft_consensus.cc:359] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.969383 20271 leader_election.cc:290] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:55.969663 20229 raft_consensus.cc:385] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.970427 20229 raft_consensus.cc:740] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.970948 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2647c183b98c49c2ae36c3441b258086" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:07:55.971662 20039 raft_consensus.cc:3060] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.971338 20229 consensus_queue.cc:260] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:55.973107 20270 leader_election.cc:290] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:55.976979 20229 ts_tablet_manager.cc:1434] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:07:55.976621 20038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c499bdb1fe4703ad7bafda1314b22b" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:07:55.978389 20229 tablet_bootstrap.cc:492] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:07:55.978286 20038 raft_consensus.cc:2468] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 0.
I20260709 13:07:55.980131 20079 leader_election.cc:304] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:07:55.981199 20270 raft_consensus.cc:2804] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:55.981685 20270 raft_consensus.cc:493] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.982092 20270 raft_consensus.cc:3060] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.995366 20229 tablet_bootstrap.cc:654] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.995652 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba6459261f244c929626958cdd5124c8" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:07:55.996352 20187 raft_consensus.cc:3060] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.004676 20186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c499bdb1fe4703ad7bafda1314b22b" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:07:56.005875 20186 raft_consensus.cc:2468] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 0.
I20260709 13:07:56.003926 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2647c183b98c49c2ae36c3441b258086" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:07:56.007938 20271 raft_consensus.cc:493] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:56.009016 20078 leader_election.cc:343] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:56.008795 20271 raft_consensus.cc:515] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.011214 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6392bb50c7e748a9b4fd371a6e59853e" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:07:56.012104 20188 raft_consensus.cc:3060] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.012413 20271 leader_election.cc:290] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:56.015216 20038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e77f8f8ac4417ca238c2951bdf9ce5" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:07:56.016388 20038 raft_consensus.cc:2468] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 0.
I20260709 13:07:56.018541 20079 leader_election.cc:304] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:07:56.020654 20271 raft_consensus.cc:2804] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.021211 20271 raft_consensus.cc:493] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.021703 20271 raft_consensus.cc:3060] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.034873 20113 raft_consensus.cc:2468] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 1.
I20260709 13:07:56.036561 20005 leader_election.cc:304] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:07:56.037796 20250 raft_consensus.cc:2804] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.037886 20237 leader_election.cc:290] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:56.038499 20250 raft_consensus.cc:697] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Becoming Leader. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:07:56.042937 20250 consensus_queue.cc:237] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.045323 20230 tablet_bootstrap.cc:492] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:07:56.045958 20230 ts_tablet_manager.cc:1403] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.118s	user 0.020s	sys 0.052s
I20260709 13:07:56.040725 20186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e77f8f8ac4417ca238c2951bdf9ce5" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:07:56.047940 20186 raft_consensus.cc:2468] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 0.
I20260709 13:07:56.048820 20230 raft_consensus.cc:359] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } } peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } }
I20260709 13:07:56.049711 20230 raft_consensus.cc:385] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.050015 20230 raft_consensus.cc:740] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.050879 20230 consensus_queue.cc:260] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } } peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } }
I20260709 13:07:56.053303 20230 ts_tablet_manager.cc:1434] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:07:56.041033 20271 raft_consensus.cc:515] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.068076 20113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c499bdb1fe4703ad7bafda1314b22b" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:07:56.068559 20228 tablet_bootstrap.cc:492] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:07:56.068724 20271 leader_election.cc:290] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:56.069329 20228 ts_tablet_manager.cc:1403] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.147s	user 0.033s	sys 0.033s
I20260709 13:07:56.069545 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91501c5709394cedbd76b094e9d8044b" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:07:56.069897 20038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e77f8f8ac4417ca238c2951bdf9ce5" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:07:56.070389 20114 raft_consensus.cc:2468] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 0.
I20260709 13:07:56.070498 20038 raft_consensus.cc:3060] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.073351 20186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e77f8f8ac4417ca238c2951bdf9ce5" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:07:56.073025 20228 raft_consensus.cc:359] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.074088 20186 raft_consensus.cc:3060] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.074527 20228 raft_consensus.cc:385] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.075011 20228 raft_consensus.cc:740] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.076402 20270 raft_consensus.cc:515] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.077068 20036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91501c5709394cedbd76b094e9d8044b" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:07:56.075873 20228 consensus_queue.cc:260] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.079604 20037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c499bdb1fe4703ad7bafda1314b22b" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:07:56.080572 20037 raft_consensus.cc:2468] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 0.
I20260709 13:07:56.081501 20039 raft_consensus.cc:2468] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 1.
I20260709 13:07:56.082523 20113 raft_consensus.cc:2393] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aeccd99157ea41799d2e4c1592044899 in current term 1: Already voted for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in this term.
I20260709 13:07:56.058108 20187 raft_consensus.cc:2468] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 1.
I20260709 13:07:56.085098 20186 raft_consensus.cc:2468] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 1.
I20260709 13:07:56.085613 20270 leader_election.cc:290] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:56.086114 20229 tablet_bootstrap.cc:492] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:07:56.086733 20229 ts_tablet_manager.cc:1403] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.109s	user 0.027s	sys 0.020s
I20260709 13:07:56.086484 20154 leader_election.cc:304] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: 
I20260709 13:07:56.087122 20228 ts_tablet_manager.cc:1434] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.001s
I20260709 13:07:56.087600 20234 raft_consensus.cc:2804] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:07:56.088047 20153 leader_election.cc:343] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:56.088548 20228 tablet_bootstrap.cc:492] T 2647c183b98c49c2ae36c3441b258086 P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:07:56.073980 20188 raft_consensus.cc:2468] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 1.
I20260709 13:07:56.089217 20153 leader_election.cc:304] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:07:56.089836 20186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c499bdb1fe4703ad7bafda1314b22b" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:07:56.086880 20079 leader_election.cc:304] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: aeccd99157ea41799d2e4c1592044899
I20260709 13:07:56.088053 20234 raft_consensus.cc:493] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.090989 20234 raft_consensus.cc:3060] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.091359 20270 raft_consensus.cc:2804] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.092868 20270 raft_consensus.cc:697] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Becoming Leader. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:07:56.093822 20270 consensus_queue.cc:237] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.097369 20228 tablet_bootstrap.cc:654] T 2647c183b98c49c2ae36c3441b258086 P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.097769 20186 raft_consensus.cc:3060] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.088052 20078 leader_election.cc:304] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: e873e0e292554636a9f2813ef8ed92a3
I20260709 13:07:56.100168 20078 leader_election.cc:304] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: 
I20260709 13:07:56.101238 20237 raft_consensus.cc:2804] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.101984 20237 raft_consensus.cc:697] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Becoming Leader. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:07:56.101399 19935 catalog_manager.cc:5697] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "e873e0e292554636a9f2813ef8ed92a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.115917 20186 raft_consensus.cc:2468] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 1.
I20260709 13:07:56.117071 20276 raft_consensus.cc:2764] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:07:56.103292 20237 consensus_queue.cc:237] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.116243 20038 raft_consensus.cc:2468] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 1.
I20260709 13:07:56.104152 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c499bdb1fe4703ad7bafda1314b22b" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:07:56.104669 20264 raft_consensus.cc:2804] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.126058 20264 raft_consensus.cc:697] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Becoming Leader. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:07:56.127861 20188 raft_consensus.cc:3060] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.127631 20264 consensus_queue.cc:237] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.135612 20078 leader_election.cc:304] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: 
I20260709 13:07:56.142975 20270 raft_consensus.cc:2804] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.144271 20270 raft_consensus.cc:697] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Becoming Leader. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:07:56.147001 20039 raft_consensus.cc:3060] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.148641 20276 raft_consensus.cc:493] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.149338 20276 raft_consensus.cc:515] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.152415 20250 consensus_queue.cc:1048] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.154640 20038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d36135d0e0d425a83c860422de70367" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:07:56.140872 19935 catalog_manager.cc:5697] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.159329 20113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d36135d0e0d425a83c860422de70367" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:07:56.158684 20276 leader_election.cc:290] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:07:56.160172 20113 raft_consensus.cc:2468] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 1.
I20260709 13:07:56.145681 20270 consensus_queue.cc:237] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.162268 20154 leader_election.cc:304] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: e873e0e292554636a9f2813ef8ed92a3
I20260709 13:07:56.165102 20276 raft_consensus.cc:2804] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:56.109853 20229 raft_consensus.cc:359] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } } peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } }
I20260709 13:07:56.166232 20229 raft_consensus.cc:385] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.166765 20276 raft_consensus.cc:493] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e873e0e292554636a9f2813ef8ed92a3)
I20260709 13:07:56.179010 20276 raft_consensus.cc:3060] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:56.179092 20233 consensus_queue.cc:1048] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.182757 20229 raft_consensus.cc:740] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.184052 20229 consensus_queue.cc:260] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } } peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } }
I20260709 13:07:56.173234 20234 raft_consensus.cc:515] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } } peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } }
I20260709 13:07:56.199599 20038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91501c5709394cedbd76b094e9d8044b" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:07:56.200368 20038 raft_consensus.cc:3060] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.177007 20249 consensus_queue.cc:1048] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.202013 20113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91501c5709394cedbd76b094e9d8044b" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:07:56.203183 20113 raft_consensus.cc:3060] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.210882 20229 ts_tablet_manager.cc:1434] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.124s	user 0.011s	sys 0.000s
I20260709 13:07:56.212812 20229 tablet_bootstrap.cc:492] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:07:56.213284 20188 raft_consensus.cc:3060] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.218554 20270 raft_consensus.cc:493] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.218899 20234 leader_election.cc:290] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:07:56.219487 20270 raft_consensus.cc:515] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.223181 20037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96c1cd6fac4472492f90a5d96208c66" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:07:56.224203 20037 raft_consensus.cc:2468] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 0.
I20260709 13:07:56.225776 20079 leader_election.cc:304] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:07:56.227010 20237 raft_consensus.cc:2804] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.227614 20237 raft_consensus.cc:493] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.228794 20237 raft_consensus.cc:3060] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.231366 20270 leader_election.cc:290] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:56.234592 20229 tablet_bootstrap.cc:654] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.240020 20038 raft_consensus.cc:2468] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 1.
I20260709 13:07:56.242020 20039 raft_consensus.cc:2468] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 1.
I20260709 13:07:56.244954 20153 leader_election.cc:304] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:07:56.246716 20234 raft_consensus.cc:2804] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.247575 20234 raft_consensus.cc:697] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Becoming Leader. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:07:56.248536 20228 tablet_bootstrap.cc:492] T 2647c183b98c49c2ae36c3441b258086 P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:07:56.248495 20234 consensus_queue.cc:237] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } } peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } }
I20260709 13:07:56.249194 20228 ts_tablet_manager.cc:1403] T 2647c183b98c49c2ae36c3441b258086 P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.161s	user 0.054s	sys 0.080s
I20260709 13:07:56.248605 20113 raft_consensus.cc:2468] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 1.
I20260709 13:07:56.251309 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96c1cd6fac4472492f90a5d96208c66" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:07:56.238010 20276 raft_consensus.cc:515] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.252607 20228 raft_consensus.cc:359] T 2647c183b98c49c2ae36c3441b258086 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.253430 20228 raft_consensus.cc:385] T 2647c183b98c49c2ae36c3441b258086 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.253796 20228 raft_consensus.cc:740] T 2647c183b98c49c2ae36c3441b258086 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.254673 20228 consensus_queue.cc:260] T 2647c183b98c49c2ae36c3441b258086 P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
W20260709 13:07:56.255630 20078 leader_election.cc:343] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:56.257536 20228 ts_tablet_manager.cc:1434] T 2647c183b98c49c2ae36c3441b258086 P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:56.259287 20228 tablet_bootstrap.cc:492] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:07:56.262014 20286 consensus_queue.cc:1048] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.263139 20186 raft_consensus.cc:1240] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Rejecting Update request from peer e873e0e292554636a9f2813ef8ed92a3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:56.267448 20233 consensus_queue.cc:1059] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.269332 20250 raft_consensus.cc:3055] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:56.269761 20250 raft_consensus.cc:740] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:07:56.269904 20237 raft_consensus.cc:515] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.270766 20250 consensus_queue.cc:260] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.271764 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d36135d0e0d425a83c860422de70367" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:07:56.271811 20250 raft_consensus.cc:3060] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:56.272369 20237 leader_election.cc:290] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:56.275311 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96c1cd6fac4472492f90a5d96208c66" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:07:56.275316 20038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96c1cd6fac4472492f90a5d96208c66" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:07:56.277789 20038 raft_consensus.cc:3060] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.291949 20276 leader_election.cc:290] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 2 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:07:56.293267 20228 tablet_bootstrap.cc:654] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:56.292985 20078 leader_election.cc:343] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:56.296239 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d36135d0e0d425a83c860422de70367" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:07:56.301676 20038 raft_consensus.cc:2468] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 1.
I20260709 13:07:56.304207 20079 leader_election.cc:304] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: aeccd99157ea41799d2e4c1592044899
I20260709 13:07:56.305373 20237 raft_consensus.cc:2804] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.305940 20237 raft_consensus.cc:697] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Becoming Leader. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:07:56.311096 20237 consensus_queue.cc:237] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.327838 19935 catalog_manager.cc:5697] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.337862 20039 raft_consensus.cc:2468] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 2.
I20260709 13:07:56.337478 19935 catalog_manager.cc:5697] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.341173 20153 leader_election.cc:304] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 939a6adbfc6b4d49adda1ebec89d8d6e
I20260709 13:07:56.340370 19935 catalog_manager.cc:5697] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.342291 20276 raft_consensus.cc:2804] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:56.343106 20276 raft_consensus.cc:697] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 2 LEADER]: Becoming Leader. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:07:56.344462 20276 consensus_queue.cc:237] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.358052 20229 tablet_bootstrap.cc:492] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:07:56.359004 20229 ts_tablet_manager.cc:1403] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.147s	user 0.026s	sys 0.012s
I20260709 13:07:56.361460 20228 tablet_bootstrap.cc:492] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:07:56.362203 20228 ts_tablet_manager.cc:1403] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.103s	user 0.011s	sys 0.004s
I20260709 13:07:56.363323 20229 raft_consensus.cc:359] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.364233 20229 raft_consensus.cc:385] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.364544 20229 raft_consensus.cc:740] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.365588 20228 raft_consensus.cc:359] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.365549 20229 consensus_queue.cc:260] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.366490 20228 raft_consensus.cc:385] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.366983 20228 raft_consensus.cc:740] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.367889 20228 consensus_queue.cc:260] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.370476 20228 ts_tablet_manager.cc:1434] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:56.372208 19934 catalog_manager.cc:5697] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 reported cstate change: term changed from 0 to 1, leader changed from <none> to aeccd99157ea41799d2e4c1592044899 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "aeccd99157ea41799d2e4c1592044899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.378741 20229 ts_tablet_manager.cc:1434] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.001s
I20260709 13:07:56.385316 19937 catalog_manager.cc:5697] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.396525 19935 catalog_manager.cc:5697] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 reported cstate change: term changed from 1 to 2, leader changed from e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129) to aeccd99157ea41799d2e4c1592044899 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "aeccd99157ea41799d2e4c1592044899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:56.429980 20250 consensus_queue.cc:1048] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.443043 20250 consensus_queue.cc:1048] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.503222 20039 raft_consensus.cc:3060] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.511854 20237 consensus_queue.cc:1048] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.515300 20187 raft_consensus.cc:3060] T 2647c183b98c49c2ae36c3441b258086 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.527554 20237 consensus_queue.cc:1048] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.529768 20271 consensus_queue.cc:1048] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.551121 20271 consensus_queue.cc:1048] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.559540 20271 consensus_queue.cc:1048] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.585963 20264 consensus_queue.cc:1048] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.600570 20187 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91501c5709394cedbd76b094e9d8044b"
dest_uuid: "aeccd99157ea41799d2e4c1592044899"
mode: GRACEFUL
new_leader_uuid: "e873e0e292554636a9f2813ef8ed92a3"
 from {username='slave'} at 127.0.0.1:56648
I20260709 13:07:56.601445 20187 raft_consensus.cc:606] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Received request to transfer leadership to TS e873e0e292554636a9f2813ef8ed92a3
I20260709 13:07:56.617456 20114 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba6459261f244c929626958cdd5124c8"
dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
mode: GRACEFUL
new_leader_uuid: "e873e0e292554636a9f2813ef8ed92a3"
 from {username='slave'} at 127.0.0.1:41210
I20260709 13:07:56.618116 20114 raft_consensus.cc:606] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Received request to transfer leadership to TS e873e0e292554636a9f2813ef8ed92a3
I20260709 13:07:56.628132 20114 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6e77f8f8ac4417ca238c2951bdf9ce5"
dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
mode: GRACEFUL
new_leader_uuid: "e873e0e292554636a9f2813ef8ed92a3"
 from {username='slave'} at 127.0.0.1:41210
I20260709 13:07:56.628917 20114 raft_consensus.cc:606] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Received request to transfer leadership to TS e873e0e292554636a9f2813ef8ed92a3
I20260709 13:07:56.631412 20264 consensus_queue.cc:1048] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.636827 20187 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d36135d0e0d425a83c860422de70367"
dest_uuid: "aeccd99157ea41799d2e4c1592044899"
mode: GRACEFUL
new_leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
 from {username='slave'} at 127.0.0.1:56648
I20260709 13:07:56.637815 20187 raft_consensus.cc:606] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 2 LEADER]: Received request to transfer leadership to TS 939a6adbfc6b4d49adda1ebec89d8d6e
I20260709 13:07:56.641981 20271 raft_consensus.cc:993] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e: : Instructing follower e873e0e292554636a9f2813ef8ed92a3 to start an election
I20260709 13:07:56.642491 20271 raft_consensus.cc:1081] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Signalling peer e873e0e292554636a9f2813ef8ed92a3 to start an election
I20260709 13:07:56.644615 20039 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a6e77f8f8ac4417ca238c2951bdf9ce5"
dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
 from {username='slave'} at 127.0.0.1:53956
I20260709 13:07:56.646039 20039 raft_consensus.cc:493] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:56.646452 20039 raft_consensus.cc:3060] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:56.656538 20276 raft_consensus.cc:493] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.657218 20264 consensus_queue.cc:1048] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.657277 20276 raft_consensus.cc:515] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.659948 20276 leader_election.cc:290] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:07:56.659799 20039 raft_consensus.cc:515] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.662664 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96c1cd6fac4472492f90a5d96208c66" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:07:56.663918 20037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96c1cd6fac4472492f90a5d96208c66" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:07:56.665012 20037 raft_consensus.cc:2393] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aeccd99157ea41799d2e4c1592044899 in current term 1: Already voted for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in this term.
I20260709 13:07:56.665221 20036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fdb29224078847b6b7344b8a4baada34"
dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
mode: GRACEFUL
new_leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
 from {username='slave'} at 127.0.0.1:53978
I20260709 13:07:56.665907 20036 raft_consensus.cc:606] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Received request to transfer leadership to TS 939a6adbfc6b4d49adda1ebec89d8d6e
I20260709 13:07:56.667371 20153 leader_election.cc:304] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aeccd99157ea41799d2e4c1592044899; no voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3
I20260709 13:07:56.668748 20276 raft_consensus.cc:3060] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.673985 20039 leader_election.cc:290] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 2 election: Requested vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:56.675244 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e77f8f8ac4417ca238c2951bdf9ce5" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:07:56.676101 20114 raft_consensus.cc:3055] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:56.676659 20114 raft_consensus.cc:740] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:07:56.677670 20114 consensus_queue.cc:260] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.679101 20114 raft_consensus.cc:3060] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:56.684247 20234 consensus_queue.cc:1048] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.685257 20276 raft_consensus.cc:2749] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:56.687359 20112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21c499bdb1fe4703ad7bafda1314b22b"
dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
mode: GRACEFUL
new_leader_uuid: "aeccd99157ea41799d2e4c1592044899"
 from {username='slave'} at 127.0.0.1:41210
I20260709 13:07:56.688243 20112 raft_consensus.cc:606] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Received request to transfer leadership to TS aeccd99157ea41799d2e4c1592044899
I20260709 13:07:56.690548 20286 raft_consensus.cc:993] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3: : Instructing follower 939a6adbfc6b4d49adda1ebec89d8d6e to start an election
I20260709 13:07:56.691187 20286 raft_consensus.cc:1081] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Signalling peer 939a6adbfc6b4d49adda1ebec89d8d6e to start an election
I20260709 13:07:56.692778 20114 raft_consensus.cc:2468] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 2.
I20260709 13:07:56.676002 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e77f8f8ac4417ca238c2951bdf9ce5" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:07:56.694552 20005 leader_election.cc:304] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:07:56.694794 20114 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fdb29224078847b6b7344b8a4baada34"
dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
 from {username='slave'} at 127.0.0.1:41190
I20260709 13:07:56.695597 20114 raft_consensus.cc:493] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:56.695842 20249 raft_consensus.cc:2804] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:56.696074 20114 raft_consensus.cc:3060] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:56.696506 20249 raft_consensus.cc:697] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [term 2 LEADER]: Becoming Leader. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:07:56.697539 20249 consensus_queue.cc:237] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.705564 20114 raft_consensus.cc:515] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.706537 20264 consensus_queue.cc:1048] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.708295 20114 leader_election.cc:290] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 2 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:56.709623 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb29224078847b6b7344b8a4baada34" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:07:56.710500 20188 raft_consensus.cc:3060] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:56.709707 20281 raft_consensus.cc:993] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899: : Instructing follower e873e0e292554636a9f2813ef8ed92a3 to start an election
I20260709 13:07:56.711808 20281 raft_consensus.cc:1081] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Signalling peer e873e0e292554636a9f2813ef8ed92a3 to start an election
I20260709 13:07:56.714445 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb29224078847b6b7344b8a4baada34" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:07:56.716289 20039 raft_consensus.cc:3055] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:56.717317 20039 raft_consensus.cc:740] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:07:56.718107 20037 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91501c5709394cedbd76b094e9d8044b"
dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
 from {username='slave'} at 127.0.0.1:53968
I20260709 13:07:56.718917 20037 raft_consensus.cc:493] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:56.719322 20037 raft_consensus.cc:3060] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:56.719714 20039 consensus_queue.cc:260] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.721066 20234 consensus_queue.cc:1048] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.722662 20039 raft_consensus.cc:3060] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:56.727793 20188 raft_consensus.cc:2468] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 2.
I20260709 13:07:56.729529 20078 leader_election.cc:304] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: 
I20260709 13:07:56.730801 20264 raft_consensus.cc:2804] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:56.731699 20264 raft_consensus.cc:697] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 LEADER]: Becoming Leader. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:07:56.732754 20264 consensus_queue.cc:237] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.734602 20037 raft_consensus.cc:515] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } } peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } }
I20260709 13:07:56.737999 19937 catalog_manager.cc:5697] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 reported cstate change: term changed from 1 to 2, leader changed from 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130) to e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129). New cstate: current_term: 2 leader_uuid: "e873e0e292554636a9f2813ef8ed92a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.741659 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91501c5709394cedbd76b094e9d8044b" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:07:56.742496 20188 raft_consensus.cc:3055] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:56.742863 20188 raft_consensus.cc:740] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:07:56.743865 20188 consensus_queue.cc:260] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } } peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } }
I20260709 13:07:56.744259 20264 consensus_queue.cc:1048] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.745188 20188 raft_consensus.cc:3060] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:56.756579 20039 raft_consensus.cc:2468] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 2.
I20260709 13:07:56.760779 20037 leader_election.cc:290] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 2 election: Requested vote from peers aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:07:56.764091 19937 catalog_manager.cc:5697] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e reported cstate change: term changed from 1 to 2, leader changed from e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129) to 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130). New cstate: current_term: 2 leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.766923 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91501c5709394cedbd76b094e9d8044b" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:07:56.767748 20112 raft_consensus.cc:3060] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:56.768707 20188 raft_consensus.cc:2468] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 2.
I20260709 13:07:56.770290 20003 leader_election.cc:304] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:07:56.771322 20233 raft_consensus.cc:2804] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:56.771802 20233 raft_consensus.cc:697] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [term 2 LEADER]: Becoming Leader. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:07:56.772621 20233 consensus_queue.cc:237] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } } peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } }
I20260709 13:07:56.785084 19934 catalog_manager.cc:5697] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 reported cstate change: term changed from 1 to 2, leader changed from aeccd99157ea41799d2e4c1592044899 (127.7.89.131) to e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129). New cstate: current_term: 2 leader_uuid: "e873e0e292554636a9f2813ef8ed92a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.788007 20112 raft_consensus.cc:2468] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 2.
I20260709 13:07:56.846944 20112 raft_consensus.cc:3060] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:56.853372 20112 raft_consensus.cc:1275] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Refusing update from remote peer aeccd99157ea41799d2e4c1592044899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:56.855087 20234 consensus_queue.cc:1048] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:56.861205 20276 raft_consensus.cc:993] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899: : Instructing follower 939a6adbfc6b4d49adda1ebec89d8d6e to start an election
I20260709 13:07:56.861758 20276 raft_consensus.cc:1081] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 2 LEADER]: Signalling peer 939a6adbfc6b4d49adda1ebec89d8d6e to start an election
I20260709 13:07:56.864041 20114 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d36135d0e0d425a83c860422de70367"
dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
 from {username='slave'} at 127.0.0.1:41196
I20260709 13:07:56.866060 20114 raft_consensus.cc:493] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:56.866696 20114 raft_consensus.cc:3060] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:56.867343 20037 raft_consensus.cc:1275] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Refusing update from remote peer aeccd99157ea41799d2e4c1592044899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:56.869200 20276 consensus_queue.cc:1048] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:56.877211 20114 raft_consensus.cc:515] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.883059 20114 leader_election.cc:290] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 3 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:56.883275 20037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d36135d0e0d425a83c860422de70367" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:07:56.884140 20037 raft_consensus.cc:3060] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:56.884582 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d36135d0e0d425a83c860422de70367" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:07:56.885385 20187 raft_consensus.cc:3055] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:56.885901 20187 raft_consensus.cc:740] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:07:56.886919 20187 consensus_queue.cc:260] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.888489 20187 raft_consensus.cc:3060] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:56.894675 20037 raft_consensus.cc:2468] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 3.
I20260709 13:07:56.897855 20187 raft_consensus.cc:2468] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 3.
I20260709 13:07:56.899807 20079 leader_election.cc:304] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:07:56.901072 20264 raft_consensus.cc:2804] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:56.901661 20264 raft_consensus.cc:697] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 LEADER]: Becoming Leader. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:07:56.902841 20264 consensus_queue.cc:237] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.911104 19934 catalog_manager.cc:5697] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e reported cstate change: term changed from 2 to 3, leader changed from aeccd99157ea41799d2e4c1592044899 (127.7.89.131) to 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130). New cstate: current_term: 3 leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.938407 20270 raft_consensus.cc:993] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e: : Instructing follower e873e0e292554636a9f2813ef8ed92a3 to start an election
I20260709 13:07:56.939013 20264 raft_consensus.cc:1081] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Signalling peer e873e0e292554636a9f2813ef8ed92a3 to start an election
I20260709 13:07:56.941112 20037 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba6459261f244c929626958cdd5124c8"
dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
 from {username='slave'} at 127.0.0.1:53956
I20260709 13:07:56.941797 20037 raft_consensus.cc:493] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:56.942194 20037 raft_consensus.cc:3060] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:56.950796 20037 raft_consensus.cc:515] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.952822 20037 leader_election.cc:290] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 2 election: Requested vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:07:56.953824 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba6459261f244c929626958cdd5124c8" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:07:56.954041 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba6459261f244c929626958cdd5124c8" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:07:56.954775 20114 raft_consensus.cc:3055] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:56.954799 20187 raft_consensus.cc:3060] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:56.955225 20114 raft_consensus.cc:740] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:07:56.956142 20114 consensus_queue.cc:260] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.957309 20114 raft_consensus.cc:3060] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:56.961651 20187 raft_consensus.cc:2468] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 2.
I20260709 13:07:56.963430 20003 leader_election.cc:304] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:07:56.964267 20114 raft_consensus.cc:2468] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 2.
I20260709 13:07:56.964462 20233 raft_consensus.cc:2804] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:56.965169 20233 raft_consensus.cc:697] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 2 LEADER]: Becoming Leader. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:07:56.966182 20233 consensus_queue.cc:237] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:56.973917 19934 catalog_manager.cc:5697] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 reported cstate change: term changed from 1 to 2, leader changed from 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130) to e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129). New cstate: current_term: 2 leader_uuid: "e873e0e292554636a9f2813ef8ed92a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:57.121454 20237 raft_consensus.cc:993] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e: : Instructing follower aeccd99157ea41799d2e4c1592044899 to start an election
I20260709 13:07:57.122135 20264 raft_consensus.cc:1081] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Signalling peer aeccd99157ea41799d2e4c1592044899 to start an election
I20260709 13:07:57.123986 20187 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21c499bdb1fe4703ad7bafda1314b22b"
dest_uuid: "aeccd99157ea41799d2e4c1592044899"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:07:57.124750 20187 raft_consensus.cc:493] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:57.125157 20187 raft_consensus.cc:3060] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:57.132395 20187 raft_consensus.cc:515] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:57.134407 20187 leader_election.cc:290] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 2 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:07:57.135541 20037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c499bdb1fe4703ad7bafda1314b22b" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:07:57.135936 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21c499bdb1fe4703ad7bafda1314b22b" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:07:57.136659 20037 raft_consensus.cc:3060] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:57.136749 20114 raft_consensus.cc:3055] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:57.137257 20114 raft_consensus.cc:740] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:07:57.138172 20114 consensus_queue.cc:260] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:57.139657 20114 raft_consensus.cc:3060] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:57.142891 20187 raft_consensus.cc:1275] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Refusing update from remote peer 939a6adbfc6b4d49adda1ebec89d8d6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:57.145053 20264 consensus_queue.cc:1048] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:57.147570 20037 raft_consensus.cc:2468] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 2.
I20260709 13:07:57.149284 20153 leader_election.cc:304] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:07:57.149797 20114 raft_consensus.cc:2468] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 2.
I20260709 13:07:57.151279 20234 raft_consensus.cc:2804] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:57.151947 20234 raft_consensus.cc:697] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 2 LEADER]: Becoming Leader. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:07:57.152926 20234 consensus_queue.cc:237] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:07:57.157287 20037 raft_consensus.cc:1275] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Refusing update from remote peer 939a6adbfc6b4d49adda1ebec89d8d6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:57.159305 20270 consensus_queue.cc:1048] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:57.164608 19935 catalog_manager.cc:5697] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 reported cstate change: term changed from 1 to 2, leader changed from 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130) to aeccd99157ea41799d2e4c1592044899 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "aeccd99157ea41799d2e4c1592044899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:57.263113 20114 raft_consensus.cc:1275] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Refusing update from remote peer e873e0e292554636a9f2813ef8ed92a3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:57.264688 20249 consensus_queue.cc:1048] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:57.266161 20187 raft_consensus.cc:3060] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:57.275664 20187 raft_consensus.cc:1275] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Refusing update from remote peer e873e0e292554636a9f2813ef8ed92a3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:57.277769 20233 consensus_queue.cc:1048] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:57.280861 20114 raft_consensus.cc:1275] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Refusing update from remote peer e873e0e292554636a9f2813ef8ed92a3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:57.284037 20233 consensus_queue.cc:1048] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:57.296198 20187 raft_consensus.cc:1275] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Refusing update from remote peer e873e0e292554636a9f2813ef8ed92a3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:57.298027 20286 consensus_queue.cc:1048] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:57.354008 20037 raft_consensus.cc:1275] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Refusing update from remote peer 939a6adbfc6b4d49adda1ebec89d8d6e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:57.355621 20264 consensus_queue.cc:1048] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:57.366873 20187 raft_consensus.cc:1275] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Refusing update from remote peer 939a6adbfc6b4d49adda1ebec89d8d6e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:57.369040 20264 consensus_queue.cc:1048] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:57.398240 20187 raft_consensus.cc:1275] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Refusing update from remote peer e873e0e292554636a9f2813ef8ed92a3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:57.399902 20286 consensus_queue.cc:1048] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:57.408728 20114 raft_consensus.cc:1275] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Refusing update from remote peer e873e0e292554636a9f2813ef8ed92a3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:57.411111 20286 consensus_queue.cc:1048] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:57.702574 20037 raft_consensus.cc:1275] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Refusing update from remote peer aeccd99157ea41799d2e4c1592044899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:57.718528 20276 consensus_queue.cc:1048] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:57.739925 20114 raft_consensus.cc:1275] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Refusing update from remote peer aeccd99157ea41799d2e4c1592044899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:57.743498 20317 consensus_queue.cc:1048] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:59.730180 19935 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52178:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:59.733651 19935 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:59.803402 20028 tablet_service.cc:1511] Processing CreateTablet for tablet 21d59234792f41bc8721313e837adf16 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:59.804410 20104 tablet_service.cc:1511] Processing CreateTablet for tablet 21d59234792f41bc8721313e837adf16 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:59.805747 20028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21d59234792f41bc8721313e837adf16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.806466 20104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21d59234792f41bc8721313e837adf16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.806589 20173 tablet_service.cc:1511] Processing CreateTablet for tablet 21d59234792f41bc8721313e837adf16 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:59.808738 20173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21d59234792f41bc8721313e837adf16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.812016 20171 tablet_service.cc:1511] Processing CreateTablet for tablet 91ea530bda574cf78cdab4f5be267aee (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:59.813316 20099 tablet_service.cc:1511] Processing CreateTablet for tablet 91ea530bda574cf78cdab4f5be267aee (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:59.813987 20171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91ea530bda574cf78cdab4f5be267aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.815140 20099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91ea530bda574cf78cdab4f5be267aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.816540 20172 tablet_service.cc:1511] Processing CreateTablet for tablet 5e468afda36b481688411ab728d7aa74 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:59.818356 20172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e468afda36b481688411ab728d7aa74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.823979 20102 tablet_service.cc:1511] Processing CreateTablet for tablet 5e468afda36b481688411ab728d7aa74 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:59.825773 20102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e468afda36b481688411ab728d7aa74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.825755 20103 tablet_service.cc:1511] Processing CreateTablet for tablet 4b2833da36fe4d26a5e8bd292db5865d (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:59.826160 20029 tablet_service.cc:1511] Processing CreateTablet for tablet 91ea530bda574cf78cdab4f5be267aee (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:59.827657 20103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b2833da36fe4d26a5e8bd292db5865d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.827716 20029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91ea530bda574cf78cdab4f5be267aee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.823299 20100 tablet_service.cc:1511] Processing CreateTablet for tablet c74c3cf888194042a7ba99ed2a7bb576 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:59.831437 20174 tablet_service.cc:1511] Processing CreateTablet for tablet c74c3cf888194042a7ba99ed2a7bb576 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:59.832558 20100 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c74c3cf888194042a7ba99ed2a7bb576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.833343 20174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c74c3cf888194042a7ba99ed2a7bb576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.837852 20176 tablet_service.cc:1511] Processing CreateTablet for tablet 4b2833da36fe4d26a5e8bd292db5865d (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:59.840175 20176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b2833da36fe4d26a5e8bd292db5865d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.855052 20021 tablet_service.cc:1511] Processing CreateTablet for tablet 5e468afda36b481688411ab728d7aa74 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:59.856953 20021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e468afda36b481688411ab728d7aa74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.865414 20101 tablet_service.cc:1511] Processing CreateTablet for tablet c9a384eab89841d09ed111f5324ec40e (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:59.867726 20101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9a384eab89841d09ed111f5324ec40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.872778 20025 tablet_service.cc:1511] Processing CreateTablet for tablet c74c3cf888194042a7ba99ed2a7bb576 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:59.874670 20025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c74c3cf888194042a7ba99ed2a7bb576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.879240 20175 tablet_service.cc:1511] Processing CreateTablet for tablet c9a384eab89841d09ed111f5324ec40e (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:59.881247 20175 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9a384eab89841d09ed111f5324ec40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.885811 20177 tablet_service.cc:1511] Processing CreateTablet for tablet c86f5fdd6d564f4a9b4c0ddf14aa77bf (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:59.887728 20177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c86f5fdd6d564f4a9b4c0ddf14aa77bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.892499 20098 tablet_service.cc:1511] Processing CreateTablet for tablet c86f5fdd6d564f4a9b4c0ddf14aa77bf (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:59.894776 20098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c86f5fdd6d564f4a9b4c0ddf14aa77bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.897075 20096 tablet_service.cc:1511] Processing CreateTablet for tablet 22b23637181d474c910b37980c6b3169 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:59.899228 20096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22b23637181d474c910b37980c6b3169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.911764 20026 tablet_service.cc:1511] Processing CreateTablet for tablet 4b2833da36fe4d26a5e8bd292db5865d (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:59.913722 20026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b2833da36fe4d26a5e8bd292db5865d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.912027 20178 tablet_service.cc:1511] Processing CreateTablet for tablet 22b23637181d474c910b37980c6b3169 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:59.915755 20178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22b23637181d474c910b37980c6b3169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.921522 20027 tablet_service.cc:1511] Processing CreateTablet for tablet c9a384eab89841d09ed111f5324ec40e (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:59.923533 20027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9a384eab89841d09ed111f5324ec40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.923120 20022 tablet_service.cc:1511] Processing CreateTablet for tablet c86f5fdd6d564f4a9b4c0ddf14aa77bf (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:59.924746 20022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c86f5fdd6d564f4a9b4c0ddf14aa77bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.923233 20102 tablet_service.cc:1511] Processing CreateTablet for tablet c7834843f08a42e08a30e38b0ebb1d29 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:59.929085 20102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7834843f08a42e08a30e38b0ebb1d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.931959 20023 tablet_service.cc:1511] Processing CreateTablet for tablet 22b23637181d474c910b37980c6b3169 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:59.933861 20023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22b23637181d474c910b37980c6b3169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.936308 20170 tablet_service.cc:1511] Processing CreateTablet for tablet c7834843f08a42e08a30e38b0ebb1d29 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:59.938107 20170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7834843f08a42e08a30e38b0ebb1d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.940414 20024 tablet_service.cc:1511] Processing CreateTablet for tablet c7834843f08a42e08a30e38b0ebb1d29 (DEFAULT_TABLE table=soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:59.942170 20024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7834843f08a42e08a30e38b0ebb1d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:59.989800 20333 tablet_bootstrap.cc:492] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:08:00.018492 20333 tablet_bootstrap.cc:654] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.043377 20334 tablet_bootstrap.cc:492] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:08:00.086004 20334 tablet_bootstrap.cc:654] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.135769 20333 tablet_bootstrap.cc:492] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:08:00.136638 20333 ts_tablet_manager.cc:1403] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.147s	user 0.023s	sys 0.015s
I20260709 13:08:00.137621 20335 tablet_bootstrap.cc:492] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:08:00.140862 20333 raft_consensus.cc:359] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.142107 20333 raft_consensus.cc:385] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.142552 20333 raft_consensus.cc:740] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.143625 20333 consensus_queue.cc:260] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.144874 20264 raft_consensus.cc:493] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.145829 20264 raft_consensus.cc:515] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.146355 20334 tablet_bootstrap.cc:492] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:08:00.147094 20334 ts_tablet_manager.cc:1403] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.104s	user 0.009s	sys 0.013s
I20260709 13:08:00.150997 20334 raft_consensus.cc:359] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.151728 20037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c74c3cf888194042a7ba99ed2a7bb576" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:08:00.154246 20334 raft_consensus.cc:385] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.155260 20334 raft_consensus.cc:740] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.152802 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c74c3cf888194042a7ba99ed2a7bb576" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
W20260709 13:08:00.155848 20079 leader_election.cc:343] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:00.153218 20264 leader_election.cc:290] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:00.156176 20334 consensus_queue.cc:260] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
W20260709 13:08:00.157095 20078 leader_election.cc:343] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:00.157568 20078 leader_election.cc:304] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e; no voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3
I20260709 13:08:00.159345 20334 ts_tablet_manager.cc:1434] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.003s
I20260709 13:08:00.162029 20334 tablet_bootstrap.cc:492] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:08:00.160095 20264 raft_consensus.cc:2749] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:00.167891 20333 ts_tablet_manager.cc:1434] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.006s
I20260709 13:08:00.169211 20333 tablet_bootstrap.cc:492] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:08:00.171240 20335 tablet_bootstrap.cc:654] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.181769 20334 tablet_bootstrap.cc:654] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.183267 20333 tablet_bootstrap.cc:654] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.187984 20335 tablet_bootstrap.cc:492] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:08:00.188861 20335 ts_tablet_manager.cc:1403] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.010s
I20260709 13:08:00.190331 20334 tablet_bootstrap.cc:492] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:08:00.191185 20334 ts_tablet_manager.cc:1403] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.012s
I20260709 13:08:00.191540 20333 tablet_bootstrap.cc:492] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:08:00.192188 20333 ts_tablet_manager.cc:1403] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.004s
I20260709 13:08:00.192881 20335 raft_consensus.cc:359] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.193847 20335 raft_consensus.cc:385] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.194433 20335 raft_consensus.cc:740] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.195124 20334 raft_consensus.cc:359] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.195915 20333 raft_consensus.cc:359] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.196135 20334 raft_consensus.cc:385] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.195577 20335 consensus_queue.cc:260] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.196873 20333 raft_consensus.cc:385] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.197036 20334 raft_consensus.cc:740] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.197301 20333 raft_consensus.cc:740] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.198185 20334 consensus_queue.cc:260] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.198253 20333 consensus_queue.cc:260] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.201573 20333 ts_tablet_manager.cc:1434] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:08:00.202986 20333 tablet_bootstrap.cc:492] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:08:00.209098 20334 ts_tablet_manager.cc:1434] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.002s
I20260709 13:08:00.210772 20334 tablet_bootstrap.cc:492] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:08:00.211812 20333 tablet_bootstrap.cc:654] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.221474 20334 tablet_bootstrap.cc:654] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.216197 20335 ts_tablet_manager.cc:1434] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.004s
I20260709 13:08:00.227412 20335 tablet_bootstrap.cc:492] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:08:00.238595 20320 raft_consensus.cc:493] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.239648 20333 tablet_bootstrap.cc:492] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:08:00.239429 20320 raft_consensus.cc:515] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.239656 20334 tablet_bootstrap.cc:492] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:08:00.240414 20333 ts_tablet_manager.cc:1403] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.010s
I20260709 13:08:00.240762 20334 ts_tablet_manager.cc:1403] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.010s
I20260709 13:08:00.242148 20320 leader_election.cc:290] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:08:00.243150 20037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d59234792f41bc8721313e837adf16" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:08:00.243708 20335 tablet_bootstrap.cc:654] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.243687 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d59234792f41bc8721313e837adf16" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:08:00.244676 20334 raft_consensus.cc:359] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
W20260709 13:08:00.245258 20153 leader_election.cc:343] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:00.245537 20334 raft_consensus.cc:385] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.245921 20334 raft_consensus.cc:740] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.246891 20334 consensus_queue.cc:260] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
W20260709 13:08:00.249071 20154 leader_election.cc:343] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:00.249904 20154 leader_election.cc:304] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aeccd99157ea41799d2e4c1592044899; no voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3
I20260709 13:08:00.251313 20334 ts_tablet_manager.cc:1434] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:08:00.252738 20334 tablet_bootstrap.cc:492] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:08:00.256438 20320 raft_consensus.cc:2749] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:00.261135 20334 tablet_bootstrap.cc:654] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.269976 20333 raft_consensus.cc:359] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.271443 20333 raft_consensus.cc:385] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.271901 20333 raft_consensus.cc:740] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.273381 20335 tablet_bootstrap.cc:492] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:08:00.272964 20333 consensus_queue.cc:260] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.273542 20334 tablet_bootstrap.cc:492] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:08:00.274125 20335 ts_tablet_manager.cc:1403] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.047s	user 0.022s	sys 0.015s
I20260709 13:08:00.274241 20334 ts_tablet_manager.cc:1403] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.002s
I20260709 13:08:00.275990 20333 ts_tablet_manager.cc:1434] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.005s
I20260709 13:08:00.278002 20333 tablet_bootstrap.cc:492] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:08:00.277765 20335 raft_consensus.cc:359] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.278663 20335 raft_consensus.cc:385] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.278158 20334 raft_consensus.cc:359] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.279018 20335 raft_consensus.cc:740] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.279187 20334 raft_consensus.cc:385] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.279652 20334 raft_consensus.cc:740] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.279975 20335 consensus_queue.cc:260] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.283182 20335 ts_tablet_manager.cc:1434] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:08:00.284338 20335 tablet_bootstrap.cc:492] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:08:00.291229 20264 raft_consensus.cc:493] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.292052 20264 raft_consensus.cc:515] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.280599 20334 consensus_queue.cc:260] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.298540 20320 raft_consensus.cc:493] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.298864 20337 raft_consensus.cc:493] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.294538 20264 leader_election.cc:290] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:00.301574 20320 raft_consensus.cc:515] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.301694 20337 raft_consensus.cc:515] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.297088 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9a384eab89841d09ed111f5324ec40e" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:08:00.304302 20334 ts_tablet_manager.cc:1434] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:08:00.305225 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e468afda36b481688411ab728d7aa74" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:08:00.305686 20334 tablet_bootstrap.cc:492] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:08:00.305902 20337 leader_election.cc:290] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:00.296849 20335 tablet_bootstrap.cc:654] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.299348 20037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9a384eab89841d09ed111f5324ec40e" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:08:00.307986 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e468afda36b481688411ab728d7aa74" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:08:00.308818 20188 raft_consensus.cc:2468] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 0.
I20260709 13:08:00.294031 20333 tablet_bootstrap.cc:654] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.310792 20003 leader_election.cc:304] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
W20260709 13:08:00.305708 20078 leader_election.cc:343] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:00.312351 20079 leader_election.cc:343] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:00.312817 20079 leader_election.cc:304] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e; no voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3
I20260709 13:08:00.306087 20039 raft_consensus.cc:2468] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 0.
I20260709 13:08:00.313761 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e468afda36b481688411ab728d7aa74" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:08:00.314530 20112 raft_consensus.cc:2468] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 0.
I20260709 13:08:00.315303 20153 leader_election.cc:304] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:08:00.315881 20334 tablet_bootstrap.cc:654] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.315464 20264 raft_consensus.cc:2749] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:00.317901 20332 raft_consensus.cc:2804] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:00.318408 20332 raft_consensus.cc:493] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.318799 20332 raft_consensus.cc:3060] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.321123 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e468afda36b481688411ab728d7aa74" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:08:00.321970 20114 raft_consensus.cc:2468] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 0.
I20260709 13:08:00.323647 20320 leader_election.cc:290] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:08:00.324267 20320 raft_consensus.cc:2804] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:00.324607 20320 raft_consensus.cc:493] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.324925 20320 raft_consensus.cc:3060] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.337527 20332 raft_consensus.cc:515] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.341953 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e468afda36b481688411ab728d7aa74" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:08:00.342882 20112 raft_consensus.cc:3060] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.350169 20320 raft_consensus.cc:515] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.356465 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e468afda36b481688411ab728d7aa74" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:08:00.357396 20039 raft_consensus.cc:2393] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate aeccd99157ea41799d2e4c1592044899 in current term 1: Already voted for candidate e873e0e292554636a9f2813ef8ed92a3 in this term.
I20260709 13:08:00.358160 20333 tablet_bootstrap.cc:492] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:08:00.359141 20333 ts_tablet_manager.cc:1403] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.082s	user 0.011s	sys 0.022s
I20260709 13:08:00.360821 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e468afda36b481688411ab728d7aa74" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:08:00.362862 20154 leader_election.cc:304] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aeccd99157ea41799d2e4c1592044899; no voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3
I20260709 13:08:00.363974 20350 raft_consensus.cc:2749] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:00.365504 20320 leader_election.cc:290] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:08:00.366741 20333 raft_consensus.cc:359] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.368041 20333 raft_consensus.cc:385] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.368472 20333 raft_consensus.cc:740] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.370229 20332 leader_election.cc:290] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Requested vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:00.372671 20335 tablet_bootstrap.cc:492] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:08:00.373157 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e468afda36b481688411ab728d7aa74" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:08:00.373984 20335 ts_tablet_manager.cc:1403] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.090s	user 0.018s	sys 0.017s
I20260709 13:08:00.374258 20188 raft_consensus.cc:2393] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e873e0e292554636a9f2813ef8ed92a3 in current term 1: Already voted for candidate aeccd99157ea41799d2e4c1592044899 in this term.
I20260709 13:08:00.369701 20333 consensus_queue.cc:260] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.372648 20112 raft_consensus.cc:2468] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 1.
I20260709 13:08:00.378140 20335 raft_consensus.cc:359] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.379221 20335 raft_consensus.cc:385] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.379597 20335 raft_consensus.cc:740] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.378502 20333 ts_tablet_manager.cc:1434] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.002s
I20260709 13:08:00.380190 20005 leader_election.cc:304] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: aeccd99157ea41799d2e4c1592044899
I20260709 13:08:00.380501 20335 consensus_queue.cc:260] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.385840 20333 tablet_bootstrap.cc:492] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:08:00.388090 20335 ts_tablet_manager.cc:1434] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.001s
I20260709 13:08:00.385485 20332 raft_consensus.cc:2804] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:00.389623 20335 tablet_bootstrap.cc:492] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:08:00.381220 20334 tablet_bootstrap.cc:492] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:08:00.390172 20332 raft_consensus.cc:697] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Becoming Leader. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:08:00.390544 20334 ts_tablet_manager.cc:1403] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.085s	user 0.031s	sys 0.028s
I20260709 13:08:00.391325 20332 consensus_queue.cc:237] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.394248 20333 tablet_bootstrap.cc:654] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.394892 20334 raft_consensus.cc:359] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.395869 20334 raft_consensus.cc:385] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.396196 20334 raft_consensus.cc:740] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.397109 20334 consensus_queue.cc:260] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.407923 20333 tablet_bootstrap.cc:492] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:08:00.412824 20333 ts_tablet_manager.cc:1403] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.007s
I20260709 13:08:00.409402 20334 ts_tablet_manager.cc:1434] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:08:00.416488 20334 tablet_bootstrap.cc:492] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:08:00.416440 20333 raft_consensus.cc:359] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.417748 20333 raft_consensus.cc:385] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.418143 20333 raft_consensus.cc:740] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.419203 20333 consensus_queue.cc:260] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.409729 20335 tablet_bootstrap.cc:654] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.422832 20333 ts_tablet_manager.cc:1434] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.003s
I20260709 13:08:00.426383 20333 tablet_bootstrap.cc:492] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:08:00.436074 20333 tablet_bootstrap.cc:654] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.442354 20334 tablet_bootstrap.cc:654] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.446696 20333 tablet_bootstrap.cc:492] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:08:00.445334 20335 tablet_bootstrap.cc:492] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:08:00.447504 20333 ts_tablet_manager.cc:1403] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 13:08:00.447623 20335 ts_tablet_manager.cc:1403] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.058s	user 0.008s	sys 0.014s
I20260709 13:08:00.451573 20335 raft_consensus.cc:359] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.451740 20333 raft_consensus.cc:359] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.454681 20335 raft_consensus.cc:385] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.454727 20333 raft_consensus.cc:385] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.455404 20335 raft_consensus.cc:740] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.455408 20333 raft_consensus.cc:740] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.456552 20335 consensus_queue.cc:260] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.456552 20333 consensus_queue.cc:260] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.460088 20333 ts_tablet_manager.cc:1434] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.004s
I20260709 13:08:00.460259 20335 ts_tablet_manager.cc:1434] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:08:00.461477 20333 tablet_bootstrap.cc:492] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:08:00.464588 20334 tablet_bootstrap.cc:492] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:08:00.465469 20334 ts_tablet_manager.cc:1403] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.009s
I20260709 13:08:00.469515 20334 raft_consensus.cc:359] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.461561 20335 tablet_bootstrap.cc:492] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:08:00.479449 20335 tablet_bootstrap.cc:654] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.470688 20334 raft_consensus.cc:385] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.488054 20334 raft_consensus.cc:740] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.489619 20334 consensus_queue.cc:260] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.470345 20333 tablet_bootstrap.cc:654] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.493032 20335 tablet_bootstrap.cc:492] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:08:00.493825 20335 ts_tablet_manager.cc:1403] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.032s	user 0.020s	sys 0.002s
I20260709 13:08:00.496735 20334 ts_tablet_manager.cc:1434] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.002s
I20260709 13:08:00.498301 20334 tablet_bootstrap.cc:492] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:08:00.497865 20335 raft_consensus.cc:359] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.498927 20335 raft_consensus.cc:385] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.499321 20335 raft_consensus.cc:740] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.500463 20335 consensus_queue.cc:260] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.503899 20335 ts_tablet_manager.cc:1434] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:08:00.505288 20335 tablet_bootstrap.cc:492] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:08:00.506655 20337 raft_consensus.cc:493] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.508369 20337 raft_consensus.cc:515] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.509994 20333 tablet_bootstrap.cc:492] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:08:00.510732 20333 ts_tablet_manager.cc:1403] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.004s
I20260709 13:08:00.512502 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86f5fdd6d564f4a9b4c0ddf14aa77bf" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:08:00.514386 20320 raft_consensus.cc:493] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.515072 20320 raft_consensus.cc:515] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.520004 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86f5fdd6d564f4a9b4c0ddf14aa77bf" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:08:00.520850 20039 raft_consensus.cc:2468] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 0.
I20260709 13:08:00.518412 20320 leader_election.cc:290] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:08:00.523967 20153 leader_election.cc:304] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
W20260709 13:08:00.522804 20005 leader_election.cc:343] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:00.524747 20333 raft_consensus.cc:359] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.525786 20333 raft_consensus.cc:385] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.526150 20333 raft_consensus.cc:740] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.525941 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86f5fdd6d564f4a9b4c0ddf14aa77bf" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:08:00.526516 20350 raft_consensus.cc:2804] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:00.527189 20350 raft_consensus.cc:493] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.527189 20333 consensus_queue.cc:260] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.527902 20350 raft_consensus.cc:3060] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:00.527953 20154 leader_election.cc:343] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:00.530452 20337 leader_election.cc:290] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:00.530730 20333 ts_tablet_manager.cc:1434] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.001s
I20260709 13:08:00.516961 20334 tablet_bootstrap.cc:654] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.535295 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86f5fdd6d564f4a9b4c0ddf14aa77bf" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:08:00.536603 20333 tablet_bootstrap.cc:492] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:08:00.551333 19934 catalog_manager.cc:5697] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "e873e0e292554636a9f2813ef8ed92a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:00.554028 20334 tablet_bootstrap.cc:492] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:08:00.554534 20335 tablet_bootstrap.cc:654] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.554730 20350 raft_consensus.cc:515] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.555474 20264 raft_consensus.cc:493] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.556263 20264 raft_consensus.cc:515] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.560297 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7834843f08a42e08a30e38b0ebb1d29" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:08:00.561100 20039 raft_consensus.cc:2468] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 0.
I20260709 13:08:00.562194 20333 tablet_bootstrap.cc:654] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.562704 20079 leader_election.cc:304] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:08:00.564399 20264 leader_election.cc:290] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:00.566594 20188 raft_consensus.cc:2393] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e873e0e292554636a9f2813ef8ed92a3 in current term 1: Already voted for candidate aeccd99157ea41799d2e4c1592044899 in this term.
I20260709 13:08:00.555029 20334 ts_tablet_manager.cc:1403] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.057s	user 0.013s	sys 0.003s
I20260709 13:08:00.567333 20339 raft_consensus.cc:2804] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:00.567987 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7834843f08a42e08a30e38b0ebb1d29" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:08:00.573323 20339 raft_consensus.cc:493] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.573836 20339 raft_consensus.cc:3060] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:00.574899 20078 leader_election.cc:343] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:00.578248 20335 tablet_bootstrap.cc:492] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:08:00.577721 20334 raft_consensus.cc:359] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.578842 20334 raft_consensus.cc:385] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.579234 20334 raft_consensus.cc:740] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.579242 20335 ts_tablet_manager.cc:1403] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.074s	user 0.019s	sys 0.001s
I20260709 13:08:00.580528 20334 consensus_queue.cc:260] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.583400 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86f5fdd6d564f4a9b4c0ddf14aa77bf" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:08:00.583964 20334 ts_tablet_manager.cc:1434] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.004s
I20260709 13:08:00.584275 20039 raft_consensus.cc:3060] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.584646 20333 tablet_bootstrap.cc:492] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:08:00.585256 20333 ts_tablet_manager.cc:1403] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.049s	user 0.006s	sys 0.010s
I20260709 13:08:00.585609 20334 tablet_bootstrap.cc:492] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:08:00.581396 20003 leader_election.cc:304] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e873e0e292554636a9f2813ef8ed92a3; no voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899
I20260709 13:08:00.600840 20335 raft_consensus.cc:359] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.609858 20335 raft_consensus.cc:385] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.616421 20335 raft_consensus.cc:740] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.617645 20335 consensus_queue.cc:260] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.606841 20339 raft_consensus.cc:515] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.623423 20320 raft_consensus.cc:493] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.624135 20320 raft_consensus.cc:515] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.625926 20335 ts_tablet_manager.cc:1434] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.046s	user 0.004s	sys 0.006s
I20260709 13:08:00.623422 20264 raft_consensus.cc:493] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.627686 20264 raft_consensus.cc:515] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.628388 20335 tablet_bootstrap.cc:492] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:08:00.629848 20264 leader_election.cc:290] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:00.631994 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7834843f08a42e08a30e38b0ebb1d29" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:08:00.632347 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2833da36fe4d26a5e8bd292db5865d" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:08:00.632753 20187 raft_consensus.cc:3060] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.611275 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86f5fdd6d564f4a9b4c0ddf14aa77bf" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:08:00.633904 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ea530bda574cf78cdab4f5be267aee" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:08:00.634686 20114 raft_consensus.cc:2468] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 0.
I20260709 13:08:00.626400 20320 leader_election.cc:290] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:08:00.607522 20333 raft_consensus.cc:359] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.635424 20350 leader_election.cc:290] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:08:00.636338 20154 leader_election.cc:304] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: 
I20260709 13:08:00.637338 20037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7834843f08a42e08a30e38b0ebb1d29" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:08:00.638114 20037 raft_consensus.cc:3060] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.638345 20036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2833da36fe4d26a5e8bd292db5865d" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:08:00.639281 20038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ea530bda574cf78cdab4f5be267aee" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
I20260709 13:08:00.640072 20038 raft_consensus.cc:2468] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 0.
I20260709 13:08:00.641232 20039 raft_consensus.cc:2468] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 1.
I20260709 13:08:00.641834 20332 raft_consensus.cc:2749] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:00.633334 20188 raft_consensus.cc:2468] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 0.
I20260709 13:08:00.642938 20153 leader_election.cc:304] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:08:00.635787 20333 raft_consensus.cc:385] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.644145 20320 raft_consensus.cc:2804] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:00.644277 20333 raft_consensus.cc:740] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.644685 20320 raft_consensus.cc:493] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.645146 20320 raft_consensus.cc:3060] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.650791 20339 leader_election.cc:290] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:00.615962 20334 tablet_bootstrap.cc:654] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.645547 20333 consensus_queue.cc:260] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.658937 20112 raft_consensus.cc:3060] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.660584 20333 ts_tablet_manager.cc:1434] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.075s	user 0.009s	sys 0.001s
I20260709 13:08:00.665057 20333 tablet_bootstrap.cc:492] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e: Bootstrap starting.
I20260709 13:08:00.657368 20187 raft_consensus.cc:2468] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 1.
I20260709 13:08:00.670096 20112 raft_consensus.cc:2468] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 1.
W20260709 13:08:00.640270 20079 leader_election.cc:343] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:00.655277 20037 raft_consensus.cc:2468] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 1.
I20260709 13:08:00.674585 20078 leader_election.cc:304] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: e873e0e292554636a9f2813ef8ed92a3
I20260709 13:08:00.655114 20320 raft_consensus.cc:515] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.666112 20335 tablet_bootstrap.cc:654] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.676795 20079 leader_election.cc:304] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:08:00.677744 20339 raft_consensus.cc:2804] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:00.680776 20339 raft_consensus.cc:493] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.681209 20339 raft_consensus.cc:3060] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.682005 20334 tablet_bootstrap.cc:492] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:08:00.682281 20333 tablet_bootstrap.cc:654] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.682335 20335 tablet_bootstrap.cc:492] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:08:00.682672 20334 ts_tablet_manager.cc:1403] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.097s	user 0.011s	sys 0.005s
I20260709 13:08:00.683075 20335 ts_tablet_manager.cc:1403] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.004s
I20260709 13:08:00.678539 20037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ea530bda574cf78cdab4f5be267aee" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:08:00.684592 20037 raft_consensus.cc:3060] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.678776 20320 leader_election.cc:290] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:08:00.667351 20350 raft_consensus.cc:2804] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:00.687909 20350 raft_consensus.cc:697] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Becoming Leader. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:08:00.688953 20350 consensus_queue.cc:237] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.689877 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ea530bda574cf78cdab4f5be267aee" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:08:00.690644 20114 raft_consensus.cc:3060] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.690469 20334 raft_consensus.cc:359] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.691344 20334 raft_consensus.cc:385] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.691634 20334 raft_consensus.cc:740] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.695053 20264 raft_consensus.cc:2804] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:00.694257 20335 raft_consensus.cc:359] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.696211 20335 raft_consensus.cc:385] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.692637 20334 consensus_queue.cc:260] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.699651 20339 raft_consensus.cc:515] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.696544 20335 raft_consensus.cc:740] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.702869 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2833da36fe4d26a5e8bd292db5865d" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:08:00.708361 20339 leader_election.cc:290] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:00.705862 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2833da36fe4d26a5e8bd292db5865d" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:08:00.706223 20320 raft_consensus.cc:493] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:00.709185 20079 leader_election.cc:343] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:00.709776 20320 raft_consensus.cc:515] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.709196 20187 raft_consensus.cc:3060] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.712580 20320 leader_election.cc:290] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:08:00.712800 20333 tablet_bootstrap.cc:492] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e: No bootstrap required, opened a new log
I20260709 13:08:00.712987 20037 raft_consensus.cc:2468] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 1.
I20260709 13:08:00.713387 20333 ts_tablet_manager.cc:1403] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent bootstrapping tablet: real 0.049s	user 0.007s	sys 0.007s
I20260709 13:08:00.705906 20335 consensus_queue.cc:260] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.715585 20153 leader_election.cc:304] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:08:00.716805 20264 raft_consensus.cc:697] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Becoming Leader. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:08:00.717042 20333 raft_consensus.cc:359] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.717830 20333 raft_consensus.cc:385] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.718122 20264 consensus_queue.cc:237] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.719184 20038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b23637181d474c910b37980c6b3169" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3" is_pre_election: true
W20260709 13:08:00.720732 20153 leader_election.cc:343] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:00.718135 20333 raft_consensus.cc:740] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.722878 20320 raft_consensus.cc:2804] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:00.723634 20320 raft_consensus.cc:697] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Becoming Leader. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:08:00.723291 20333 consensus_queue.cc:260] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.724711 20320 consensus_queue.cc:237] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.726047 20333 ts_tablet_manager.cc:1434] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.006s
I20260709 13:08:00.726416 20187 raft_consensus.cc:2468] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 1.
I20260709 13:08:00.726490 20114 raft_consensus.cc:2468] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 1.
I20260709 13:08:00.727962 20078 leader_election.cc:304] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: e873e0e292554636a9f2813ef8ed92a3
I20260709 13:08:00.701547 20334 ts_tablet_manager.cc:1434] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:08:00.729239 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b23637181d474c910b37980c6b3169" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:08:00.730551 20112 raft_consensus.cc:2468] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 0.
I20260709 13:08:00.732194 20154 leader_election.cc:304] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: e873e0e292554636a9f2813ef8ed92a3
I20260709 13:08:00.733246 20335 ts_tablet_manager.cc:1434] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.001s
I20260709 13:08:00.735087 20334 tablet_bootstrap.cc:492] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899: Bootstrap starting.
I20260709 13:08:00.735103 20347 raft_consensus.cc:2804] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:00.735947 20347 raft_consensus.cc:493] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.736517 20347 raft_consensus.cc:3060] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.743342 19935 catalog_manager.cc:5697] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 reported cstate change: term changed from 0 to 1, leader changed from <none> to aeccd99157ea41799d2e4c1592044899 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "aeccd99157ea41799d2e4c1592044899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:00.745136 20332 raft_consensus.cc:493] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.745808 20332 raft_consensus.cc:515] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.729493 20370 raft_consensus.cc:2804] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:00.747364 20370 raft_consensus.cc:697] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Becoming Leader. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:08:00.747962 20334 tablet_bootstrap.cc:654] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.748339 20370 consensus_queue.cc:237] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.750990 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c74c3cf888194042a7ba99ed2a7bb576" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:08:00.751762 20187 raft_consensus.cc:2468] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 0.
I20260709 13:08:00.745211 20335 tablet_bootstrap.cc:492] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:08:00.757121 20003 leader_election.cc:304] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:08:00.759627 20337 raft_consensus.cc:2804] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:08:00.759945 19985 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:00.760263 20337 raft_consensus.cc:493] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.760771 20337 raft_consensus.cc:3060] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.762508 20347 raft_consensus.cc:515] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.763185 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c74c3cf888194042a7ba99ed2a7bb576" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:08:00.763931 20112 raft_consensus.cc:2468] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 0.
I20260709 13:08:00.765040 20347 leader_election.cc:290] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:08:00.759781 20332 leader_election.cc:290] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:00.766551 20037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b23637181d474c910b37980c6b3169" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:08:00.766861 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b23637181d474c910b37980c6b3169" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:08:00.767817 20114 raft_consensus.cc:3060] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:00.768043 20153 leader_election.cc:343] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:00.772150 20335 tablet_bootstrap.cc:654] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.776711 19935 catalog_manager.cc:5697] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:00.783980 20114 raft_consensus.cc:2468] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 1.
I20260709 13:08:00.783929 20337 raft_consensus.cc:515] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.785856 20154 leader_election.cc:304] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: e873e0e292554636a9f2813ef8ed92a3
I20260709 13:08:00.787657 20334 tablet_bootstrap.cc:492] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899: No bootstrap required, opened a new log
I20260709 13:08:00.788335 20334 ts_tablet_manager.cc:1403] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899: Time spent bootstrapping tablet: real 0.058s	user 0.016s	sys 0.003s
I20260709 13:08:00.789626 20337 leader_election.cc:290] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Requested vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:00.790096 20347 raft_consensus.cc:2804] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:00.791534 20347 raft_consensus.cc:697] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Becoming Leader. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:08:00.792546 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c74c3cf888194042a7ba99ed2a7bb576" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:08:00.790468 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c74c3cf888194042a7ba99ed2a7bb576" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:08:00.793295 20187 raft_consensus.cc:3060] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.793490 20114 raft_consensus.cc:3060] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.795145 20334 raft_consensus.cc:359] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.792672 20347 consensus_queue.cc:237] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.795915 20334 raft_consensus.cc:385] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.796227 20334 raft_consensus.cc:740] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.798131 20334 consensus_queue.cc:260] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.800904 20334 ts_tablet_manager.cc:1434] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:08:00.802106 20187 raft_consensus.cc:2468] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 1.
I20260709 13:08:00.806229 20003 leader_election.cc:304] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:08:00.807293 20337 raft_consensus.cc:2804] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:00.808835 20337 raft_consensus.cc:697] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Becoming Leader. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:08:00.810230 20335 tablet_bootstrap.cc:492] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:08:00.809971 20337 consensus_queue.cc:237] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.811057 20335 ts_tablet_manager.cc:1403] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.077s	user 0.016s	sys 0.005s
I20260709 13:08:00.814496 20335 raft_consensus.cc:359] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.815435 20335 raft_consensus.cc:385] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.810375 20114 raft_consensus.cc:2468] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 1.
I20260709 13:08:00.815742 20335 raft_consensus.cc:740] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.816823 20335 consensus_queue.cc:260] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.811544 19937 catalog_manager.cc:5697] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 reported cstate change: term changed from 0 to 1, leader changed from <none> to aeccd99157ea41799d2e4c1592044899 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "aeccd99157ea41799d2e4c1592044899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:00.819527 20335 ts_tablet_manager.cc:1434] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:08:00.800741 19935 catalog_manager.cc:5697] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130). New cstate: current_term: 1 leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:00.826772 20335 tablet_bootstrap.cc:492] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3: Bootstrap starting.
I20260709 13:08:00.841248 20335 tablet_bootstrap.cc:654] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.848285 19937 catalog_manager.cc:5697] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "e873e0e292554636a9f2813ef8ed92a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:00.848418 19934 catalog_manager.cc:5697] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 reported cstate change: term changed from 0 to 1, leader changed from <none> to aeccd99157ea41799d2e4c1592044899 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "aeccd99157ea41799d2e4c1592044899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:00.853883 20335 tablet_bootstrap.cc:492] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3: No bootstrap required, opened a new log
I20260709 13:08:00.854692 20335 ts_tablet_manager.cc:1403] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.008s
I20260709 13:08:00.858136 20335 raft_consensus.cc:359] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.858989 20335 raft_consensus.cc:385] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.859277 20335 raft_consensus.cc:740] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.860015 20335 consensus_queue.cc:260] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.862252 20335 ts_tablet_manager.cc:1434] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:08:00.900280 20337 raft_consensus.cc:493] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.900928 20337 raft_consensus.cc:515] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.903844 20337 leader_election.cc:290] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:00.904834 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d59234792f41bc8721313e837adf16" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:08:00.905562 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d59234792f41bc8721313e837adf16" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:08:00.905742 20112 raft_consensus.cc:2468] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 0.
I20260709 13:08:00.906378 20187 raft_consensus.cc:2468] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 0.
I20260709 13:08:00.907229 20005 leader_election.cc:304] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:08:00.908177 20337 raft_consensus.cc:2804] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:00.908617 20337 raft_consensus.cc:493] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.908934 20337 raft_consensus.cc:3060] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.916479 20337 raft_consensus.cc:515] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.918699 20337 leader_election.cc:290] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Requested vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:00.919729 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d59234792f41bc8721313e837adf16" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:08:00.919831 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d59234792f41bc8721313e837adf16" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:08:00.920563 20187 raft_consensus.cc:3060] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.920563 20112 raft_consensus.cc:3060] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.929630 20187 raft_consensus.cc:2468] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 1.
I20260709 13:08:00.929630 20112 raft_consensus.cc:2468] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 1.
I20260709 13:08:00.931283 20003 leader_election.cc:304] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:08:00.932214 20337 raft_consensus.cc:2804] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:00.932664 20337 raft_consensus.cc:697] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Becoming Leader. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:08:00.933485 20337 consensus_queue.cc:237] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:00.942678 19934 catalog_manager.cc:5697] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "e873e0e292554636a9f2813ef8ed92a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:00.984721 20337 consensus_queue.cc:1048] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:00.999650 20383 consensus_queue.cc:1048] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.041678 20337 raft_consensus.cc:493] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.042205 20337 raft_consensus.cc:515] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.043531 20383 raft_consensus.cc:493] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.044126 20383 raft_consensus.cc:515] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.044536 20337 leader_election.cc:290] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:01.045823 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9a384eab89841d09ed111f5324ec40e" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:08:01.046322 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9a384eab89841d09ed111f5324ec40e" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:08:01.046710 20112 raft_consensus.cc:2468] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 0.
I20260709 13:08:01.047055 20187 raft_consensus.cc:2468] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 0.
I20260709 13:08:01.047219 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b23637181d474c910b37980c6b3169" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:08:01.048136 20003 leader_election.cc:304] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:08:01.048210 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b23637181d474c910b37980c6b3169" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:08:01.047430 20383 leader_election.cc:290] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:01.048151 20114 raft_consensus.cc:2393] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e873e0e292554636a9f2813ef8ed92a3 in current term 1: Already voted for candidate aeccd99157ea41799d2e4c1592044899 in this term.
I20260709 13:08:01.049417 20383 raft_consensus.cc:2804] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:01.049983 20383 raft_consensus.cc:493] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.050418 20383 raft_consensus.cc:3060] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:01.050554 20005 leader_election.cc:304] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e873e0e292554636a9f2813ef8ed92a3; no voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899
I20260709 13:08:01.051539 20337 raft_consensus.cc:3060] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:01.060464 20337 raft_consensus.cc:2749] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:01.060441 20383 raft_consensus.cc:515] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.063153 20383 leader_election.cc:290] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Requested vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:01.063899 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9a384eab89841d09ed111f5324ec40e" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:08:01.064285 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9a384eab89841d09ed111f5324ec40e" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:08:01.064644 20114 raft_consensus.cc:3060] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:01.065021 20188 raft_consensus.cc:3060] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:01.072705 20188 raft_consensus.cc:2468] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 1.
I20260709 13:08:01.072854 20114 raft_consensus.cc:2468] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 1.
I20260709 13:08:01.074883 20003 leader_election.cc:304] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:08:01.076026 20383 raft_consensus.cc:2804] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:01.076829 20383 raft_consensus.cc:697] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Becoming Leader. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:08:01.077963 20383 consensus_queue.cc:237] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.091885 19937 catalog_manager.cc:5697] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "e873e0e292554636a9f2813ef8ed92a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:01.096583 20383 raft_consensus.cc:493] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.097363 20383 raft_consensus.cc:515] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.099852 20383 leader_election.cc:290] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:01.101153 20188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2833da36fe4d26a5e8bd292db5865d" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeccd99157ea41799d2e4c1592044899" is_pre_election: true
I20260709 13:08:01.101553 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2833da36fe4d26a5e8bd292db5865d" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" is_pre_election: true
I20260709 13:08:01.104265 20188 raft_consensus.cc:2393] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e873e0e292554636a9f2813ef8ed92a3 in current term 1: Already voted for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in this term.
I20260709 13:08:01.105466 20003 leader_election.cc:304] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e873e0e292554636a9f2813ef8ed92a3; no voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899
I20260709 13:08:01.106549 20337 raft_consensus.cc:3060] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:01.114192 20347 consensus_queue.cc:1048] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.117924 20337 raft_consensus.cc:2749] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:01.121551 20264 consensus_queue.cc:1048] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.131142 20347 consensus_queue.cc:1048] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.144155 20390 consensus_queue.cc:1048] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.154457 20390 consensus_queue.cc:1048] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.166457 20369 consensus_queue.cc:1048] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.201129 20337 consensus_queue.cc:1048] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.211081 20188 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91ea530bda574cf78cdab4f5be267aee"
dest_uuid: "aeccd99157ea41799d2e4c1592044899"
mode: GRACEFUL
new_leader_uuid: "e873e0e292554636a9f2813ef8ed92a3"
 from {username='slave'} at 127.0.0.1:56648
I20260709 13:08:01.211962 20188 raft_consensus.cc:606] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Received request to transfer leadership to TS e873e0e292554636a9f2813ef8ed92a3
I20260709 13:08:01.213785 20370 consensus_queue.cc:1048] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.235332 20369 consensus_queue.cc:1048] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.241935 20337 consensus_queue.cc:1048] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:01.243297 20038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9a384eab89841d09ed111f5324ec40e"
dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
mode: GRACEFUL
new_leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
 from {username='slave'} at 127.0.0.1:53978
I20260709 13:08:01.244123 20038 raft_consensus.cc:606] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Received request to transfer leadership to TS 939a6adbfc6b4d49adda1ebec89d8d6e
I20260709 13:08:01.260535 20350 consensus_queue.cc:1048] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.261824 20186 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c86f5fdd6d564f4a9b4c0ddf14aa77bf"
dest_uuid: "aeccd99157ea41799d2e4c1592044899"
mode: GRACEFUL
new_leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
 from {username='slave'} at 127.0.0.1:56648
I20260709 13:08:01.262651 20186 raft_consensus.cc:606] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Received request to transfer leadership to TS 939a6adbfc6b4d49adda1ebec89d8d6e
I20260709 13:08:01.269718 20187 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22b23637181d474c910b37980c6b3169"
dest_uuid: "aeccd99157ea41799d2e4c1592044899"
mode: GRACEFUL
new_leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
 from {username='slave'} at 127.0.0.1:56648
I20260709 13:08:01.270278 20187 raft_consensus.cc:606] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Received request to transfer leadership to TS 939a6adbfc6b4d49adda1ebec89d8d6e
I20260709 13:08:01.271641 20320 raft_consensus.cc:993] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899: : Instructing follower 939a6adbfc6b4d49adda1ebec89d8d6e to start an election
I20260709 13:08:01.272167 20320 raft_consensus.cc:1081] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Signalling peer 939a6adbfc6b4d49adda1ebec89d8d6e to start an election
I20260709 13:08:01.274048 20112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22b23637181d474c910b37980c6b3169"
dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
 from {username='slave'} at 127.0.0.1:41196
I20260709 13:08:01.274796 20112 raft_consensus.cc:493] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:01.275207 20112 raft_consensus.cc:3060] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.278450 20114 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7834843f08a42e08a30e38b0ebb1d29"
dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
mode: GRACEFUL
new_leader_uuid: "aeccd99157ea41799d2e4c1592044899"
 from {username='slave'} at 127.0.0.1:41210
I20260709 13:08:01.279191 20114 raft_consensus.cc:606] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Received request to transfer leadership to TS aeccd99157ea41799d2e4c1592044899
I20260709 13:08:01.279249 20391 consensus_queue.cc:1048] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.284929 20112 raft_consensus.cc:515] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.291019 20114 raft_consensus.cc:1240] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Rejecting Update request from peer aeccd99157ea41799d2e4c1592044899 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:01.292196 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b23637181d474c910b37980c6b3169" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:08:01.293321 20187 raft_consensus.cc:3055] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:01.293701 20187 raft_consensus.cc:740] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:08:01.294597 20187 consensus_queue.cc:260] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.295776 20187 raft_consensus.cc:3060] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.296731 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b23637181d474c910b37980c6b3169" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:08:01.297508 20039 raft_consensus.cc:3060] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.305565 20112 leader_election.cc:290] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 2 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:01.306648 20187 raft_consensus.cc:2468] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 2.
I20260709 13:08:01.308360 20078 leader_election.cc:304] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: 
I20260709 13:08:01.309485 20369 raft_consensus.cc:2804] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:01.310173 20369 raft_consensus.cc:697] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 LEADER]: Becoming Leader. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:08:01.311471 20369 consensus_queue.cc:237] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.313112 20039 raft_consensus.cc:2468] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 2.
I20260709 13:08:01.320737 19933 catalog_manager.cc:5697] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e reported cstate change: term changed from 1 to 2, leader changed from aeccd99157ea41799d2e4c1592044899 (127.7.89.131) to 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130). New cstate: current_term: 2 leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:01.462455 20332 consensus_queue.cc:1048] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.465255 20332 consensus_queue.cc:1048] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:01.496558 20332 consensus_queue.cc:1048] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.505360 20337 raft_consensus.cc:993] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3: : Instructing follower 939a6adbfc6b4d49adda1ebec89d8d6e to start an election
I20260709 13:08:01.505921 20383 raft_consensus.cc:1081] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Signalling peer 939a6adbfc6b4d49adda1ebec89d8d6e to start an election
I20260709 13:08:01.508844 20112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9a384eab89841d09ed111f5324ec40e"
dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
 from {username='slave'} at 127.0.0.1:41190
I20260709 13:08:01.513443 20112 raft_consensus.cc:493] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:01.513425 20337 consensus_queue.cc:1048] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.514254 20112 raft_consensus.cc:3060] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.524045 20112 raft_consensus.cc:515] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.530378 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9a384eab89841d09ed111f5324ec40e" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:08:01.530668 20112 leader_election.cc:290] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 2 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:01.530455 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9a384eab89841d09ed111f5324ec40e" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:08:01.531219 20039 raft_consensus.cc:3055] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:01.531759 20039 raft_consensus.cc:740] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:08:01.531822 20187 raft_consensus.cc:3060] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.532891 20039 consensus_queue.cc:260] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.534385 20039 raft_consensus.cc:3060] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.541414 20187 raft_consensus.cc:2468] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 2.
I20260709 13:08:01.543188 20078 leader_election.cc:304] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: 
I20260709 13:08:01.544126 20339 raft_consensus.cc:2804] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:01.544466 20039 raft_consensus.cc:2468] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 2.
I20260709 13:08:01.544813 20339 raft_consensus.cc:697] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 LEADER]: Becoming Leader. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:08:01.545900 20339 consensus_queue.cc:237] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.554999 19934 catalog_manager.cc:5697] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e reported cstate change: term changed from 1 to 2, leader changed from e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129) to 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130). New cstate: current_term: 2 leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:01.656086 20339 raft_consensus.cc:993] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e: : Instructing follower aeccd99157ea41799d2e4c1592044899 to start an election
I20260709 13:08:01.656623 20369 raft_consensus.cc:1081] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Signalling peer aeccd99157ea41799d2e4c1592044899 to start an election
I20260709 13:08:01.658485 20187 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7834843f08a42e08a30e38b0ebb1d29"
dest_uuid: "aeccd99157ea41799d2e4c1592044899"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:08:01.659027 20187 raft_consensus.cc:493] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:01.659276 20187 raft_consensus.cc:3060] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.666126 20187 raft_consensus.cc:515] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.668133 20187 leader_election.cc:290] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 2 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:08:01.669319 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7834843f08a42e08a30e38b0ebb1d29" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:08:01.669525 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7834843f08a42e08a30e38b0ebb1d29" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:08:01.670143 20039 raft_consensus.cc:3060] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.670212 20112 raft_consensus.cc:3055] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:01.670800 20112 raft_consensus.cc:740] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:08:01.671634 20112 consensus_queue.cc:260] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.672818 20112 raft_consensus.cc:3060] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.680127 20039 raft_consensus.cc:2468] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 2.
I20260709 13:08:01.681677 20153 leader_election.cc:304] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:08:01.682610 20391 raft_consensus.cc:2804] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:01.683173 20391 raft_consensus.cc:697] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [term 2 LEADER]: Becoming Leader. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:08:01.684150 20112 raft_consensus.cc:2468] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 2.
I20260709 13:08:01.684036 20391 consensus_queue.cc:237] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.693241 19934 catalog_manager.cc:5697] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 reported cstate change: term changed from 1 to 2, leader changed from 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130) to aeccd99157ea41799d2e4c1592044899 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "aeccd99157ea41799d2e4c1592044899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:01.694350 20039 raft_consensus.cc:1275] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Refusing update from remote peer 939a6adbfc6b4d49adda1ebec89d8d6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:01.695994 20369 consensus_queue.cc:1048] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:01.706152 20187 raft_consensus.cc:1275] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Refusing update from remote peer 939a6adbfc6b4d49adda1ebec89d8d6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:01.710579 20369 consensus_queue.cc:1048] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:01.727311 20320 raft_consensus.cc:993] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899: : Instructing follower e873e0e292554636a9f2813ef8ed92a3 to start an election
I20260709 13:08:01.727834 20320 raft_consensus.cc:1081] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Signalling peer e873e0e292554636a9f2813ef8ed92a3 to start an election
I20260709 13:08:01.729586 20039 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91ea530bda574cf78cdab4f5be267aee"
dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
 from {username='slave'} at 127.0.0.1:53968
I20260709 13:08:01.730233 20039 raft_consensus.cc:493] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:01.730588 20039 raft_consensus.cc:3060] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.739756 20039 raft_consensus.cc:515] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.742503 20039 leader_election.cc:290] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 2 election: Requested vote from peers 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:01.743602 20112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ea530bda574cf78cdab4f5be267aee" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:08:01.743781 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ea530bda574cf78cdab4f5be267aee" candidate_uuid: "e873e0e292554636a9f2813ef8ed92a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:08:01.744560 20112 raft_consensus.cc:3060] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.744616 20187 raft_consensus.cc:3055] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:01.745090 20187 raft_consensus.cc:740] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:08:01.745807 20187 consensus_queue.cc:260] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.746932 20187 raft_consensus.cc:3060] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.751782 20112 raft_consensus.cc:2468] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 2.
I20260709 13:08:01.753368 20005 leader_election.cc:304] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:08:01.753368 20187 raft_consensus.cc:2468] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e873e0e292554636a9f2813ef8ed92a3 in term 2.
I20260709 13:08:01.754341 20337 raft_consensus.cc:2804] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:01.754908 20337 raft_consensus.cc:697] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 2 LEADER]: Becoming Leader. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:08:01.755832 20337 consensus_queue.cc:237] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.764367 19934 catalog_manager.cc:5697] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 reported cstate change: term changed from 1 to 2, leader changed from aeccd99157ea41799d2e4c1592044899 (127.7.89.131) to e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129). New cstate: current_term: 2 leader_uuid: "e873e0e292554636a9f2813ef8ed92a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:01.775672 20391 raft_consensus.cc:993] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899: : Instructing follower 939a6adbfc6b4d49adda1ebec89d8d6e to start an election
I20260709 13:08:01.776252 20391 raft_consensus.cc:1081] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Signalling peer 939a6adbfc6b4d49adda1ebec89d8d6e to start an election
I20260709 13:08:01.778149 20112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c86f5fdd6d564f4a9b4c0ddf14aa77bf"
dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
 from {username='slave'} at 127.0.0.1:41196
I20260709 13:08:01.778856 20112 raft_consensus.cc:493] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:01.779246 20112 raft_consensus.cc:3060] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.787626 20112 raft_consensus.cc:515] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.789891 20112 leader_election.cc:290] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 2 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661)
I20260709 13:08:01.791157 20187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86f5fdd6d564f4a9b4c0ddf14aa77bf" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aeccd99157ea41799d2e4c1592044899"
I20260709 13:08:01.791155 20039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86f5fdd6d564f4a9b4c0ddf14aa77bf" candidate_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:08:01.792104 20187 raft_consensus.cc:3055] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:01.792587 20039 raft_consensus.cc:3060] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.792749 20187 raft_consensus.cc:740] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:08:01.793651 20187 consensus_queue.cc:260] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.794984 20187 raft_consensus.cc:3060] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.801255 20039 raft_consensus.cc:2468] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 2.
I20260709 13:08:01.801822 20187 raft_consensus.cc:2468] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939a6adbfc6b4d49adda1ebec89d8d6e in term 2.
I20260709 13:08:01.803208 20078 leader_election.cc:304] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: 
I20260709 13:08:01.804188 20370 raft_consensus.cc:2804] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:01.804821 20370 raft_consensus.cc:697] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 LEADER]: Becoming Leader. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:08:01.805716 20370 consensus_queue.cc:237] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:01.814533 19934 catalog_manager.cc:5697] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e reported cstate change: term changed from 1 to 2, leader changed from aeccd99157ea41799d2e4c1592044899 (127.7.89.131) to 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130). New cstate: current_term: 2 leader_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:02.006461 20187 raft_consensus.cc:1275] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Refusing update from remote peer 939a6adbfc6b4d49adda1ebec89d8d6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:02.007915 20369 consensus_queue.cc:1048] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:02.020624 20039 raft_consensus.cc:1275] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Refusing update from remote peer 939a6adbfc6b4d49adda1ebec89d8d6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:02.022174 20370 consensus_queue.cc:1048] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:02.185125 20039 raft_consensus.cc:1275] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Refusing update from remote peer 939a6adbfc6b4d49adda1ebec89d8d6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:02.186512 20370 consensus_queue.cc:1048] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:02.198365 20187 raft_consensus.cc:1275] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Refusing update from remote peer 939a6adbfc6b4d49adda1ebec89d8d6e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:02.200143 20339 consensus_queue.cc:1048] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:02.224972 20187 raft_consensus.cc:1275] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Refusing update from remote peer e873e0e292554636a9f2813ef8ed92a3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:02.226424 20383 consensus_queue.cc:1048] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:02.243799 20112 raft_consensus.cc:1275] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Refusing update from remote peer aeccd99157ea41799d2e4c1592044899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:02.244513 20114 raft_consensus.cc:1275] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Refusing update from remote peer e873e0e292554636a9f2813ef8ed92a3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:02.245784 20391 consensus_queue.cc:1048] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:02.246255 20332 consensus_queue.cc:1048] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:02.260946 20039 raft_consensus.cc:1275] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Refusing update from remote peer aeccd99157ea41799d2e4c1592044899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:02.262782 20391 consensus_queue.cc:1048] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:04.287416 19934 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52178:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:08:04.309778 19934 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52178:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:08:04.344970  7526 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:08:04.506340 20449 tablet.cc:1724] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:08:04.555925 20447 log.cc:927] Time spent T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3: Append to log took a long time: real 0.098s	user 0.002s	sys 0.000s
W20260709 13:08:04.643730 20140 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:c9a384eab89841d09ed111f5324ec40e)
W20260709 13:08:04.739514 20140 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:c9a384eab89841d09ed111f5324ec40e)
W20260709 13:08:04.747382 20140 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:c9a384eab89841d09ed111f5324ec40e)
I20260709 13:08:04.762336 20473 tablet.cc:1724] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:04.682235 20453 tablet.cc:1724] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:04.807840 20479 tablet.cc:1724] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:04.833674 20476 tablet.cc:1724] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:04.842988 20484 tablet.cc:1724] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:04.768456 20471 tablet.cc:1724] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:04.897349 20110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22b23637181d474c910b37980c6b3169"
dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
mode: GRACEFUL
new_leader_uuid: "aeccd99157ea41799d2e4c1592044899"
 from {username='slave'} at 127.0.0.1:34976
I20260709 13:08:04.898296 20110 raft_consensus.cc:606] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 LEADER]: Received request to transfer leadership to TS aeccd99157ea41799d2e4c1592044899
I20260709 13:08:04.846271 20482 tablet.cc:1724] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:04.846940 20481 tablet.cc:1724] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:04.926476 20487 tablet.cc:1724] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:04.929215 20449 tablet.cc:1724] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:04.947826 20481 tablet.cc:1724] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:04.965687 20445 raft_consensus.cc:993] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e: : Instructing follower aeccd99157ea41799d2e4c1592044899 to start an election
I20260709 13:08:04.966766 20438 raft_consensus.cc:1081] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 LEADER]: Signalling peer aeccd99157ea41799d2e4c1592044899 to start an election
I20260709 13:08:04.978547 20476 tablet.cc:1724] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:05.004630 20484 tablet.cc:1724] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:05.013973 20487 tablet.cc:1724] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:05.017133 20182 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22b23637181d474c910b37980c6b3169"
dest_uuid: "aeccd99157ea41799d2e4c1592044899"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:08:05.018021 20182 raft_consensus.cc:493] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:05.018497 20182 raft_consensus.cc:3060] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.004551 20482 tablet.cc:1724] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:05.021905 19933 catalog_manager.cc:5789] TS 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545) does not have the latest schema for tablet 4b2833da36fe4d26a5e8bd292db5865d (table soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]). Expected version 1 got 0
I20260709 13:08:05.022347 20449 tablet.cc:1724] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:04.993129 20453 tablet.cc:1724] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:05.030869 20489 tablet.cc:1724] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:05.043076 20486 tablet.cc:1724] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:05.053500 20471 tablet.cc:1724] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:04.992479 20473 tablet.cc:1724] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:05.069686 20489 tablet.cc:1724] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:05.082676 20182 raft_consensus.cc:515] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:05.067960 20479 tablet.cc:1724] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:05.079536 19934 catalog_manager.cc:5789] TS e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539) does not have the latest schema for tablet 91ea530bda574cf78cdab4f5be267aee (table soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]). Expected version 1 got 0
I20260709 13:08:05.094414 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b23637181d474c910b37980c6b3169" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:08:05.095145 20114 raft_consensus.cc:3055] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:05.095481 20114 raft_consensus.cc:740] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:08:05.096500 20114 consensus_queue.cc:260] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:05.097790 20114 raft_consensus.cc:3060] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.099081 20036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b23637181d474c910b37980c6b3169" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:08:05.099896 20036 raft_consensus.cc:3060] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.101995 20481 tablet.cc:1724] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:05.104315 20482 tablet.cc:1724] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:05.110767 20182 leader_election.cc:290] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 3 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:08:05.117470 20114 raft_consensus.cc:2468] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 3.
I20260709 13:08:05.123658 20154 leader_election.cc:304] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: 
I20260709 13:08:05.125275 20414 raft_consensus.cc:2804] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:05.125882 20414 raft_consensus.cc:697] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 3 LEADER]: Becoming Leader. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:08:05.126818 20414 consensus_queue.cc:237] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:05.137701 20486 tablet.cc:1724] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:05.138026 19933 catalog_manager.cc:5697] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 reported cstate change: term changed from 2 to 3, leader changed from 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130) to aeccd99157ea41799d2e4c1592044899 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "aeccd99157ea41799d2e4c1592044899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:05.140516 19933 catalog_manager.cc:5789] TS aeccd99157ea41799d2e4c1592044899 (127.7.89.131:46661) does not have the latest schema for tablet 22b23637181d474c910b37980c6b3169 (table soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d]). Expected version 1 got 0
I20260709 13:08:05.146404 20039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91ea530bda574cf78cdab4f5be267aee"
dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
mode: GRACEFUL
new_leader_uuid: "aeccd99157ea41799d2e4c1592044899"
 from {username='slave'} at 127.0.0.1:41188
I20260709 13:08:05.147259 20039 raft_consensus.cc:606] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 2 LEADER]: Received request to transfer leadership to TS aeccd99157ea41799d2e4c1592044899
I20260709 13:08:05.148914  7526 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:08:05.149696  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:05.156373 20036 raft_consensus.cc:2468] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 3.
I20260709 13:08:05.161887 20036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba6459261f244c929626958cdd5124c8"
dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
mode: GRACEFUL
new_leader_uuid: "aeccd99157ea41799d2e4c1592044899"
 from {username='slave'} at 127.0.0.1:41188
I20260709 13:08:05.162638 20036 raft_consensus.cc:606] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 2 LEADER]: Received request to transfer leadership to TS aeccd99157ea41799d2e4c1592044899
I20260709 13:08:05.166013 20114 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fdb29224078847b6b7344b8a4baada34"
dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
mode: GRACEFUL
new_leader_uuid: "aeccd99157ea41799d2e4c1592044899"
 from {username='slave'} at 127.0.0.1:34976
I20260709 13:08:05.166716 20114 raft_consensus.cc:606] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 LEADER]: Received request to transfer leadership to TS aeccd99157ea41799d2e4c1592044899
I20260709 13:08:05.168121  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:08:05.169085  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:05.176445 19934 catalog_manager.cc:6242] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47: soft_deleted_table [id=f4f844d3dccd44d9a81ac96f8602bb1d] alter complete (version 1)
I20260709 13:08:05.203771 20446 raft_consensus.cc:993] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3: : Instructing follower aeccd99157ea41799d2e4c1592044899 to start an election
I20260709 13:08:05.204341 20446 raft_consensus.cc:1081] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 2 LEADER]: Signalling peer aeccd99157ea41799d2e4c1592044899 to start an election
I20260709 13:08:05.206965 20184 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91ea530bda574cf78cdab4f5be267aee"
dest_uuid: "aeccd99157ea41799d2e4c1592044899"
 from {username='slave'} at 127.0.0.1:56642
I20260709 13:08:05.207602 20184 raft_consensus.cc:493] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:05.207978 20184 raft_consensus.cc:3060] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.214934 20184 raft_consensus.cc:515] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:05.217656 20184 leader_election.cc:290] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 3 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:08:05.218019 20036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ea530bda574cf78cdab4f5be267aee" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:08:05.218838 20036 raft_consensus.cc:3055] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:05.219259 20036 raft_consensus.cc:740] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:08:05.220366 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ea530bda574cf78cdab4f5be267aee" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:08:05.220207 20036 consensus_queue.cc:260] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:05.221107 20114 raft_consensus.cc:3060] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.221649 20036 raft_consensus.cc:3060] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.227937 20114 raft_consensus.cc:2468] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 3.
I20260709 13:08:05.228610 20036 raft_consensus.cc:2468] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 3.
I20260709 13:08:05.229497 20154 leader_election.cc:304] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: 
I20260709 13:08:05.230594 20414 raft_consensus.cc:2804] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:05.231170 20414 raft_consensus.cc:697] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 3 LEADER]: Becoming Leader. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:08:05.232110 20414 consensus_queue.cc:237] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:05.240145 19935 catalog_manager.cc:5697] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 reported cstate change: term changed from 2 to 3, leader changed from e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129) to aeccd99157ea41799d2e4c1592044899 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "aeccd99157ea41799d2e4c1592044899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:05.384351 20415 raft_consensus.cc:993] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3: : Instructing follower aeccd99157ea41799d2e4c1592044899 to start an election
I20260709 13:08:05.384950 20415 raft_consensus.cc:1081] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 2 LEADER]: Signalling peer aeccd99157ea41799d2e4c1592044899 to start an election
I20260709 13:08:05.387388 20182 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba6459261f244c929626958cdd5124c8"
dest_uuid: "aeccd99157ea41799d2e4c1592044899"
 from {username='slave'} at 127.0.0.1:56642
I20260709 13:08:05.388115 20182 raft_consensus.cc:493] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:05.388521 20182 raft_consensus.cc:3060] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.399838 20182 raft_consensus.cc:515] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:05.402355 20182 leader_election.cc:290] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 3 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:08:05.403832 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba6459261f244c929626958cdd5124c8" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:08:05.404673 20114 raft_consensus.cc:3060] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.405289 20036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba6459261f244c929626958cdd5124c8" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:08:05.406047 20036 raft_consensus.cc:3055] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:05.406430 20036 raft_consensus.cc:740] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e873e0e292554636a9f2813ef8ed92a3, State: Running, Role: LEADER
I20260709 13:08:05.408178 20036 consensus_queue.cc:260] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:05.409456 20036 raft_consensus.cc:3060] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.413585 20114 raft_consensus.cc:2468] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 3.
I20260709 13:08:05.414542 20111 raft_consensus.cc:1240] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Rejecting Update request from peer e873e0e292554636a9f2813ef8ed92a3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:05.417570 20154 leader_election.cc:304] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939a6adbfc6b4d49adda1ebec89d8d6e, aeccd99157ea41799d2e4c1592044899; no voters: 
I20260709 13:08:05.418671 20414 raft_consensus.cc:2804] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:05.419198 20414 raft_consensus.cc:697] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 3 LEADER]: Becoming Leader. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:08:05.420187 20414 consensus_queue.cc:237] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:05.420594 20036 raft_consensus.cc:2468] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 3.
I20260709 13:08:05.430212 19935 catalog_manager.cc:5697] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 reported cstate change: term changed from 2 to 3, leader changed from e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129) to aeccd99157ea41799d2e4c1592044899 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "aeccd99157ea41799d2e4c1592044899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:05.620074 20114 raft_consensus.cc:1275] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Refusing update from remote peer aeccd99157ea41799d2e4c1592044899: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:08:05.621649 20414 consensus_queue.cc:1048] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:05.632948 20036 raft_consensus.cc:1275] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Refusing update from remote peer aeccd99157ea41799d2e4c1592044899: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:08:05.635305 20496 consensus_queue.cc:1048] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:05.671361 20485 raft_consensus.cc:993] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e: : Instructing follower aeccd99157ea41799d2e4c1592044899 to start an election
I20260709 13:08:05.672151 20445 raft_consensus.cc:1081] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 LEADER]: Signalling peer aeccd99157ea41799d2e4c1592044899 to start an election
I20260709 13:08:05.674105 20182 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fdb29224078847b6b7344b8a4baada34"
dest_uuid: "aeccd99157ea41799d2e4c1592044899"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:08:05.674751 20182 raft_consensus.cc:493] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:05.675072 20182 raft_consensus.cc:3060] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.681633 20182 raft_consensus.cc:515] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:05.684003 20182 leader_election.cc:290] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 3 election: Requested vote from peers e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539), 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545)
I20260709 13:08:05.684964 20036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb29224078847b6b7344b8a4baada34" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e873e0e292554636a9f2813ef8ed92a3"
I20260709 13:08:05.685767 20036 raft_consensus.cc:3060] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.686058 20114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdb29224078847b6b7344b8a4baada34" candidate_uuid: "aeccd99157ea41799d2e4c1592044899" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e"
I20260709 13:08:05.686767 20114 raft_consensus.cc:3055] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:05.687108 20114 raft_consensus.cc:740] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 939a6adbfc6b4d49adda1ebec89d8d6e, State: Running, Role: LEADER
I20260709 13:08:05.687791 20039 raft_consensus.cc:1275] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Refusing update from remote peer aeccd99157ea41799d2e4c1592044899: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:08:05.687956 20114 consensus_queue.cc:260] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:05.689419 20114 raft_consensus.cc:3060] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.689669 20496 consensus_queue.cc:1048] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:05.694694 20036 raft_consensus.cc:2468] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 3.
I20260709 13:08:05.696271 20153 leader_election.cc:304] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeccd99157ea41799d2e4c1592044899, e873e0e292554636a9f2813ef8ed92a3; no voters: 
I20260709 13:08:05.697121 20496 raft_consensus.cc:2804] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:05.697956 20496 raft_consensus.cc:697] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [term 3 LEADER]: Becoming Leader. State: Replica: aeccd99157ea41799d2e4c1592044899, State: Running, Role: LEADER
I20260709 13:08:05.700131 20114 raft_consensus.cc:2468] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeccd99157ea41799d2e4c1592044899 in term 3.
I20260709 13:08:05.702648 20496 consensus_queue.cc:237] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } }
I20260709 13:08:05.703488 20114 raft_consensus.cc:1275] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Refusing update from remote peer aeccd99157ea41799d2e4c1592044899: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:08:05.707760 20500 consensus_queue.cc:1048] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:05.716423 19935 catalog_manager.cc:5697] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 reported cstate change: term changed from 2 to 3, leader changed from 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130) to aeccd99157ea41799d2e4c1592044899 (127.7.89.131). New cstate: current_term: 3 leader_uuid: "aeccd99157ea41799d2e4c1592044899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeccd99157ea41799d2e4c1592044899" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 46661 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:05.821273 20114 raft_consensus.cc:1275] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Refusing update from remote peer aeccd99157ea41799d2e4c1592044899: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:05.822837 20500 consensus_queue.cc:1048] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:05.835467 20036 raft_consensus.cc:1275] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Refusing update from remote peer aeccd99157ea41799d2e4c1592044899: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:05.837594 20500 consensus_queue.cc:1048] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:06.169310 20036 raft_consensus.cc:1275] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Refusing update from remote peer aeccd99157ea41799d2e4c1592044899: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:06.170905 20500 consensus_queue.cc:1048] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e873e0e292554636a9f2813ef8ed92a3" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:06.187119 20114 raft_consensus.cc:1275] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Refusing update from remote peer aeccd99157ea41799d2e4c1592044899: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:06.195868 20463 consensus_queue.cc:1048] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939a6adbfc6b4d49adda1ebec89d8d6e" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:06.256978  7526 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:08:06.263120  7526 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:08:06.263590  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:06.268409  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:08:06.269009  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:07.306834  7526 tablet_server.cc:179] TabletServer@127.7.89.129:0 shutting down...
W20260709 13:08:07.332913 20079 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 8 similar messages]
W20260709 13:08:07.339326 20079 consensus_peers.cc:597] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e -> Peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Couldn't send request to peer e873e0e292554636a9f2813ef8ed92a3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:07.347388  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:07.348332  7526 tablet_replica.cc:333] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.349035  7526 raft_consensus.cc:2243] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.349629  7526 raft_consensus.cc:2272] T 22b23637181d474c910b37980c6b3169 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.352178  7526 tablet_replica.cc:333] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.352900  7526 raft_consensus.cc:2243] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.353430  7526 raft_consensus.cc:2272] T c9a384eab89841d09ed111f5324ec40e P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.355590  7526 tablet_replica.cc:333] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.356225  7526 raft_consensus.cc:2243] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.356693  7526 raft_consensus.cc:2272] T 91ea530bda574cf78cdab4f5be267aee P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.358824  7526 tablet_replica.cc:333] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.359588  7526 raft_consensus.cc:2243] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.360607  7526 raft_consensus.cc:2272] T 21d59234792f41bc8721313e837adf16 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.362805  7526 tablet_replica.cc:333] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.363458  7526 raft_consensus.cc:2243] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.363946  7526 raft_consensus.cc:2272] T c7834843f08a42e08a30e38b0ebb1d29 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:07.364374 20153 consensus_peers.cc:597] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 -> Peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Couldn't send request to peer e873e0e292554636a9f2813ef8ed92a3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:07.366034  7526 tablet_replica.cc:333] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.366879  7526 raft_consensus.cc:2243] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.367761  7526 raft_consensus.cc:2272] T 5e468afda36b481688411ab728d7aa74 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.369967  7526 tablet_replica.cc:333] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.370697  7526 raft_consensus.cc:2243] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.371213  7526 raft_consensus.cc:2272] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.373605  7526 tablet_replica.cc:333] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.374255  7526 raft_consensus.cc:2243] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.375454  7526 raft_consensus.cc:2272] T 91501c5709394cedbd76b094e9d8044b P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.378047  7526 tablet_replica.cc:333] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.378803  7526 raft_consensus.cc:2243] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.379735  7526 raft_consensus.cc:2272] T a6e77f8f8ac4417ca238c2951bdf9ce5 P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.382092  7526 tablet_replica.cc:333] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.382757  7526 raft_consensus.cc:2243] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.383257  7526 raft_consensus.cc:2272] T 2647c183b98c49c2ae36c3441b258086 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.385571  7526 tablet_replica.cc:333] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.386204  7526 raft_consensus.cc:2243] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.387189  7526 raft_consensus.cc:2272] T c74c3cf888194042a7ba99ed2a7bb576 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.389418  7526 tablet_replica.cc:333] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.390080  7526 raft_consensus.cc:2243] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.391074  7526 raft_consensus.cc:2272] T 6392bb50c7e748a9b4fd371a6e59853e P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.393428  7526 tablet_replica.cc:333] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.394042  7526 raft_consensus.cc:2243] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.394591  7526 raft_consensus.cc:2272] T 4b2833da36fe4d26a5e8bd292db5865d P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.396545  7526 tablet_replica.cc:333] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.397188  7526 raft_consensus.cc:2243] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.397717  7526 raft_consensus.cc:2272] T fdb29224078847b6b7344b8a4baada34 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.400380  7526 tablet_replica.cc:333] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.401014  7526 raft_consensus.cc:2243] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.401476  7526 raft_consensus.cc:2272] T ba6459261f244c929626958cdd5124c8 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.403923  7526 tablet_replica.cc:333] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.404560  7526 raft_consensus.cc:2243] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.405043  7526 raft_consensus.cc:2272] T a96c1cd6fac4472492f90a5d96208c66 P e873e0e292554636a9f2813ef8ed92a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.407860  7526 tablet_replica.cc:333] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.408509  7526 raft_consensus.cc:2243] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.409039  7526 raft_consensus.cc:2272] T 21c499bdb1fe4703ad7bafda1314b22b P e873e0e292554636a9f2813ef8ed92a3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.411536  7526 tablet_replica.cc:333] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3: stopping tablet replica
I20260709 13:08:07.412127  7526 raft_consensus.cc:2243] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.412689  7526 raft_consensus.cc:2272] T 7d36135d0e0d425a83c860422de70367 P e873e0e292554636a9f2813ef8ed92a3 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:07.419298 20079 consensus_peers.cc:597] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e -> Peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Couldn't send request to peer e873e0e292554636a9f2813ef8ed92a3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:07.455013 20153 consensus_peers.cc:597] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 -> Peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Couldn't send request to peer e873e0e292554636a9f2813ef8ed92a3. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:38539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:07.460915  7526 tablet_server.cc:196] TabletServer@127.7.89.129:0 shutdown complete.
W20260709 13:08:07.485569 20153 consensus_peers.cc:597] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 -> Peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Couldn't send request to peer e873e0e292554636a9f2813ef8ed92a3. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:38539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:07.503640  7526 tablet_server.cc:179] TabletServer@127.7.89.130:0 shutting down...
W20260709 13:08:07.544435 20154 consensus_peers.cc:597] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 -> Peer 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545): Couldn't send request to peer 939a6adbfc6b4d49adda1ebec89d8d6e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:07.546273  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:07.547307  7526 tablet_replica.cc:333] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.548400  7526 raft_consensus.cc:2243] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.549206  7526 raft_consensus.cc:2272] T 91ea530bda574cf78cdab4f5be267aee P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.551766  7526 tablet_replica.cc:333] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.552435  7526 raft_consensus.cc:2243] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.552928  7526 raft_consensus.cc:2272] T 21d59234792f41bc8721313e837adf16 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.555135  7526 tablet_replica.cc:333] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.555745  7526 raft_consensus.cc:2243] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.556352  7526 raft_consensus.cc:2272] T c7834843f08a42e08a30e38b0ebb1d29 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.559206  7526 tablet_replica.cc:333] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.560048  7526 raft_consensus.cc:2243] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.560760  7526 raft_consensus.cc:2272] T 22b23637181d474c910b37980c6b3169 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.563335  7526 tablet_replica.cc:333] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.564004  7526 raft_consensus.cc:2243] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.564883  7526 raft_consensus.cc:2272] T c9a384eab89841d09ed111f5324ec40e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.567198  7526 tablet_replica.cc:333] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.567812  7526 raft_consensus.cc:2243] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.568284  7526 raft_consensus.cc:2272] T 5e468afda36b481688411ab728d7aa74 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.570308  7526 tablet_replica.cc:333] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.570995  7526 raft_consensus.cc:2243] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.571449  7526 raft_consensus.cc:2272] T c74c3cf888194042a7ba99ed2a7bb576 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.573484  7526 tablet_replica.cc:333] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.574069  7526 raft_consensus.cc:2243] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.574493  7526 raft_consensus.cc:2272] T 6392bb50c7e748a9b4fd371a6e59853e P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.576934  7526 tablet_replica.cc:333] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.577620  7526 raft_consensus.cc:2243] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.578554  7526 raft_consensus.cc:2272] T 7d36135d0e0d425a83c860422de70367 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.581470  7526 tablet_replica.cc:333] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.582122  7526 raft_consensus.cc:2243] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.582617  7526 raft_consensus.cc:2272] T 21c499bdb1fe4703ad7bafda1314b22b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.584887  7526 tablet_replica.cc:333] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.585464  7526 raft_consensus.cc:2243] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.586336  7526 raft_consensus.cc:2272] T 2647c183b98c49c2ae36c3441b258086 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.588660  7526 tablet_replica.cc:333] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.589200  7526 raft_consensus.cc:2243] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.590073  7526 raft_consensus.cc:2272] T 4b2833da36fe4d26a5e8bd292db5865d P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.592227  7526 tablet_replica.cc:333] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.592806  7526 raft_consensus.cc:2243] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.593232  7526 raft_consensus.cc:2272] T fdb29224078847b6b7344b8a4baada34 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.595474  7526 tablet_replica.cc:333] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.596199  7526 raft_consensus.cc:2243] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.597256  7526 raft_consensus.cc:2272] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.599488  7526 tablet_replica.cc:333] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.600008  7526 raft_consensus.cc:2243] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.600433  7526 raft_consensus.cc:2272] T 91501c5709394cedbd76b094e9d8044b P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.602764  7526 tablet_replica.cc:333] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.603302  7526 raft_consensus.cc:2243] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.603782  7526 raft_consensus.cc:2272] T a6e77f8f8ac4417ca238c2951bdf9ce5 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.606020  7526 tablet_replica.cc:333] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.606527  7526 raft_consensus.cc:2243] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.607441  7526 raft_consensus.cc:2272] T a96c1cd6fac4472492f90a5d96208c66 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.609745  7526 tablet_replica.cc:333] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e: stopping tablet replica
I20260709 13:08:07.610221  7526 raft_consensus.cc:2243] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.610700  7526 raft_consensus.cc:2272] T ba6459261f244c929626958cdd5124c8 P 939a6adbfc6b4d49adda1ebec89d8d6e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.654805  7526 tablet_server.cc:196] TabletServer@127.7.89.130:0 shutdown complete.
W20260709 13:08:07.657637 20153 consensus_peers.cc:597] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 -> Peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Couldn't send request to peer e873e0e292554636a9f2813ef8ed92a3. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:38539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:07.658465 20154 consensus_peers.cc:597] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 -> Peer 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545): Couldn't send request to peer 939a6adbfc6b4d49adda1ebec89d8d6e. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:41545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:07.662758 20153 consensus_peers.cc:597] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 -> Peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Couldn't send request to peer e873e0e292554636a9f2813ef8ed92a3. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:38539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:07.686733 20153 consensus_peers.cc:597] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 -> Peer e873e0e292554636a9f2813ef8ed92a3 (127.7.89.129:38539): Couldn't send request to peer e873e0e292554636a9f2813ef8ed92a3. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:38539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:07.695660  7526 tablet_server.cc:179] TabletServer@127.7.89.131:0 shutting down...
I20260709 13:08:07.768467  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:07.769996  7526 tablet_replica.cc:333] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.771890  7526 raft_consensus.cc:2243] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.773047  7526 raft_consensus.cc:2272] T c9a384eab89841d09ed111f5324ec40e P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.777168  7526 tablet_replica.cc:333] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.778559  7526 raft_consensus.cc:2243] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.780601  7526 raft_consensus.cc:2272] T 22b23637181d474c910b37980c6b3169 P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.786944  7526 tablet_replica.cc:333] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.788640  7526 raft_consensus.cc:2243] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.792069  7526 raft_consensus.cc:2272] T 91ea530bda574cf78cdab4f5be267aee P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:07.792761 20154 consensus_peers.cc:597] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 -> Peer 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545): Couldn't send request to peer 939a6adbfc6b4d49adda1ebec89d8d6e. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:41545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:07.796208  7526 tablet_replica.cc:333] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.797668  7526 raft_consensus.cc:2243] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.798774  7526 raft_consensus.cc:2272] T 5e468afda36b481688411ab728d7aa74 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.802826  7526 tablet_replica.cc:333] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.804020  7526 raft_consensus.cc:2243] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.806344  7526 raft_consensus.cc:2272] T c7834843f08a42e08a30e38b0ebb1d29 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.810213  7526 tablet_replica.cc:333] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.811132  7526 raft_consensus.cc:2243] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.812076  7526 raft_consensus.cc:2272] T 21d59234792f41bc8721313e837adf16 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.815578  7526 tablet_replica.cc:333] T 2647c183b98c49c2ae36c3441b258086 P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.816335  7526 raft_consensus.cc:2243] T 2647c183b98c49c2ae36c3441b258086 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.817165  7526 raft_consensus.cc:2272] T 2647c183b98c49c2ae36c3441b258086 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.820962  7526 tablet_replica.cc:333] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.821823  7526 raft_consensus.cc:2243] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.822765  7526 raft_consensus.cc:2272] T 4b2833da36fe4d26a5e8bd292db5865d P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.826264  7526 tablet_replica.cc:333] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.827100  7526 raft_consensus.cc:2243] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.828752  7526 raft_consensus.cc:2272] T fdb29224078847b6b7344b8a4baada34 P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.832688  7526 tablet_replica.cc:333] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.833596  7526 raft_consensus.cc:2243] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.835037  7526 raft_consensus.cc:2272] T c74c3cf888194042a7ba99ed2a7bb576 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.840060  7526 tablet_replica.cc:333] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.841068  7526 raft_consensus.cc:2243] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:07.841369 20154 consensus_peers.cc:597] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 -> Peer 939a6adbfc6b4d49adda1ebec89d8d6e (127.7.89.130:41545): Couldn't send request to peer 939a6adbfc6b4d49adda1ebec89d8d6e. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:41545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:07.842262  7526 raft_consensus.cc:2272] T 6392bb50c7e748a9b4fd371a6e59853e P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.845702  7526 tablet_replica.cc:333] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.846504  7526 raft_consensus.cc:2243] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.847708  7526 raft_consensus.cc:2272] T 7d36135d0e0d425a83c860422de70367 P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.850955  7526 tablet_replica.cc:333] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.851693  7526 raft_consensus.cc:2243] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.853464  7526 raft_consensus.cc:2272] T 21c499bdb1fe4703ad7bafda1314b22b P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.857010  7526 tablet_replica.cc:333] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.857805  7526 raft_consensus.cc:2243] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.858713  7526 raft_consensus.cc:2272] T c86f5fdd6d564f4a9b4c0ddf14aa77bf P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.861840  7526 tablet_replica.cc:333] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.862536  7526 raft_consensus.cc:2243] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.863328  7526 raft_consensus.cc:2272] T 91501c5709394cedbd76b094e9d8044b P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.866344  7526 tablet_replica.cc:333] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.867055  7526 raft_consensus.cc:2243] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.867789  7526 raft_consensus.cc:2272] T a6e77f8f8ac4417ca238c2951bdf9ce5 P aeccd99157ea41799d2e4c1592044899 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.870756  7526 tablet_replica.cc:333] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.871404  7526 raft_consensus.cc:2243] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:07.872082  7526 raft_consensus.cc:2272] T a96c1cd6fac4472492f90a5d96208c66 P aeccd99157ea41799d2e4c1592044899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.875268  7526 tablet_replica.cc:333] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899: stopping tablet replica
I20260709 13:08:07.876062  7526 raft_consensus.cc:2243] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.877502  7526 raft_consensus.cc:2272] T ba6459261f244c929626958cdd5124c8 P aeccd99157ea41799d2e4c1592044899 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.946007  7526 tablet_server.cc:196] TabletServer@127.7.89.131:0 shutdown complete.
I20260709 13:08:08.006601  7526 master.cc:562] Master@127.7.89.190:32917 shutting down...
I20260709 13:08:08.036360  7526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:08.037201  7526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:08.037559  7526 tablet_replica.cc:333] T 00000000000000000000000000000000 P ecdbb3aebe6a4cdba0bf3720a8654c47: stopping tablet replica
I20260709 13:08:08.062724  7526 master.cc:584] Master@127.7.89.190:32917 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (13663 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:08:08.132740  7526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.89.190:40917
I20260709 13:08:08.134173  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:08.141769 20514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:08.141889 20513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:08.142355 20516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:08.143262  7526 server_base.cc:1061] running on GCE node
I20260709 13:08:08.144374  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:08.144626  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:08.144824  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602488144800 us; error 0 us; skew 500 ppm
I20260709 13:08:08.145394  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:08.150197  7526 webserver.cc:533] Webserver started at http://127.7.89.190:40717/ using document root <none> and password file <none>
I20260709 13:08:08.150904  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:08.151144  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:08.151451  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:08.152971  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/master-0-root/instance:
uuid: "6f0c6ba509b540dd9c2f43ef3431126a"
format_stamp: "Formatted at 2026-07-09 13:08:08 on dist-test-slave-h0rz"
I20260709 13:08:08.159082  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 13:08:08.163719 20521 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:08.164815  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:08:08.165194  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/master-0-root
uuid: "6f0c6ba509b540dd9c2f43ef3431126a"
format_stamp: "Formatted at 2026-07-09 13:08:08 on dist-test-slave-h0rz"
I20260709 13:08:08.165834  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:08.189299  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:08.190861  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:08.244997  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.190:40917
I20260709 13:08:08.245126 20572 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.190:40917 every 8 connection(s)
I20260709 13:08:08.250458 20573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.266877 20573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a: Bootstrap starting.
I20260709 13:08:08.273227 20573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.278703 20573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a: No bootstrap required, opened a new log
I20260709 13:08:08.281534 20573 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f0c6ba509b540dd9c2f43ef3431126a" member_type: VOTER }
I20260709 13:08:08.282059 20573 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.282343 20573 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f0c6ba509b540dd9c2f43ef3431126a, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.283069 20573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f0c6ba509b540dd9c2f43ef3431126a" member_type: VOTER }
I20260709 13:08:08.283636 20573 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:08:08.283890 20573 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:08:08.284192 20573 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.290169 20573 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f0c6ba509b540dd9c2f43ef3431126a" member_type: VOTER }
I20260709 13:08:08.290834 20573 leader_election.cc:304] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f0c6ba509b540dd9c2f43ef3431126a; no voters: 
I20260709 13:08:08.292135 20573 leader_election.cc:290] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:08:08.292557 20576 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:08.294214 20576 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [term 1 LEADER]: Becoming Leader. State: Replica: 6f0c6ba509b540dd9c2f43ef3431126a, State: Running, Role: LEADER
I20260709 13:08:08.295055 20576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f0c6ba509b540dd9c2f43ef3431126a" member_type: VOTER }
I20260709 13:08:08.295468 20573 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:08:08.302330 20578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f0c6ba509b540dd9c2f43ef3431126a. Latest consensus state: current_term: 1 leader_uuid: "6f0c6ba509b540dd9c2f43ef3431126a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f0c6ba509b540dd9c2f43ef3431126a" member_type: VOTER } }
I20260709 13:08:08.303541 20578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:08.303838 20577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f0c6ba509b540dd9c2f43ef3431126a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f0c6ba509b540dd9c2f43ef3431126a" member_type: VOTER } }
I20260709 13:08:08.304596 20577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:08.306298 20585 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:08:08.313761 20585 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:08:08.314977  7526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:08:08.324415 20585 catalog_manager.cc:1383] Generated new cluster ID: 28bfee30d61d4371915bbd37210d5b46
I20260709 13:08:08.324725 20585 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:08:08.344924 20585 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:08:08.346690 20585 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:08:08.367331 20585 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a: Generated new TSK 0
I20260709 13:08:08.368086 20585 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:08:08.382220  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:08.390129 20594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:08.390519 20595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:08.393899  7526 server_base.cc:1061] running on GCE node
W20260709 13:08:08.394279 20597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:08.395471  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:08.395689  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:08.395824  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602488395812 us; error 0 us; skew 500 ppm
I20260709 13:08:08.396347  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:08.399112  7526 webserver.cc:533] Webserver started at http://127.7.89.129:38207/ using document root <none> and password file <none>
I20260709 13:08:08.399715  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:08.399911  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:08.400146  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:08.401500  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-0-root/instance:
uuid: "fabb72df3c844f928dc062705b599484"
format_stamp: "Formatted at 2026-07-09 13:08:08 on dist-test-slave-h0rz"
I20260709 13:08:08.406924  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 13:08:08.411170 20602 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:08.412034  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.004s
I20260709 13:08:08.412405  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-0-root
uuid: "fabb72df3c844f928dc062705b599484"
format_stamp: "Formatted at 2026-07-09 13:08:08 on dist-test-slave-h0rz"
I20260709 13:08:08.412699  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:08.438138  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:08.439713  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:08.441429  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:08.444252  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:08.444469  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:08.444800  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:08.444968  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:08.495584  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.129:45101
I20260709 13:08:08.495718 20664 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.129:45101 every 8 connection(s)
I20260709 13:08:08.502385  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:08.511402 20669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:08.512789 20670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:08.516299 20672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:08.517249  7526 server_base.cc:1061] running on GCE node
I20260709 13:08:08.517676 20665 heartbeater.cc:344] Connected to a master server at 127.7.89.190:40917
I20260709 13:08:08.518237 20665 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:08.519038  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:08.519299  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:08.519359 20665 heartbeater.cc:507] Master 127.7.89.190:40917 requested a full tablet report, sending...
I20260709 13:08:08.519582  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602488519557 us; error 0 us; skew 500 ppm
I20260709 13:08:08.520362  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:08.522493 20538 ts_manager.cc:194] Registered new tserver with Master: fabb72df3c844f928dc062705b599484 (127.7.89.129:45101)
I20260709 13:08:08.523815  7526 webserver.cc:533] Webserver started at http://127.7.89.130:41991/ using document root <none> and password file <none>
I20260709 13:08:08.524662  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:08.524943  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:08.525275  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:08.525589 20538 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37296
I20260709 13:08:08.527349  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-1-root/instance:
uuid: "9d91f828e5fa453982afb0ada2c45170"
format_stamp: "Formatted at 2026-07-09 13:08:08 on dist-test-slave-h0rz"
I20260709 13:08:08.532925  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:08:08.537230 20677 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:08.538371  7526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:08:08.538838  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-1-root
uuid: "9d91f828e5fa453982afb0ada2c45170"
format_stamp: "Formatted at 2026-07-09 13:08:08 on dist-test-slave-h0rz"
I20260709 13:08:08.539187  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:08.559291  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:08.560870  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:08.562742  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:08.565728  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:08.566009  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:08.566284  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:08.566471  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:08.619771  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.130:42475
I20260709 13:08:08.619892 20739 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.130:42475 every 8 connection(s)
I20260709 13:08:08.626084  7526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:08.636456 20743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:08.638154 20744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:08.640421 20740 heartbeater.cc:344] Connected to a master server at 127.7.89.190:40917
I20260709 13:08:08.641144 20740 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:08.641450  7526 server_base.cc:1061] running on GCE node
W20260709 13:08:08.641644 20747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:08.642596 20740 heartbeater.cc:507] Master 127.7.89.190:40917 requested a full tablet report, sending...
I20260709 13:08:08.643070  7526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:08.643409  7526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:08.643625  7526 hybrid_clock.cc:648] HybridClock initialized: now 1783602488643604 us; error 0 us; skew 500 ppm
I20260709 13:08:08.644418  7526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:08.645421 20538 ts_manager.cc:194] Registered new tserver with Master: 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:08.647462  7526 webserver.cc:533] Webserver started at http://127.7.89.131:44827/ using document root <none> and password file <none>
I20260709 13:08:08.647544 20538 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37308
I20260709 13:08:08.648144  7526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:08.648412  7526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:08.648710  7526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:08.650563  7526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-2-root/instance:
uuid: "602a0d7ba5984442865daf5400b33251"
format_stamp: "Formatted at 2026-07-09 13:08:08 on dist-test-slave-h0rz"
I20260709 13:08:08.655486  7526 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:08:08.659195 20751 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:08.660043  7526 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 13:08:08.660396  7526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-2-root
uuid: "602a0d7ba5984442865daf5400b33251"
format_stamp: "Formatted at 2026-07-09 13:08:08 on dist-test-slave-h0rz"
I20260709 13:08:08.660737  7526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRuHtD3/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602061986899-7526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:08.683928  7526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:08.685432  7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:08.687219  7526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:08.690019  7526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:08.690331  7526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:08.690644  7526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:08.690855  7526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:08:08.740760  7526 rpc_server.cc:307] RPC server started. Bound to: 127.7.89.131:42897
I20260709 13:08:08.740918 20813 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.89.131:42897 every 8 connection(s)
I20260709 13:08:08.757774 20814 heartbeater.cc:344] Connected to a master server at 127.7.89.190:40917
I20260709 13:08:08.758358 20814 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:08.759348 20814 heartbeater.cc:507] Master 127.7.89.190:40917 requested a full tablet report, sending...
I20260709 13:08:08.761571 20538 ts_manager.cc:194] Registered new tserver with Master: 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:08.761873  7526 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017337805s
I20260709 13:08:08.763746 20538 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37322
I20260709 13:08:08.794173 20538 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37334:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:08:08.797557 20538 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:08:08.888810 20779 tablet_service.cc:1511] Processing CreateTablet for tablet 81ca28ac68414a7fafb169b5d088b6d6 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:08:08.890715 20779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81ca28ac68414a7fafb169b5d088b6d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.898581 20778 tablet_service.cc:1511] Processing CreateTablet for tablet 769284c8e4f34e83a6ed1163c5ca3a85 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:08:08.900594 20778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 769284c8e4f34e83a6ed1163c5ca3a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.904119 20630 tablet_service.cc:1511] Processing CreateTablet for tablet 81ca28ac68414a7fafb169b5d088b6d6 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:08:08.904119 20777 tablet_service.cc:1511] Processing CreateTablet for tablet 98ba4deec7b446128f9c734b613ca13d (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:08:08.906001 20630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81ca28ac68414a7fafb169b5d088b6d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.907579 20776 tablet_service.cc:1511] Processing CreateTablet for tablet 40d9711960b64959a444115884f4a687 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:08:08.907961 20628 tablet_service.cc:1511] Processing CreateTablet for tablet 98ba4deec7b446128f9c734b613ca13d (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:08:08.909399 20627 tablet_service.cc:1511] Processing CreateTablet for tablet 40d9711960b64959a444115884f4a687 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:08:08.910908 20629 tablet_service.cc:1511] Processing CreateTablet for tablet 769284c8e4f34e83a6ed1163c5ca3a85 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:08:08.912192 20775 tablet_service.cc:1511] Processing CreateTablet for tablet 16975b7e56d04f8783a7b1250c87e478 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:08:08.912369 20626 tablet_service.cc:1511] Processing CreateTablet for tablet 16975b7e56d04f8783a7b1250c87e478 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:08:08.906188 20777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98ba4deec7b446128f9c734b613ca13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.916008 20625 tablet_service.cc:1511] Processing CreateTablet for tablet eea8f92e67394209ae7554aa9f4a0ac6 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:08:08.917467 20776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40d9711960b64959a444115884f4a687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.916275 20774 tablet_service.cc:1511] Processing CreateTablet for tablet eea8f92e67394209ae7554aa9f4a0ac6 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:08:08.918767 20774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eea8f92e67394209ae7554aa9f4a0ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.920977 20775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16975b7e56d04f8783a7b1250c87e478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.924719 20773 tablet_service.cc:1511] Processing CreateTablet for tablet 9f44644e1f5f4610b1dab75f07c2993e (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:08:08.927218 20625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eea8f92e67394209ae7554aa9f4a0ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.927357 20773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f44644e1f5f4610b1dab75f07c2993e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.929180 20628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98ba4deec7b446128f9c734b613ca13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.930550 20626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16975b7e56d04f8783a7b1250c87e478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.932008 20627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40d9711960b64959a444115884f4a687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.933976 20629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 769284c8e4f34e83a6ed1163c5ca3a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.933717 20772 tablet_service.cc:1511] Processing CreateTablet for tablet cf8d4d44b9b943c5b5858f393fc5c8fb (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:08:08.939986 20772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf8d4d44b9b943c5b5858f393fc5c8fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.962792 20771 tablet_service.cc:1511] Processing CreateTablet for tablet 63ac558c2e934cbdb38fa0c88858f4b7 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:08:08.964840 20771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63ac558c2e934cbdb38fa0c88858f4b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.967963 20624 tablet_service.cc:1511] Processing CreateTablet for tablet 9f44644e1f5f4610b1dab75f07c2993e (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:08:08.969937 20624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f44644e1f5f4610b1dab75f07c2993e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.972632 20705 tablet_service.cc:1511] Processing CreateTablet for tablet 81ca28ac68414a7fafb169b5d088b6d6 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:08:08.975481 20699 tablet_service.cc:1511] Processing CreateTablet for tablet 9f44644e1f5f4610b1dab75f07c2993e (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:08:08.976878 20704 tablet_service.cc:1511] Processing CreateTablet for tablet 769284c8e4f34e83a6ed1163c5ca3a85 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:08:08.978869 20698 tablet_service.cc:1511] Processing CreateTablet for tablet cf8d4d44b9b943c5b5858f393fc5c8fb (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:08:08.979903 20703 tablet_service.cc:1511] Processing CreateTablet for tablet 98ba4deec7b446128f9c734b613ca13d (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:08:08.982913 20702 tablet_service.cc:1511] Processing CreateTablet for tablet 40d9711960b64959a444115884f4a687 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:08:08.986182 20701 tablet_service.cc:1511] Processing CreateTablet for tablet 16975b7e56d04f8783a7b1250c87e478 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:08:08.989153 20700 tablet_service.cc:1511] Processing CreateTablet for tablet eea8f92e67394209ae7554aa9f4a0ac6 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:08:08.990684 20705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81ca28ac68414a7fafb169b5d088b6d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.991588 20702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40d9711960b64959a444115884f4a687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.993672 20700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eea8f92e67394209ae7554aa9f4a0ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.994481 20703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98ba4deec7b446128f9c734b613ca13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:08.997694 20701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16975b7e56d04f8783a7b1250c87e478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:09.000109 20697 tablet_service.cc:1511] Processing CreateTablet for tablet 63ac558c2e934cbdb38fa0c88858f4b7 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:08:08.998498 20704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 769284c8e4f34e83a6ed1163c5ca3a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:09.001873 20697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63ac558c2e934cbdb38fa0c88858f4b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:09.002719 20699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f44644e1f5f4610b1dab75f07c2993e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:09.005400 20698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf8d4d44b9b943c5b5858f393fc5c8fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:09.029347 20829 tablet_bootstrap.cc:492] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:09.033529 20623 tablet_service.cc:1511] Processing CreateTablet for tablet cf8d4d44b9b943c5b5858f393fc5c8fb (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:08:09.037876 20622 tablet_service.cc:1511] Processing CreateTablet for tablet 63ac558c2e934cbdb38fa0c88858f4b7 (DEFAULT_TABLE table=test-workload [id=d7ff61ed503447b8b00ed4f1e11091d7]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:08:09.041492 20623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf8d4d44b9b943c5b5858f393fc5c8fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:09.043699 20622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63ac558c2e934cbdb38fa0c88858f4b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:09.056129 20829 tablet_bootstrap.cc:654] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.122012 20829 tablet_bootstrap.cc:492] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:09.122678 20829 ts_tablet_manager.cc:1403] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.094s	user 0.039s	sys 0.050s
I20260709 13:08:09.126498 20829 raft_consensus.cc:359] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.127451 20829 raft_consensus.cc:385] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.127791 20829 raft_consensus.cc:740] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.128907 20829 consensus_queue.cc:260] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.152717 20814 heartbeater.cc:499] Master 127.7.89.190:40917 was elected leader, sending a full tablet report...
I20260709 13:08:09.169819 20829 ts_tablet_manager.cc:1434] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.047s	user 0.005s	sys 0.013s
I20260709 13:08:09.173385 20829 tablet_bootstrap.cc:492] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:09.181749 20829 tablet_bootstrap.cc:654] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.198523 20829 tablet_bootstrap.cc:492] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:09.199148 20829 ts_tablet_manager.cc:1403] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.004s
I20260709 13:08:09.202785 20829 raft_consensus.cc:359] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.203720 20829 raft_consensus.cc:385] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.204042 20829 raft_consensus.cc:740] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.204996 20829 consensus_queue.cc:260] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.210309 20829 ts_tablet_manager.cc:1434] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:08:09.211807 20829 tablet_bootstrap.cc:492] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:09.220310 20829 tablet_bootstrap.cc:654] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.229511 20834 tablet_bootstrap.cc:492] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:09.236485 20832 tablet_bootstrap.cc:492] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:09.241899 20829 tablet_bootstrap.cc:492] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:09.242856 20829 ts_tablet_manager.cc:1403] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.031s	user 0.019s	sys 0.010s
I20260709 13:08:09.245870 20832 tablet_bootstrap.cc:654] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.245875 20834 tablet_bootstrap.cc:654] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.249944 20829 raft_consensus.cc:359] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.252708 20829 raft_consensus.cc:385] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.253249 20829 raft_consensus.cc:740] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.254206 20829 consensus_queue.cc:260] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.257683 20829 ts_tablet_manager.cc:1434] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 13:08:09.259076 20829 tablet_bootstrap.cc:492] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:09.259893 20832 tablet_bootstrap.cc:492] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:09.260890 20832 ts_tablet_manager.cc:1403] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.000s
I20260709 13:08:09.266966 20829 tablet_bootstrap.cc:654] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.285153 20832 raft_consensus.cc:359] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.286584 20832 raft_consensus.cc:385] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.287498 20832 raft_consensus.cc:740] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.289189 20832 consensus_queue.cc:260] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.300516 20832 ts_tablet_manager.cc:1434] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.009s
I20260709 13:08:09.301795 20740 heartbeater.cc:499] Master 127.7.89.190:40917 was elected leader, sending a full tablet report...
W20260709 13:08:09.308917 20586 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:09.320307 20840 raft_consensus.cc:493] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.321337 20840 raft_consensus.cc:515] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.327885 20832 tablet_bootstrap.cc:492] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:09.340003 20840 leader_election.cc:290] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:09.342509 20834 tablet_bootstrap.cc:492] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:09.342232 20829 tablet_bootstrap.cc:492] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:09.343262 20829 ts_tablet_manager.cc:1403] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.084s	user 0.032s	sys 0.049s
I20260709 13:08:09.343467 20832 tablet_bootstrap.cc:654] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.343262 20834 ts_tablet_manager.cc:1403] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.114s	user 0.046s	sys 0.040s
I20260709 13:08:09.347283 20829 raft_consensus.cc:359] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.348232 20829 raft_consensus.cc:385] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.348584 20829 raft_consensus.cc:740] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.349566 20829 consensus_queue.cc:260] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.350517 20834 raft_consensus.cc:359] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.351503 20834 raft_consensus.cc:385] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.351893 20834 raft_consensus.cc:740] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.352312 20829 ts_tablet_manager.cc:1434] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:08:09.353173 20834 consensus_queue.cc:260] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.354357 20829 tablet_bootstrap.cc:492] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:09.362079 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d9711960b64959a444115884f4a687" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
W20260709 13:08:09.364522 20680 leader_election.cc:343] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.367821 20829 tablet_bootstrap.cc:654] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.368588 20832 tablet_bootstrap.cc:492] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:09.369460 20832 ts_tablet_manager.cc:1403] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.005s
I20260709 13:08:09.373636 20832 raft_consensus.cc:359] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.374605 20832 raft_consensus.cc:385] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.375010 20832 raft_consensus.cc:740] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.376123 20832 consensus_queue.cc:260] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.377800 20831 raft_consensus.cc:493] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.378660 20831 raft_consensus.cc:515] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.379215 20832 ts_tablet_manager.cc:1434] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
W20260709 13:08:09.379961 20741 tablet.cc:2406] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:08:09.383450 20831 leader_election.cc:290] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:09.387041 20665 heartbeater.cc:499] Master 127.7.89.190:40917 was elected leader, sending a full tablet report...
I20260709 13:08:09.386914 20844 raft_consensus.cc:493] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.388757 20834 ts_tablet_manager.cc:1434] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.045s	user 0.012s	sys 0.004s
I20260709 13:08:09.391916 20832 tablet_bootstrap.cc:492] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:09.403762 20829 tablet_bootstrap.cc:492] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:09.404491 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d9711960b64959a444115884f4a687" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:09.405388 20834 tablet_bootstrap.cc:492] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484: Bootstrap starting.
W20260709 13:08:09.407089 20680 leader_election.cc:343] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.407716 20680 leader_election.cc:304] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170; no voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484
I20260709 13:08:09.397341 20844 raft_consensus.cc:515] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.409134 20840 raft_consensus.cc:2749] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.411947 20844 leader_election.cc:290] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:09.415263 20834 tablet_bootstrap.cc:654] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.404585 20829 ts_tablet_manager.cc:1403] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.051s	user 0.027s	sys 0.015s
I20260709 13:08:09.423424 20832 tablet_bootstrap.cc:654] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.438665 20829 raft_consensus.cc:359] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.444972 20834 tablet_bootstrap.cc:492] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:09.445768 20834 ts_tablet_manager.cc:1403] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.041s	user 0.020s	sys 0.008s
I20260709 13:08:09.449499 20834 raft_consensus.cc:359] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.450353 20834 raft_consensus.cc:385] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.450917 20834 raft_consensus.cc:740] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.451992 20834 consensus_queue.cc:260] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.455660 20834 ts_tablet_manager.cc:1434] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:08:09.457262 20834 tablet_bootstrap.cc:492] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:09.453279 20829 raft_consensus.cc:385] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.459937 20829 raft_consensus.cc:740] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.460987 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16975b7e56d04f8783a7b1250c87e478" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:09.461184 20829 consensus_queue.cc:260] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.461858 20789 raft_consensus.cc:2468] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 0.
I20260709 13:08:09.464072 20829 ts_tablet_manager.cc:1434] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.004s
I20260709 13:08:09.464886 20605 leader_election.cc:304] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:09.465406 20829 tablet_bootstrap.cc:492] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:09.467475 20844 raft_consensus.cc:2804] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:09.468297 20844 raft_consensus.cc:493] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.468676 20844 raft_consensus.cc:3060] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.472493 20834 tablet_bootstrap.cc:654] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.478088 20715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16975b7e56d04f8783a7b1250c87e478" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:09.479723 20829 tablet_bootstrap.cc:654] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:09.480445 20753 leader_election.cc:343] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.483897 20832 tablet_bootstrap.cc:492] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:09.485106 20832 ts_tablet_manager.cc:1403] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.094s	user 0.017s	sys 0.003s
I20260709 13:08:09.488554 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16975b7e56d04f8783a7b1250c87e478" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:09.490826 20844 raft_consensus.cc:515] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.490206 20832 raft_consensus.cc:359] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.494030 20640 raft_consensus.cc:2393] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 602a0d7ba5984442865daf5400b33251 in current term 1: Already voted for candidate fabb72df3c844f928dc062705b599484 in this term.
I20260709 13:08:09.494095 20832 raft_consensus.cc:385] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.494669 20832 raft_consensus.cc:740] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.495425 20834 tablet_bootstrap.cc:492] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:09.496291 20834 ts_tablet_manager.cc:1403] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.039s	user 0.021s	sys 0.010s
I20260709 13:08:09.496871 20832 consensus_queue.cc:260] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.501629 20754 leader_election.cc:304] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 602a0d7ba5984442865daf5400b33251; no voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484
I20260709 13:08:09.502642 20832 ts_tablet_manager.cc:1434] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.001s
I20260709 13:08:09.503324 20831 raft_consensus.cc:3060] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.503960 20832 tablet_bootstrap.cc:492] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:09.513016 20832 tablet_bootstrap.cc:654] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.516183 20840 raft_consensus.cc:493] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.516914 20840 raft_consensus.cc:515] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.521009 20840 leader_election.cc:290] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:09.521279 20834 raft_consensus.cc:359] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.524519 20834 raft_consensus.cc:385] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.524909 20834 raft_consensus.cc:740] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.525353 20829 tablet_bootstrap.cc:492] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:09.526080 20834 consensus_queue.cc:260] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.527060 20831 raft_consensus.cc:2749] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.530546 20715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16975b7e56d04f8783a7b1250c87e478" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:09.531188 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16975b7e56d04f8783a7b1250c87e478" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
W20260709 13:08:09.533862 20604 leader_election.cc:343] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:09.535298 20604 leader_election.cc:343] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.536825 20834 ts_tablet_manager.cc:1434] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.001s
I20260709 13:08:09.536978 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8d4d44b9b943c5b5858f393fc5c8fb" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:09.538733 20789 raft_consensus.cc:2468] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 0.
I20260709 13:08:09.540289 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8d4d44b9b943c5b5858f393fc5c8fb" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:09.542477 20680 leader_election.cc:304] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: 
I20260709 13:08:09.543370 20844 leader_election.cc:290] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Requested vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
W20260709 13:08:09.543644 20680 leader_election.cc:343] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.544039 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16975b7e56d04f8783a7b1250c87e478" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:09.526280 20829 ts_tablet_manager.cc:1403] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.061s	user 0.016s	sys 0.016s
I20260709 13:08:09.544824 20840 raft_consensus.cc:2804] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:09.545351 20840 raft_consensus.cc:493] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.545737 20840 raft_consensus.cc:3060] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.540421 20831 raft_consensus.cc:493] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.547847 20831 raft_consensus.cc:515] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.549825 20829 raft_consensus.cc:359] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.551342 20829 raft_consensus.cc:385] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.551600 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ba4deec7b446128f9c734b613ca13d" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:09.551846 20829 raft_consensus.cc:740] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
W20260709 13:08:09.553668 20754 leader_election.cc:343] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.553226 20829 consensus_queue.cc:260] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.554980 20834 tablet_bootstrap.cc:492] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:09.565862 20832 tablet_bootstrap.cc:492] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:09.566322 20845 raft_consensus.cc:493] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.557669 20829 ts_tablet_manager.cc:1434] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:08:09.566773 20832 ts_tablet_manager.cc:1403] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.063s	user 0.023s	sys 0.033s
I20260709 13:08:09.567166 20845 raft_consensus.cc:515] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.568706 20829 tablet_bootstrap.cc:492] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:09.571406 20832 raft_consensus.cc:359] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.571641 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "769284c8e4f34e83a6ed1163c5ca3a85" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:09.571514 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "769284c8e4f34e83a6ed1163c5ca3a85" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:09.572678 20788 raft_consensus.cc:2468] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 0.
I20260709 13:08:09.573792 20832 raft_consensus.cc:385] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.559744 20789 raft_consensus.cc:2468] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 1.
I20260709 13:08:09.574185 20832 raft_consensus.cc:740] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.571758 20845 leader_election.cc:290] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
W20260709 13:08:09.574136 20604 leader_election.cc:343] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.575778 20832 consensus_queue.cc:260] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.576714 20829 tablet_bootstrap.cc:654] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.577564 20605 leader_election.cc:304] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:09.578655 20832 ts_tablet_manager.cc:1434] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:08:09.579651 20856 raft_consensus.cc:493] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.580119 20832 tablet_bootstrap.cc:492] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:09.580557 20856 raft_consensus.cc:515] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.583206 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ba4deec7b446128f9c734b613ca13d" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:09.585156 20856 leader_election.cc:290] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:09.586119 20605 leader_election.cc:304] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 9d91f828e5fa453982afb0ada2c45170
W20260709 13:08:09.587111 20753 leader_election.cc:343] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.587260 20844 raft_consensus.cc:2804] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:09.587713 20753 leader_election.cc:304] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 602a0d7ba5984442865daf5400b33251; no voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484
I20260709 13:08:09.587817 20845 raft_consensus.cc:2804] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:09.587831 20844 raft_consensus.cc:493] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.588685 20844 raft_consensus.cc:3060] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.590057 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f44644e1f5f4610b1dab75f07c2993e" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:09.590757 20831 leader_election.cc:290] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:09.593218 20832 tablet_bootstrap.cc:654] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.594290 20831 raft_consensus.cc:2749] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.591576 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f44644e1f5f4610b1dab75f07c2993e" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
W20260709 13:08:09.598416 20753 leader_election.cc:343] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:09.599259 20754 leader_election.cc:343] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.599252 20844 raft_consensus.cc:515] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.600370 20754 leader_election.cc:304] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 602a0d7ba5984442865daf5400b33251; no voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484
I20260709 13:08:09.601895 20856 raft_consensus.cc:2749] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.604321 20844 leader_election.cc:290] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Requested vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:09.604645 20845 raft_consensus.cc:697] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Becoming Leader. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:09.605851 20845 consensus_queue.cc:237] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.606398 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "769284c8e4f34e83a6ed1163c5ca3a85" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
W20260709 13:08:09.608440 20604 leader_election.cc:343] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.592916 20834 tablet_bootstrap.cc:654] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.610110 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "769284c8e4f34e83a6ed1163c5ca3a85" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:09.611188 20789 raft_consensus.cc:3060] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.608639 20832 tablet_bootstrap.cc:492] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:09.614992 20832 ts_tablet_manager.cc:1403] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.007s
I20260709 13:08:09.619311 20832 raft_consensus.cc:359] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.615137 20840 raft_consensus.cc:515] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.621898 20832 raft_consensus.cc:385] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.622531 20832 raft_consensus.cc:740] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.623687 20832 consensus_queue.cc:260] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.625288 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8d4d44b9b943c5b5858f393fc5c8fb" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:09.627024 20832 ts_tablet_manager.cc:1434] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
W20260709 13:08:09.627637 20680 leader_election.cc:343] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.628465 20832 tablet_bootstrap.cc:492] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:09.628702 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8d4d44b9b943c5b5858f393fc5c8fb" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:09.629483 20788 raft_consensus.cc:3060] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.638720 20840 leader_election.cc:290] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:09.639633 20832 tablet_bootstrap.cc:654] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.641820 20788 raft_consensus.cc:2468] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 1.
I20260709 13:08:09.642941 20840 raft_consensus.cc:493] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.643704 20840 raft_consensus.cc:515] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.641453 20538 catalog_manager.cc:5697] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 reported cstate change: term changed from 0 to 1, leader changed from <none> to fabb72df3c844f928dc062705b599484 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "fabb72df3c844f928dc062705b599484" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.646967 20680 leader_election.cc:304] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: fabb72df3c844f928dc062705b599484
I20260709 13:08:09.647431 20840 leader_election.cc:290] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:09.648123 20839 raft_consensus.cc:2804] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:09.648641 20839 raft_consensus.cc:697] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Becoming Leader. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Running, Role: LEADER
I20260709 13:08:09.649744 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "769284c8e4f34e83a6ed1163c5ca3a85" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:09.649823 20839 consensus_queue.cc:237] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.650664 20834 tablet_bootstrap.cc:492] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:09.650837 20640 raft_consensus.cc:2393] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9d91f828e5fa453982afb0ada2c45170 in current term 1: Already voted for candidate fabb72df3c844f928dc062705b599484 in this term.
I20260709 13:08:09.651376 20834 ts_tablet_manager.cc:1403] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.097s	user 0.011s	sys 0.005s
I20260709 13:08:09.654484 20789 raft_consensus.cc:2468] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 1.
I20260709 13:08:09.655871 20605 leader_election.cc:304] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:09.655073 20834 raft_consensus.cc:359] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.656153 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "769284c8e4f34e83a6ed1163c5ca3a85" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:09.656697 20834 raft_consensus.cc:385] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.656917 20845 raft_consensus.cc:2804] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:09.657138 20788 raft_consensus.cc:2393] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9d91f828e5fa453982afb0ada2c45170 in current term 1: Already voted for candidate fabb72df3c844f928dc062705b599484 in this term.
I20260709 13:08:09.657648 20845 raft_consensus.cc:697] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Becoming Leader. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:09.658711 20680 leader_election.cc:304] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170; no voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484
I20260709 13:08:09.658711 20845 consensus_queue.cc:237] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.657081 20834 raft_consensus.cc:740] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.661031 20829 tablet_bootstrap.cc:492] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:09.661768 20829 ts_tablet_manager.cc:1403] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.093s	user 0.030s	sys 0.020s
I20260709 13:08:09.666978 20832 tablet_bootstrap.cc:492] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:09.668491 20832 ts_tablet_manager.cc:1403] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.011s
I20260709 13:08:09.661700 20834 consensus_queue.cc:260] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.661826 20840 raft_consensus.cc:3060] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.672889 20832 raft_consensus.cc:359] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.674173 20832 raft_consensus.cc:385] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.674646 20832 raft_consensus.cc:740] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.676118 20834 ts_tablet_manager.cc:1434] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.005s
I20260709 13:08:09.676558 20832 consensus_queue.cc:260] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.678066 20834 tablet_bootstrap.cc:492] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:09.668879 20829 raft_consensus.cc:359] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.679339 20829 raft_consensus.cc:385] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.679705 20829 raft_consensus.cc:740] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.680805 20829 consensus_queue.cc:260] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.687980 20845 raft_consensus.cc:493] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.689006 20845 raft_consensus.cc:515] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.688604 20537 catalog_manager.cc:5697] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "9d91f828e5fa453982afb0ada2c45170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.693845 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f44644e1f5f4610b1dab75f07c2993e" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:09.694370 20845 leader_election.cc:290] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
W20260709 13:08:09.695775 20604 leader_election.cc:343] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.695996 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f44644e1f5f4610b1dab75f07c2993e" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:09.696870 20788 raft_consensus.cc:2468] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 0.
I20260709 13:08:09.698792 20605 leader_election.cc:304] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:09.700357 20845 raft_consensus.cc:2804] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:09.700841 20845 raft_consensus.cc:493] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.701256 20845 raft_consensus.cc:3060] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.710238 20840 raft_consensus.cc:2749] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.711663 20845 raft_consensus.cc:515] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.712965 20829 ts_tablet_manager.cc:1434] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.051s	user 0.009s	sys 0.000s
I20260709 13:08:09.715749 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f44644e1f5f4610b1dab75f07c2993e" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:09.716626 20845 leader_election.cc:290] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Requested vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
W20260709 13:08:09.717823 20604 leader_election.cc:343] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.718277 20829 tablet_bootstrap.cc:492] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:09.715991 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f44644e1f5f4610b1dab75f07c2993e" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:09.719074 20788 raft_consensus.cc:3060] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.724037 20834 tablet_bootstrap.cc:654] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.725189 20829 tablet_bootstrap.cc:654] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.725342 20845 raft_consensus.cc:493] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.726179 20845 raft_consensus.cc:515] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.730521 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d9711960b64959a444115884f4a687" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:09.731511 20714 raft_consensus.cc:2468] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 0.
I20260709 13:08:09.733407 20604 leader_election.cc:304] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:09.734699 20844 raft_consensus.cc:2804] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:09.735241 20844 raft_consensus.cc:493] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.735703 20844 raft_consensus.cc:3060] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.743573 20829 tablet_bootstrap.cc:492] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:09.746066 20829 ts_tablet_manager.cc:1403] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.003s
I20260709 13:08:09.744064 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d9711960b64959a444115884f4a687" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:09.747160 20789 raft_consensus.cc:2468] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 0.
I20260709 13:08:09.746091 20832 ts_tablet_manager.cc:1434] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.077s	user 0.022s	sys 0.013s
I20260709 13:08:09.749429 20832 tablet_bootstrap.cc:492] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:09.749298 20788 raft_consensus.cc:2468] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 1.
I20260709 13:08:09.750856 20605 leader_election.cc:304] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:09.743732 20845 leader_election.cc:290] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:09.752331 20859 raft_consensus.cc:2804] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:09.752955 20859 raft_consensus.cc:697] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Becoming Leader. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:09.754706 20834 tablet_bootstrap.cc:492] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:09.755326 20834 ts_tablet_manager.cc:1403] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.078s	user 0.028s	sys 0.009s
I20260709 13:08:09.753947 20859 consensus_queue.cc:237] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.758399 20844 raft_consensus.cc:515] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.759579 20834 raft_consensus.cc:359] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.760001 20829 raft_consensus.cc:359] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } }
I20260709 13:08:09.760946 20829 raft_consensus.cc:385] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.761365 20829 raft_consensus.cc:740] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.762406 20829 consensus_queue.cc:260] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } }
I20260709 13:08:09.765933 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d9711960b64959a444115884f4a687" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:09.767661 20844 leader_election.cc:290] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Requested vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:09.769230 20840 raft_consensus.cc:493] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.769856 20840 raft_consensus.cc:515] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.768074 20832 tablet_bootstrap.cc:654] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.771291 20714 raft_consensus.cc:3060] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.772377 20840 leader_election.cc:290] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:09.775465 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63ac558c2e934cbdb38fa0c88858f4b7" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:09.779242 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63ac558c2e934cbdb38fa0c88858f4b7" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:09.780411 20789 raft_consensus.cc:2468] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 0.
I20260709 13:08:09.782210 20832 tablet_bootstrap.cc:492] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:09.768653 20829 ts_tablet_manager.cc:1434] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 13:08:09.760845 20834 raft_consensus.cc:385] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.782979 20832 ts_tablet_manager.cc:1403] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.003s
I20260709 13:08:09.783210 20834 raft_consensus.cc:740] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.783701 20829 tablet_bootstrap.cc:492] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:09.784456 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d9711960b64959a444115884f4a687" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:09.784976 20714 raft_consensus.cc:2468] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 1.
I20260709 13:08:09.785166 20788 raft_consensus.cc:3060] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.786698 20604 leader_election.cc:304] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
W20260709 13:08:09.781944 20680 leader_election.cc:343] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.786887 20832 raft_consensus.cc:359] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.788173 20680 leader_election.cc:304] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: fabb72df3c844f928dc062705b599484
I20260709 13:08:09.788594 20832 raft_consensus.cc:385] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.788985 20832 raft_consensus.cc:740] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.797252 20840 raft_consensus.cc:2804] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:09.798025 20840 raft_consensus.cc:493] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.798547 20840 raft_consensus.cc:3060] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.798090 20834 consensus_queue.cc:260] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.802081 20832 consensus_queue.cc:260] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.812862 20832 ts_tablet_manager.cc:1434] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.001s
I20260709 13:08:09.803627 20829 tablet_bootstrap.cc:654] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.803808 20788 raft_consensus.cc:2468] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 1.
I20260709 13:08:09.806919 20845 raft_consensus.cc:2804] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:09.808966 20834 ts_tablet_manager.cc:1434] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.053s	user 0.009s	sys 0.000s
I20260709 13:08:09.815096 20845 raft_consensus.cc:697] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Becoming Leader. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:09.815832 20834 tablet_bootstrap.cc:492] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:09.816196 20845 consensus_queue.cc:237] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.818687 20832 tablet_bootstrap.cc:492] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:09.813082 20537 catalog_manager.cc:5697] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 reported cstate change: term changed from 0 to 1, leader changed from <none> to fabb72df3c844f928dc062705b599484 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "fabb72df3c844f928dc062705b599484" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.824280 20537 catalog_manager.cc:5697] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 reported cstate change: term changed from 0 to 1, leader changed from <none> to fabb72df3c844f928dc062705b599484 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "fabb72df3c844f928dc062705b599484" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.826640 20834 tablet_bootstrap.cc:654] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.827798 20832 tablet_bootstrap.cc:654] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.809404 20840 raft_consensus.cc:515] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.831779 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63ac558c2e934cbdb38fa0c88858f4b7" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484"
W20260709 13:08:09.834542 20680 leader_election.cc:343] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.837177 20840 leader_election.cc:290] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:09.838806 20829 tablet_bootstrap.cc:492] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:09.839497 20829 ts_tablet_manager.cc:1403] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.000s
I20260709 13:08:09.840588 20832 tablet_bootstrap.cc:492] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:09.841277 20832 ts_tablet_manager.cc:1403] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.008s
I20260709 13:08:09.841679 20834 tablet_bootstrap.cc:492] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:09.842363 20834 ts_tablet_manager.cc:1403] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.010s
I20260709 13:08:09.842659 20856 raft_consensus.cc:493] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.842985 20829 raft_consensus.cc:359] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.843338 20856 raft_consensus.cc:515] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } }
I20260709 13:08:09.843809 20829 raft_consensus.cc:385] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.844169 20829 raft_consensus.cc:740] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.844405 20832 raft_consensus.cc:359] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } }
I20260709 13:08:09.845184 20832 raft_consensus.cc:385] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.845731 20832 raft_consensus.cc:740] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.845152 20829 consensus_queue.cc:260] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.845973 20834 raft_consensus.cc:359] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.846873 20834 raft_consensus.cc:385] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.847160 20834 raft_consensus.cc:740] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.847815 20829 ts_tablet_manager.cc:1434] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:08:09.847729 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ca28ac68414a7fafb169b5d088b6d6" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:09.848068 20834 consensus_queue.cc:260] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.848811 20832 consensus_queue.cc:260] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } }
I20260709 13:08:09.849283 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ca28ac68414a7fafb169b5d088b6d6" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:09.849861 20714 raft_consensus.cc:2468] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 0.
I20260709 13:08:09.850687 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63ac558c2e934cbdb38fa0c88858f4b7" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:09.851075 20834 ts_tablet_manager.cc:1434] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:08:09.851373 20788 raft_consensus.cc:3060] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.851999 20753 leader_election.cc:304] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: 
I20260709 13:08:09.852227 20832 ts_tablet_manager.cc:1434] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:08:09.852313 20834 tablet_bootstrap.cc:492] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:09.852983 20831 raft_consensus.cc:2804] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:09.853315 20832 tablet_bootstrap.cc:492] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:09.853359 20831 raft_consensus.cc:493] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.854140 20831 raft_consensus.cc:3060] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:09.855758 20754 leader_election.cc:343] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.861088 20832 tablet_bootstrap.cc:654] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.861088 20834 tablet_bootstrap.cc:654] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.849225 20856 leader_election.cc:290] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:09.868467 20831 raft_consensus.cc:515] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } }
I20260709 13:08:09.869458 20856 raft_consensus.cc:493] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.870509 20856 raft_consensus.cc:515] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.873451 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ca28ac68414a7fafb169b5d088b6d6" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:09.875761 20639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea8f92e67394209ae7554aa9f4a0ac6" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
W20260709 13:08:09.879122 20754 leader_election.cc:343] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:09.880180 20754 leader_election.cc:343] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.882006 20831 leader_election.cc:290] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:09.884189 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea8f92e67394209ae7554aa9f4a0ac6" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:09.884824 20715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ca28ac68414a7fafb169b5d088b6d6" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:09.885066 20714 raft_consensus.cc:2468] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 0.
I20260709 13:08:09.885878 20715 raft_consensus.cc:3060] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.885979 20832 tablet_bootstrap.cc:492] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:09.886941 20753 leader_election.cc:304] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: fabb72df3c844f928dc062705b599484
I20260709 13:08:09.888792 20832 ts_tablet_manager.cc:1403] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.013s
I20260709 13:08:09.889288 20831 raft_consensus.cc:2804] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:09.889776 20831 raft_consensus.cc:493] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.890125 20831 raft_consensus.cc:3060] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.892339 20832 raft_consensus.cc:359] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.893273 20832 raft_consensus.cc:385] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.893575 20832 raft_consensus.cc:740] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.894043 20856 leader_election.cc:290] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:09.894459 20832 consensus_queue.cc:260] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.900183 20831 raft_consensus.cc:515] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.904211 20832 ts_tablet_manager.cc:1434] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:08:09.904429 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea8f92e67394209ae7554aa9f4a0ac6" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484"
W20260709 13:08:09.906486 20754 leader_election.cc:343] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.909248 20831 leader_election.cc:290] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:09.900878 20788 raft_consensus.cc:2468] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 1.
I20260709 13:08:09.911389 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea8f92e67394209ae7554aa9f4a0ac6" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:09.912349 20714 raft_consensus.cc:3060] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.913188 20715 raft_consensus.cc:2468] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 1.
I20260709 13:08:09.913301 20680 leader_election.cc:304] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: fabb72df3c844f928dc062705b599484
I20260709 13:08:09.914837 20840 raft_consensus.cc:2804] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:09.915501 20834 tablet_bootstrap.cc:492] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:09.916257 20834 ts_tablet_manager.cc:1403] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.064s	user 0.025s	sys 0.034s
I20260709 13:08:09.915395 20840 raft_consensus.cc:697] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Becoming Leader. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Running, Role: LEADER
I20260709 13:08:09.917407 20840 consensus_queue.cc:237] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.920083 20834 raft_consensus.cc:359] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } }
I20260709 13:08:09.921053 20834 raft_consensus.cc:385] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.921391 20834 raft_consensus.cc:740] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.922302 20834 consensus_queue.cc:260] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } }
I20260709 13:08:09.923377 20714 raft_consensus.cc:2468] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 1.
I20260709 13:08:09.925035 20753 leader_election.cc:304] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: fabb72df3c844f928dc062705b599484
I20260709 13:08:09.925176 20834 ts_tablet_manager.cc:1434] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:08:09.926677 20831 raft_consensus.cc:2804] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:09.927003 20834 tablet_bootstrap.cc:492] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:09.927306 20831 raft_consensus.cc:697] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Becoming Leader. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Running, Role: LEADER
I20260709 13:08:09.928334 20831 consensus_queue.cc:237] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.936691 20834 tablet_bootstrap.cc:654] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.938123 20753 leader_election.cc:304] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: fabb72df3c844f928dc062705b599484
I20260709 13:08:09.939747 20856 raft_consensus.cc:2804] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:09.939735 20537 catalog_manager.cc:5697] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 reported cstate change: term changed from 0 to 1, leader changed from <none> to fabb72df3c844f928dc062705b599484 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "fabb72df3c844f928dc062705b599484" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.942603 20856 raft_consensus.cc:697] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Becoming Leader. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Running, Role: LEADER
I20260709 13:08:09.943773 20856 consensus_queue.cc:237] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } }
I20260709 13:08:09.945230 20840 raft_consensus.cc:493] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.946076 20834 tablet_bootstrap.cc:492] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:09.945967 20840 raft_consensus.cc:515] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.946890 20834 ts_tablet_manager.cc:1403] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.003s
I20260709 13:08:09.948863 20840 leader_election.cc:290] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:09.949719 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f44644e1f5f4610b1dab75f07c2993e" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:09.950466 20834 raft_consensus.cc:359] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.951409 20834 raft_consensus.cc:385] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.951725 20834 raft_consensus.cc:740] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.952184 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f44644e1f5f4610b1dab75f07c2993e" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:09.950696 20536 catalog_manager.cc:5697] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 reported cstate change: term changed from 0 to 1, leader changed from <none> to 602a0d7ba5984442865daf5400b33251 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "602a0d7ba5984442865daf5400b33251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:09.952561 20834 consensus_queue.cc:260] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.953474 20788 raft_consensus.cc:2393] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9d91f828e5fa453982afb0ada2c45170 in current term 1: Already voted for candidate fabb72df3c844f928dc062705b599484 in this term.
I20260709 13:08:09.955271 20834 ts_tablet_manager.cc:1434] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:08:09.955843 20680 leader_election.cc:304] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170; no voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484
I20260709 13:08:09.956725 20834 tablet_bootstrap.cc:492] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:09.956815 20840 raft_consensus.cc:3060] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.971666 20867 raft_consensus.cc:493] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.972393 20867 raft_consensus.cc:515] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.974300 20840 raft_consensus.cc:2749] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.974728 20867 leader_election.cc:290] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:09.974743 20834 tablet_bootstrap.cc:654] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.976850 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ba4deec7b446128f9c734b613ca13d" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:09.977623 20640 raft_consensus.cc:2468] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 0.
I20260709 13:08:09.978734 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ba4deec7b446128f9c734b613ca13d" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:09.979029 20680 leader_election.cc:304] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:09.979533 20788 raft_consensus.cc:2468] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 0.
I20260709 13:08:09.980014 20867 raft_consensus.cc:2804] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:09.980535 20867 raft_consensus.cc:493] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.980914 20867 raft_consensus.cc:3060] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.980149 20535 catalog_manager.cc:5697] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "9d91f828e5fa453982afb0ada2c45170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.988675 20834 tablet_bootstrap.cc:492] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:09.989491 20834 ts_tablet_manager.cc:1403] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.033s	user 0.022s	sys 0.001s
I20260709 13:08:09.987731 20536 catalog_manager.cc:5697] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 reported cstate change: term changed from 0 to 1, leader changed from <none> to 602a0d7ba5984442865daf5400b33251 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "602a0d7ba5984442865daf5400b33251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.992686 20867 raft_consensus.cc:515] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:09.996903 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ba4deec7b446128f9c734b613ca13d" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:09.997741 20640 raft_consensus.cc:3060] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.996750 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ba4deec7b446128f9c734b613ca13d" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:10.003530 20788 raft_consensus.cc:3060] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.995590 20867 leader_election.cc:290] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:10.012197 20640 raft_consensus.cc:2468] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 1.
I20260709 13:08:10.012996 20788 raft_consensus.cc:2468] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 1.
I20260709 13:08:10.013789 20680 leader_election.cc:304] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:10.013068 20834 raft_consensus.cc:359] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.015441 20834 raft_consensus.cc:385] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.015615 20867 raft_consensus.cc:2804] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:10.015764 20834 raft_consensus.cc:740] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.016788 20867 raft_consensus.cc:697] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Becoming Leader. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Running, Role: LEADER
I20260709 13:08:10.016925 20834 consensus_queue.cc:260] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.018069 20867 consensus_queue.cc:237] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.019214 20834 ts_tablet_manager.cc:1434] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.005s
I20260709 13:08:10.026669 20845 raft_consensus.cc:493] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.027388 20845 raft_consensus.cc:515] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.028543 20536 catalog_manager.cc:5697] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "9d91f828e5fa453982afb0ada2c45170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:10.030341 20845 leader_election.cc:290] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:10.030576 20715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63ac558c2e934cbdb38fa0c88858f4b7" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:10.030754 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63ac558c2e934cbdb38fa0c88858f4b7" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:10.031734 20788 raft_consensus.cc:2393] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fabb72df3c844f928dc062705b599484 in current term 1: Already voted for candidate 9d91f828e5fa453982afb0ada2c45170 in this term.
I20260709 13:08:10.032948 20605 leader_election.cc:304] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fabb72df3c844f928dc062705b599484; no voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:10.033771 20845 raft_consensus.cc:3060] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.034267 20844 raft_consensus.cc:493] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.034960 20844 raft_consensus.cc:515] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } }
I20260709 13:08:10.037317 20844 leader_election.cc:290] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:10.039309 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ca28ac68414a7fafb169b5d088b6d6" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:10.041740 20715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ca28ac68414a7fafb169b5d088b6d6" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:10.042551 20715 raft_consensus.cc:2393] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fabb72df3c844f928dc062705b599484 in current term 1: Already voted for candidate 602a0d7ba5984442865daf5400b33251 in this term.
I20260709 13:08:10.043085 20845 raft_consensus.cc:2749] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:10.043830 20604 leader_election.cc:304] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fabb72df3c844f928dc062705b599484; no voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:10.044670 20845 raft_consensus.cc:3060] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.050825 20845 raft_consensus.cc:2749] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:10.063019 20640 raft_consensus.cc:3060] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.071470 20867 consensus_queue.cc:1048] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:10.084868 20867 consensus_queue.cc:1048] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.096297 20845 consensus_queue.cc:1048] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:10.107841 20715 raft_consensus.cc:3060] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.119112 20859 consensus_queue.cc:1048] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.122958 20859 consensus_queue.cc:1048] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:10.136966 20844 consensus_queue.cc:1048] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.235150 20844 consensus_queue.cc:1048] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.250535 20844 consensus_queue.cc:1048] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.257699 20844 consensus_queue.cc:1048] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.266849 20844 consensus_queue.cc:1048] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.312170 20856 consensus_queue.cc:1048] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:10.318938 20789 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81ca28ac68414a7fafb169b5d088b6d6"
dest_uuid: "602a0d7ba5984442865daf5400b33251"
mode: GRACEFUL
new_leader_uuid: "fabb72df3c844f928dc062705b599484"
 from {username='slave'} at 127.0.0.1:44546
I20260709 13:08:10.320096 20789 raft_consensus.cc:606] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Received request to transfer leadership to TS fabb72df3c844f928dc062705b599484
I20260709 13:08:10.328096 20640 raft_consensus.cc:3060] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.341710 20844 raft_consensus.cc:493] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.342293 20844 raft_consensus.cc:515] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.342895 20831 consensus_queue.cc:1048] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.347877 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea8f92e67394209ae7554aa9f4a0ac6" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:10.347919 20715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea8f92e67394209ae7554aa9f4a0ac6" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:10.349947 20604 leader_election.cc:304] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fabb72df3c844f928dc062705b599484; no voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:10.350144 20844 leader_election.cc:290] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:10.352731 20844 raft_consensus.cc:2749] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:10.354099 20715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98ba4deec7b446128f9c734b613ca13d"
dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
mode: GRACEFUL
new_leader_uuid: "fabb72df3c844f928dc062705b599484"
 from {username='slave'} at 127.0.0.1:44014
I20260709 13:08:10.355083 20715 raft_consensus.cc:606] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Received request to transfer leadership to TS fabb72df3c844f928dc062705b599484
I20260709 13:08:10.376549 20640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16975b7e56d04f8783a7b1250c87e478"
dest_uuid: "fabb72df3c844f928dc062705b599484"
mode: GRACEFUL
new_leader_uuid: "9d91f828e5fa453982afb0ada2c45170"
 from {username='slave'} at 127.0.0.1:38856
I20260709 13:08:10.377169 20640 raft_consensus.cc:606] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Received request to transfer leadership to TS 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:10.383575 20789 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eea8f92e67394209ae7554aa9f4a0ac6"
dest_uuid: "602a0d7ba5984442865daf5400b33251"
mode: GRACEFUL
new_leader_uuid: "9d91f828e5fa453982afb0ada2c45170"
 from {username='slave'} at 127.0.0.1:44546
I20260709 13:08:10.384222 20789 raft_consensus.cc:606] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Received request to transfer leadership to TS 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:10.390040 20640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f44644e1f5f4610b1dab75f07c2993e"
dest_uuid: "fabb72df3c844f928dc062705b599484"
mode: GRACEFUL
new_leader_uuid: "9d91f828e5fa453982afb0ada2c45170"
 from {username='slave'} at 127.0.0.1:38856
I20260709 13:08:10.390727 20640 raft_consensus.cc:606] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Received request to transfer leadership to TS 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:10.398722 20839 consensus_queue.cc:1048] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:10.400403 20715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63ac558c2e934cbdb38fa0c88858f4b7"
dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
mode: GRACEFUL
new_leader_uuid: "602a0d7ba5984442865daf5400b33251"
 from {username='slave'} at 127.0.0.1:44014
I20260709 13:08:10.401221 20715 raft_consensus.cc:606] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Received request to transfer leadership to TS 602a0d7ba5984442865daf5400b33251
I20260709 13:08:10.420274 20840 consensus_queue.cc:1048] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.432235 20867 raft_consensus.cc:993] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170: : Instructing follower 602a0d7ba5984442865daf5400b33251 to start an election
I20260709 13:08:10.432839 20840 raft_consensus.cc:1081] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Signalling peer 602a0d7ba5984442865daf5400b33251 to start an election
I20260709 13:08:10.434888 20789 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "63ac558c2e934cbdb38fa0c88858f4b7"
dest_uuid: "602a0d7ba5984442865daf5400b33251"
 from {username='slave'} at 127.0.0.1:44516
I20260709 13:08:10.435546 20789 raft_consensus.cc:493] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:10.435925 20789 raft_consensus.cc:3060] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.444285 20789 raft_consensus.cc:515] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.446357 20789 leader_election.cc:290] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 2 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:10.447537 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63ac558c2e934cbdb38fa0c88858f4b7" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:10.447682 20715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63ac558c2e934cbdb38fa0c88858f4b7" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:10.448416 20640 raft_consensus.cc:3060] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.448437 20715 raft_consensus.cc:3055] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:10.448899 20715 raft_consensus.cc:740] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Running, Role: LEADER
I20260709 13:08:10.449630 20715 consensus_queue.cc:260] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.450791 20715 raft_consensus.cc:3060] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.456005 20640 raft_consensus.cc:2468] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 2.
I20260709 13:08:10.457626 20754 leader_election.cc:304] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:10.457854 20715 raft_consensus.cc:2468] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 2.
I20260709 13:08:10.458580 20831 raft_consensus.cc:2804] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:10.459189 20831 raft_consensus.cc:697] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [term 2 LEADER]: Becoming Leader. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Running, Role: LEADER
I20260709 13:08:10.460098 20831 consensus_queue.cc:237] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.465513 20840 consensus_queue.cc:1048] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.468307 20536 catalog_manager.cc:5697] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 reported cstate change: term changed from 1 to 2, leader changed from 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130) to 602a0d7ba5984442865daf5400b33251 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "602a0d7ba5984442865daf5400b33251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:10.481451 20839 consensus_queue.cc:1048] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.492587 20867 raft_consensus.cc:993] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170: : Instructing follower fabb72df3c844f928dc062705b599484 to start an election
I20260709 13:08:10.493162 20839 raft_consensus.cc:1081] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Signalling peer fabb72df3c844f928dc062705b599484 to start an election
I20260709 13:08:10.495549 20640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98ba4deec7b446128f9c734b613ca13d"
dest_uuid: "fabb72df3c844f928dc062705b599484"
 from {username='slave'} at 127.0.0.1:38834
I20260709 13:08:10.496325 20640 raft_consensus.cc:493] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:10.496688 20640 raft_consensus.cc:3060] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.502549 20831 consensus_queue.cc:1048] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.505471 20640 raft_consensus.cc:515] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.509199 20715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ba4deec7b446128f9c734b613ca13d" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:10.509961 20715 raft_consensus.cc:3055] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:10.510383 20715 raft_consensus.cc:740] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Running, Role: LEADER
I20260709 13:08:10.510142 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ba4deec7b446128f9c734b613ca13d" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:10.510962 20640 leader_election.cc:290] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 2 election: Requested vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:10.511554 20715 consensus_queue.cc:260] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.511620 20789 raft_consensus.cc:3060] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.513609 20831 raft_consensus.cc:993] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251: : Instructing follower fabb72df3c844f928dc062705b599484 to start an election
I20260709 13:08:10.514156 20831 raft_consensus.cc:1081] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Signalling peer fabb72df3c844f928dc062705b599484 to start an election
I20260709 13:08:10.525219 20789 raft_consensus.cc:2468] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 2.
I20260709 13:08:10.526682 20605 leader_election.cc:304] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:10.526501 20831 consensus_queue.cc:1048] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:10.529093 20844 raft_consensus.cc:2804] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:10.529731 20844 raft_consensus.cc:697] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [term 2 LEADER]: Becoming Leader. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:10.519626 20715 raft_consensus.cc:3060] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.534965 20640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81ca28ac68414a7fafb169b5d088b6d6"
dest_uuid: "fabb72df3c844f928dc062705b599484"
 from {username='slave'} at 127.0.0.1:38842
I20260709 13:08:10.536105 20640 raft_consensus.cc:493] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:10.536562 20640 raft_consensus.cc:3060] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.530808 20844 consensus_queue.cc:237] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.545225 20715 raft_consensus.cc:2468] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 2.
I20260709 13:08:10.551890 20640 raft_consensus.cc:515] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } }
I20260709 13:08:10.551409 20536 catalog_manager.cc:5697] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 reported cstate change: term changed from 1 to 2, leader changed from 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130) to fabb72df3c844f928dc062705b599484 (127.7.89.129). New cstate: current_term: 2 leader_uuid: "fabb72df3c844f928dc062705b599484" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:10.554291 20640 leader_election.cc:290] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 2 election: Requested vote from peers 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:10.555558 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ca28ac68414a7fafb169b5d088b6d6" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:10.556456 20789 raft_consensus.cc:3055] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:10.556865 20789 raft_consensus.cc:740] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Running, Role: LEADER
I20260709 13:08:10.557822 20789 consensus_queue.cc:260] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } }
I20260709 13:08:10.555523 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ca28ac68414a7fafb169b5d088b6d6" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:10.559409 20789 raft_consensus.cc:3060] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.559672 20714 raft_consensus.cc:3060] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.569304 20789 raft_consensus.cc:2468] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 2.
I20260709 13:08:10.569342 20714 raft_consensus.cc:2468] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 2.
I20260709 13:08:10.571046 20604 leader_election.cc:304] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:10.571938 20844 raft_consensus.cc:2804] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:10.572510 20844 raft_consensus.cc:697] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [term 2 LEADER]: Becoming Leader. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:10.573272 20844 consensus_queue.cc:237] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } }
I20260709 13:08:10.580463 20536 catalog_manager.cc:5697] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 reported cstate change: term changed from 1 to 2, leader changed from 602a0d7ba5984442865daf5400b33251 (127.7.89.131) to fabb72df3c844f928dc062705b599484 (127.7.89.129). New cstate: current_term: 2 leader_uuid: "fabb72df3c844f928dc062705b599484" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:10.639991 20844 raft_consensus.cc:993] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484: : Instructing follower 9d91f828e5fa453982afb0ada2c45170 to start an election
I20260709 13:08:10.640574 20845 raft_consensus.cc:1081] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Signalling peer 9d91f828e5fa453982afb0ada2c45170 to start an election
I20260709 13:08:10.642315 20714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16975b7e56d04f8783a7b1250c87e478"
dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
 from {username='slave'} at 127.0.0.1:44002
I20260709 13:08:10.643004 20714 raft_consensus.cc:493] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:10.643432 20714 raft_consensus.cc:3060] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.650403 20714 raft_consensus.cc:515] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.653450 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16975b7e56d04f8783a7b1250c87e478" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:10.653654 20714 leader_election.cc:290] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 2 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:10.654202 20640 raft_consensus.cc:3055] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:10.654270 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16975b7e56d04f8783a7b1250c87e478" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:10.654639 20640 raft_consensus.cc:740] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:10.655031 20789 raft_consensus.cc:3060] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.655591 20640 consensus_queue.cc:260] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.656795 20640 raft_consensus.cc:3060] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.662421 20789 raft_consensus.cc:2468] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 2.
I20260709 13:08:10.663112 20640 raft_consensus.cc:2468] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 2.
I20260709 13:08:10.663954 20680 leader_election.cc:304] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: 
I20260709 13:08:10.664844 20840 raft_consensus.cc:2804] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:10.665400 20840 raft_consensus.cc:697] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [term 2 LEADER]: Becoming Leader. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Running, Role: LEADER
I20260709 13:08:10.666347 20840 consensus_queue.cc:237] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.674201 20536 catalog_manager.cc:5697] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 reported cstate change: term changed from 1 to 2, leader changed from fabb72df3c844f928dc062705b599484 (127.7.89.129) to 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "9d91f828e5fa453982afb0ada2c45170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:10.732187 20856 raft_consensus.cc:993] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251: : Instructing follower 9d91f828e5fa453982afb0ada2c45170 to start an election
I20260709 13:08:10.732769 20831 raft_consensus.cc:1081] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Signalling peer 9d91f828e5fa453982afb0ada2c45170 to start an election
I20260709 13:08:10.734493 20714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eea8f92e67394209ae7554aa9f4a0ac6"
dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
 from {username='slave'} at 127.0.0.1:43996
I20260709 13:08:10.735181 20714 raft_consensus.cc:493] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:10.735531 20714 raft_consensus.cc:3060] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.742028 20714 raft_consensus.cc:515] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.744009 20714 leader_election.cc:290] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 2 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:10.745179 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea8f92e67394209ae7554aa9f4a0ac6" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:10.745373 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eea8f92e67394209ae7554aa9f4a0ac6" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:10.745992 20640 raft_consensus.cc:3060] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.746119 20789 raft_consensus.cc:3055] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:10.746580 20789 raft_consensus.cc:740] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Running, Role: LEADER
I20260709 13:08:10.747419 20789 consensus_queue.cc:260] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.748541 20789 raft_consensus.cc:3060] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.752566 20640 raft_consensus.cc:2468] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 2.
I20260709 13:08:10.753923 20680 leader_election.cc:304] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:10.754781 20840 raft_consensus.cc:2804] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:10.754902 20789 raft_consensus.cc:2468] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 2.
I20260709 13:08:10.755211 20840 raft_consensus.cc:697] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [term 2 LEADER]: Becoming Leader. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Running, Role: LEADER
I20260709 13:08:10.756268 20840 consensus_queue.cc:237] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.763291 20536 catalog_manager.cc:5697] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 reported cstate change: term changed from 1 to 2, leader changed from 602a0d7ba5984442865daf5400b33251 (127.7.89.131) to 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "9d91f828e5fa453982afb0ada2c45170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:10.850540 20845 raft_consensus.cc:993] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484: : Instructing follower 9d91f828e5fa453982afb0ada2c45170 to start an election
I20260709 13:08:10.851100 20859 raft_consensus.cc:1081] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Signalling peer 9d91f828e5fa453982afb0ada2c45170 to start an election
I20260709 13:08:10.852939 20714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f44644e1f5f4610b1dab75f07c2993e"
dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
 from {username='slave'} at 127.0.0.1:44002
I20260709 13:08:10.853530 20714 raft_consensus.cc:493] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:10.853977 20714 raft_consensus.cc:3060] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.860740 20714 raft_consensus.cc:515] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.862764 20714 leader_election.cc:290] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 2 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:10.864008 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f44644e1f5f4610b1dab75f07c2993e" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:10.864112 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f44644e1f5f4610b1dab75f07c2993e" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:10.864813 20640 raft_consensus.cc:3055] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:10.864826 20789 raft_consensus.cc:3060] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.865252 20640 raft_consensus.cc:740] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:10.866081 20640 consensus_queue.cc:260] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.867229 20640 raft_consensus.cc:3060] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.872344 20789 raft_consensus.cc:2468] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 2.
I20260709 13:08:10.873910 20680 leader_election.cc:304] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: 
I20260709 13:08:10.874432 20640 raft_consensus.cc:2468] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 2.
I20260709 13:08:10.875027 20840 raft_consensus.cc:2804] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:10.875661 20840 raft_consensus.cc:697] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [term 2 LEADER]: Becoming Leader. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Running, Role: LEADER
I20260709 13:08:10.876597 20840 consensus_queue.cc:237] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:10.883363 20536 catalog_manager.cc:5697] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 reported cstate change: term changed from 1 to 2, leader changed from fabb72df3c844f928dc062705b599484 (127.7.89.129) to 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "9d91f828e5fa453982afb0ada2c45170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:10.941671 20714 raft_consensus.cc:1275] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Refusing update from remote peer 602a0d7ba5984442865daf5400b33251: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:10.943280 20831 consensus_queue.cc:1048] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:10.953358 20715 raft_consensus.cc:1275] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Refusing update from remote peer fabb72df3c844f928dc062705b599484: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:10.956750 20640 raft_consensus.cc:1275] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Refusing update from remote peer 602a0d7ba5984442865daf5400b33251: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:10.956547 20859 consensus_queue.cc:1048] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:10.958748 20884 consensus_queue.cc:1048] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:10.968787 20789 raft_consensus.cc:1275] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Refusing update from remote peer fabb72df3c844f928dc062705b599484: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:10.971316 20845 consensus_queue.cc:1048] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:10.994273 20714 raft_consensus.cc:1275] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Refusing update from remote peer fabb72df3c844f928dc062705b599484: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:10.995857 20845 consensus_queue.cc:1048] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:11.004931 20789 raft_consensus.cc:1275] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Refusing update from remote peer fabb72df3c844f928dc062705b599484: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:11.007308 20844 consensus_queue.cc:1048] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:11.210932 20640 raft_consensus.cc:1275] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Refusing update from remote peer 9d91f828e5fa453982afb0ada2c45170: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:11.213989 20840 consensus_queue.cc:1048] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:11.232862 20788 raft_consensus.cc:1275] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Refusing update from remote peer 9d91f828e5fa453982afb0ada2c45170: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:11.235031 20914 consensus_queue.cc:1048] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:11.242825 20789 raft_consensus.cc:1275] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Refusing update from remote peer 9d91f828e5fa453982afb0ada2c45170: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:11.245119 20914 consensus_queue.cc:1048] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:11.258150 20640 raft_consensus.cc:1275] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Refusing update from remote peer 9d91f828e5fa453982afb0ada2c45170: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:11.261255 20839 consensus_queue.cc:1048] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:11.269789 20789 raft_consensus.cc:1275] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Refusing update from remote peer 9d91f828e5fa453982afb0ada2c45170: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:11.273269 20839 consensus_queue.cc:1048] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:11.293021 20640 raft_consensus.cc:1275] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Refusing update from remote peer 9d91f828e5fa453982afb0ada2c45170: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:11.294739 20914 consensus_queue.cc:1048] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:13.530834 20536 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37338:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:08:13.533922 20536 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:08:13.595212 20623 tablet_service.cc:1511] Processing CreateTablet for tablet c2f5838bfc4240de8f14073d0c85fcab (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:08:13.596694 20623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f5838bfc4240de8f14073d0c85fcab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.596421 20701 tablet_service.cc:1511] Processing CreateTablet for tablet c2f5838bfc4240de8f14073d0c85fcab (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:08:13.597973 20701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f5838bfc4240de8f14073d0c85fcab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.600028 20625 tablet_service.cc:1511] Processing CreateTablet for tablet 5d74c21c7a154be5a0057771d3e63f0c (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:08:13.601696 20625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d74c21c7a154be5a0057771d3e63f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.603984 20705 tablet_service.cc:1511] Processing CreateTablet for tablet 5d74c21c7a154be5a0057771d3e63f0c (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:08:13.605597 20705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d74c21c7a154be5a0057771d3e63f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.605261 20779 tablet_service.cc:1511] Processing CreateTablet for tablet 5d74c21c7a154be5a0057771d3e63f0c (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:08:13.600028 20774 tablet_service.cc:1511] Processing CreateTablet for tablet c2f5838bfc4240de8f14073d0c85fcab (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:08:13.609189 20699 tablet_service.cc:1511] Processing CreateTablet for tablet 3c5385cd8d0742d7ab0cdd98e49e3a5f (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:08:13.610749 20699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c5385cd8d0742d7ab0cdd98e49e3a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.615114 20702 tablet_service.cc:1511] Processing CreateTablet for tablet 6e8109831d6b4ada87ba80fa510b715d (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:08:13.615736 20622 tablet_service.cc:1511] Processing CreateTablet for tablet 6e8109831d6b4ada87ba80fa510b715d (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:08:13.617403 20702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e8109831d6b4ada87ba80fa510b715d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.617669 20622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e8109831d6b4ada87ba80fa510b715d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.619129 20630 tablet_service.cc:1511] Processing CreateTablet for tablet 3c5385cd8d0742d7ab0cdd98e49e3a5f (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:08:13.620653 20630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c5385cd8d0742d7ab0cdd98e49e3a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.609181 20776 tablet_service.cc:1511] Processing CreateTablet for tablet 3c5385cd8d0742d7ab0cdd98e49e3a5f (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:08:13.623659 20778 tablet_service.cc:1511] Processing CreateTablet for tablet 6e8109831d6b4ada87ba80fa510b715d (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:08:13.627060 20779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d74c21c7a154be5a0057771d3e63f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.631633 20703 tablet_service.cc:1511] Processing CreateTablet for tablet 27876839904a43d0aa2d0c1c992e5738 (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:08:13.633315 20703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27876839904a43d0aa2d0c1c992e5738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.635753 20771 tablet_service.cc:1511] Processing CreateTablet for tablet 27876839904a43d0aa2d0c1c992e5738 (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:08:13.637360 20771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27876839904a43d0aa2d0c1c992e5738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.638006 20698 tablet_service.cc:1511] Processing CreateTablet for tablet ee5967cbef3b48bc87d03588f9b4981d (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:08:13.639604 20698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee5967cbef3b48bc87d03588f9b4981d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.641633 20772 tablet_service.cc:1511] Processing CreateTablet for tablet ee5967cbef3b48bc87d03588f9b4981d (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:08:13.643404 20772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee5967cbef3b48bc87d03588f9b4981d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.644271 20704 tablet_service.cc:1511] Processing CreateTablet for tablet 2c3b313ca1784f4194bae419112cb6cb (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:08:13.645704 20704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c3b313ca1784f4194bae419112cb6cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.647712 20777 tablet_service.cc:1511] Processing CreateTablet for tablet 2c3b313ca1784f4194bae419112cb6cb (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:08:13.648416 20628 tablet_service.cc:1511] Processing CreateTablet for tablet 27876839904a43d0aa2d0c1c992e5738 (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:08:13.649291 20778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e8109831d6b4ada87ba80fa510b715d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.649888 20628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27876839904a43d0aa2d0c1c992e5738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.650275 20624 tablet_service.cc:1511] Processing CreateTablet for tablet ee5967cbef3b48bc87d03588f9b4981d (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:08:13.651146 20774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f5838bfc4240de8f14073d0c85fcab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.651692 20624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee5967cbef3b48bc87d03588f9b4981d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.654983 20776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c5385cd8d0742d7ab0cdd98e49e3a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.655143 20629 tablet_service.cc:1511] Processing CreateTablet for tablet 2c3b313ca1784f4194bae419112cb6cb (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:08:13.656978 20629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c3b313ca1784f4194bae419112cb6cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.659773 20626 tablet_service.cc:1511] Processing CreateTablet for tablet 50538deb9d0641d6811ee0d600ec6c24 (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:08:13.661371 20626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50538deb9d0641d6811ee0d600ec6c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.663028 20777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c3b313ca1784f4194bae419112cb6cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.708947 20700 tablet_service.cc:1511] Processing CreateTablet for tablet 50538deb9d0641d6811ee0d600ec6c24 (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:08:13.715456 20700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50538deb9d0641d6811ee0d600ec6c24. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:08:13.731221 20661 debug-util.cc:398] Leaking SignalData structure 0x7b0800037cc0 after lost signal to thread 7529
W20260709 13:08:13.739382 20661 debug-util.cc:398] Leaking SignalData structure 0x7b0800154e00 after lost signal to thread 20664
W20260709 13:08:13.741813 20661 debug-util.cc:398] Leaking SignalData structure 0x7b08002cd740 after lost signal to thread 20739
W20260709 13:08:13.744087 20661 debug-util.cc:398] Leaking SignalData structure 0x7b08001e5880 after lost signal to thread 20813
I20260709 13:08:13.757791 20775 tablet_service.cc:1511] Processing CreateTablet for tablet 50538deb9d0641d6811ee0d600ec6c24 (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:08:13.759536 20775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50538deb9d0641d6811ee0d600ec6c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.768630 20935 tablet_bootstrap.cc:492] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:13.770011 20933 tablet_bootstrap.cc:492] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:13.774432 20623 tablet_service.cc:1511] Processing CreateTablet for tablet 58d0ebe3dbfe4a3b9d21682bf14d1333 (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:08:13.788017 20623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58d0ebe3dbfe4a3b9d21682bf14d1333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.799981 20698 tablet_service.cc:1511] Processing CreateTablet for tablet 58d0ebe3dbfe4a3b9d21682bf14d1333 (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:08:13.801367 20773 tablet_service.cc:1511] Processing CreateTablet for tablet 58d0ebe3dbfe4a3b9d21682bf14d1333 (DEFAULT_TABLE table=soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:08:13.803246 20773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58d0ebe3dbfe4a3b9d21682bf14d1333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.806957 20933 tablet_bootstrap.cc:654] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.829849 20698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58d0ebe3dbfe4a3b9d21682bf14d1333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:13.856560 20933 tablet_bootstrap.cc:492] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:13.857364 20933 ts_tablet_manager.cc:1403] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.088s	user 0.007s	sys 0.009s
I20260709 13:08:13.861470 20933 raft_consensus.cc:359] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:13.863624 20935 tablet_bootstrap.cc:654] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.862589 20933 raft_consensus.cc:385] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.867458 20933 raft_consensus.cc:740] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.883509 20933 consensus_queue.cc:260] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:13.892362 20937 tablet_bootstrap.cc:492] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:13.893555 20933 ts_tablet_manager.cc:1434] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.007s
I20260709 13:08:13.902748 20935 tablet_bootstrap.cc:492] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:13.903565 20935 ts_tablet_manager.cc:1403] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.135s	user 0.019s	sys 0.018s
I20260709 13:08:13.907445 20935 raft_consensus.cc:359] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:13.908368 20935 raft_consensus.cc:385] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.908684 20935 raft_consensus.cc:740] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.909466 20935 consensus_queue.cc:260] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:13.912057 20934 raft_consensus.cc:493] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.912513 20935 ts_tablet_manager.cc:1434] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:08:13.912626 20934 raft_consensus.cc:515] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:13.914093 20935 tablet_bootstrap.cc:492] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:13.916534 20639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5967cbef3b48bc87d03588f9b4981d" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
W20260709 13:08:13.918782 20680 leader_election.cc:343] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.920262 20934 leader_election.cc:290] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:13.922439 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5967cbef3b48bc87d03588f9b4981d" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:13.923372 20935 tablet_bootstrap.cc:654] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:13.924274 20680 leader_election.cc:343] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.924988 20680 leader_election.cc:304] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170; no voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484
I20260709 13:08:13.926383 20934 raft_consensus.cc:2749] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:13.936751 20935 tablet_bootstrap.cc:492] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:13.937496 20935 ts_tablet_manager.cc:1403] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.024s	user 0.003s	sys 0.016s
I20260709 13:08:13.928583 20937 tablet_bootstrap.cc:654] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.939335 20933 tablet_bootstrap.cc:492] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:13.941416 20935 raft_consensus.cc:359] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:13.942425 20935 raft_consensus.cc:385] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.942781 20935 raft_consensus.cc:740] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.943672 20935 consensus_queue.cc:260] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:13.947010 20935 ts_tablet_manager.cc:1434] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.001s
I20260709 13:08:13.948794 20935 tablet_bootstrap.cc:492] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:13.956947 20935 tablet_bootstrap.cc:654] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.958287 20933 tablet_bootstrap.cc:654] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.960799 20937 tablet_bootstrap.cc:492] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:13.961580 20937 ts_tablet_manager.cc:1403] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.070s	user 0.017s	sys 0.000s
I20260709 13:08:13.964299 20935 tablet_bootstrap.cc:492] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:13.965287 20935 ts_tablet_manager.cc:1403] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.000s
I20260709 13:08:13.965881 20937 raft_consensus.cc:359] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:13.967043 20937 raft_consensus.cc:385] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.967463 20937 raft_consensus.cc:740] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.968582 20937 consensus_queue.cc:260] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:13.969631 20935 raft_consensus.cc:359] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:13.970497 20935 raft_consensus.cc:385] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.970840 20935 raft_consensus.cc:740] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.971706 20935 consensus_queue.cc:260] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:13.980012 20933 tablet_bootstrap.cc:492] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:13.980755 20933 ts_tablet_manager.cc:1403] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.006s
I20260709 13:08:13.982587 20935 ts_tablet_manager.cc:1434] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 13:08:13.983973 20935 tablet_bootstrap.cc:492] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:13.996723 20935 tablet_bootstrap.cc:654] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.996816 20937 ts_tablet_manager.cc:1434] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:08:13.984562 20933 raft_consensus.cc:359] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:13.998320 20933 raft_consensus.cc:385] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.998747 20933 raft_consensus.cc:740] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.999778 20933 consensus_queue.cc:260] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:13.998827 20937 tablet_bootstrap.cc:492] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:14.003192 20933 ts_tablet_manager.cc:1434] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.022s	user 0.002s	sys 0.008s
I20260709 13:08:14.004535 20933 tablet_bootstrap.cc:492] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:14.010776 20937 tablet_bootstrap.cc:654] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.013657 20933 tablet_bootstrap.cc:654] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.018942 20934 raft_consensus.cc:493] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.019919 20934 raft_consensus.cc:515] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.024366 20639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e8109831d6b4ada87ba80fa510b715d" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:14.024545 20934 leader_election.cc:290] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
W20260709 13:08:14.026651 20680 leader_election.cc:343] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.028479 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e8109831d6b4ada87ba80fa510b715d" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
W20260709 13:08:14.031816 20680 leader_election.cc:343] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.032696 20933 tablet_bootstrap.cc:492] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:14.032523 20680 leader_election.cc:304] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170; no voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484
I20260709 13:08:14.034336 20934 raft_consensus.cc:2749] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.035586 20933 ts_tablet_manager.cc:1403] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.031s	user 0.006s	sys 0.016s
I20260709 13:08:14.039746 20933 raft_consensus.cc:359] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.043601 20933 raft_consensus.cc:385] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.044047 20933 raft_consensus.cc:740] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.045167 20933 consensus_queue.cc:260] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.047044 20935 tablet_bootstrap.cc:492] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:14.047742 20935 ts_tablet_manager.cc:1403] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.064s	user 0.023s	sys 0.006s
I20260709 13:08:14.048283 20933 ts_tablet_manager.cc:1434] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:08:14.050377 20933 tablet_bootstrap.cc:492] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:14.051172 20935 raft_consensus.cc:359] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.052060 20935 raft_consensus.cc:385] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.052539 20935 raft_consensus.cc:740] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.053429 20935 consensus_queue.cc:260] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.055621 20937 tablet_bootstrap.cc:492] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:14.056264 20937 ts_tablet_manager.cc:1403] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.058s	user 0.011s	sys 0.012s
I20260709 13:08:14.059163 20937 raft_consensus.cc:359] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.059993 20937 raft_consensus.cc:385] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.060312 20937 raft_consensus.cc:740] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.061116 20937 consensus_queue.cc:260] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.063711 20937 ts_tablet_manager.cc:1434] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.007s
I20260709 13:08:14.063949 20933 tablet_bootstrap.cc:654] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.066370 20935 ts_tablet_manager.cc:1434] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:08:14.071616 20934 raft_consensus.cc:493] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.072053 20935 tablet_bootstrap.cc:492] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:14.072389 20934 raft_consensus.cc:515] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.077088 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d74c21c7a154be5a0057771d3e63f0c" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:14.067663 20937 tablet_bootstrap.cc:492] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
W20260709 13:08:14.079005 20680 leader_election.cc:343] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.079725 20934 leader_election.cc:290] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.085129 20935 tablet_bootstrap.cc:654] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.093981 20937 tablet_bootstrap.cc:654] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.100319 20639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d74c21c7a154be5a0057771d3e63f0c" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
W20260709 13:08:14.101986 20680 leader_election.cc:343] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.102579 20680 leader_election.cc:304] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170; no voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484
I20260709 13:08:14.104136 20939 raft_consensus.cc:493] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.104784 20939 raft_consensus.cc:515] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.108222 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5838bfc4240de8f14073d0c85fcab" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:14.109010 20714 raft_consensus.cc:2468] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 0.
I20260709 13:08:14.110442 20604 leader_election.cc:304] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:14.110455 20934 raft_consensus.cc:2749] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.111361 20844 raft_consensus.cc:2804] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:14.111770 20844 raft_consensus.cc:493] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.112274 20844 raft_consensus.cc:3060] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.115098 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5838bfc4240de8f14073d0c85fcab" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:14.111747 20939 leader_election.cc:290] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.131507 20789 raft_consensus.cc:2468] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 0.
I20260709 13:08:14.136883 20844 raft_consensus.cc:515] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.140686 20933 tablet_bootstrap.cc:492] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:14.140784 20935 tablet_bootstrap.cc:492] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:14.141484 20933 ts_tablet_manager.cc:1403] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.091s	user 0.021s	sys 0.005s
I20260709 13:08:14.141579 20935 ts_tablet_manager.cc:1403] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.070s	user 0.005s	sys 0.011s
I20260709 13:08:14.141830 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5838bfc4240de8f14073d0c85fcab" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:14.142764 20714 raft_consensus.cc:3060] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.145546 20933 raft_consensus.cc:359] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.146486 20933 raft_consensus.cc:385] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.146927 20933 raft_consensus.cc:740] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.146127 20935 raft_consensus.cc:359] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.148190 20935 raft_consensus.cc:385] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.147836 20933 consensus_queue.cc:260] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.148552 20935 raft_consensus.cc:740] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.148975 20937 tablet_bootstrap.cc:492] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:14.149650 20937 ts_tablet_manager.cc:1403] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.001s
I20260709 13:08:14.149507 20935 consensus_queue.cc:260] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.150560 20933 ts_tablet_manager.cc:1434] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 13:08:14.163228 20714 raft_consensus.cc:2468] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 1.
I20260709 13:08:14.163523 20933 tablet_bootstrap.cc:492] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:14.165386 20604 leader_election.cc:304] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:14.166822 20950 raft_consensus.cc:2804] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:14.167789 20950 raft_consensus.cc:697] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Becoming Leader. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:14.160681 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5838bfc4240de8f14073d0c85fcab" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:14.156378 20935 ts_tablet_manager.cc:1434] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:08:14.169644 20789 raft_consensus.cc:3060] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.171113 20935 tablet_bootstrap.cc:492] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:14.173935 20933 tablet_bootstrap.cc:654] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.178956 20939 raft_consensus.cc:493] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.179759 20939 raft_consensus.cc:515] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.184217 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50538deb9d0641d6811ee0d600ec6c24" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
W20260709 13:08:14.186635 20605 leader_election.cc:343] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.187580 20939 leader_election.cc:290] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.189904 20715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50538deb9d0641d6811ee0d600ec6c24" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
W20260709 13:08:14.192232 20604 leader_election.cc:343] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.192843 20604 leader_election.cc:304] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fabb72df3c844f928dc062705b599484; no voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:14.194396 20939 raft_consensus.cc:2749] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.153419 20937 raft_consensus.cc:359] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.194697 20935 tablet_bootstrap.cc:654] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.195693 20937 raft_consensus.cc:385] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.196247 20937 raft_consensus.cc:740] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.197423 20937 consensus_queue.cc:260] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.159000 20844 leader_election.cc:290] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Requested vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.201902 20937 ts_tablet_manager.cc:1434] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.052s	user 0.004s	sys 0.005s
I20260709 13:08:14.206956 20937 tablet_bootstrap.cc:492] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:14.205088 20789 raft_consensus.cc:2468] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 1.
I20260709 13:08:14.202491 20950 consensus_queue.cc:237] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.225593 20937 tablet_bootstrap.cc:654] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.236270 20935 tablet_bootstrap.cc:492] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:14.237205 20935 ts_tablet_manager.cc:1403] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.066s	user 0.012s	sys 0.012s
I20260709 13:08:14.245213 20935 raft_consensus.cc:359] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.246258 20935 raft_consensus.cc:385] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.246703 20935 raft_consensus.cc:740] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.247743 20937 tablet_bootstrap.cc:492] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:14.257591 20937 ts_tablet_manager.cc:1403] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.011s
I20260709 13:08:14.257151 20935 consensus_queue.cc:260] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.260079 20935 ts_tablet_manager.cc:1434] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.007s
I20260709 13:08:14.261828 20935 tablet_bootstrap.cc:492] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:14.261845 20937 raft_consensus.cc:359] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.263222 20937 raft_consensus.cc:385] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.263578 20937 raft_consensus.cc:740] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.264468 20937 consensus_queue.cc:260] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.267180 20937 ts_tablet_manager.cc:1434] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:08:14.268664 20937 tablet_bootstrap.cc:492] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:14.225725 20933 tablet_bootstrap.cc:492] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:14.274534 20933 ts_tablet_manager.cc:1403] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.111s	user 0.027s	sys 0.023s
I20260709 13:08:14.277827 20935 tablet_bootstrap.cc:654] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.285806 20937 tablet_bootstrap.cc:654] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.295159 20950 raft_consensus.cc:493] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.296033 20950 raft_consensus.cc:515] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.298146 20937 tablet_bootstrap.cc:492] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:14.299082 20937 ts_tablet_manager.cc:1403] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.002s
I20260709 13:08:14.300730 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27876839904a43d0aa2d0c1c992e5738" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:14.302667 20714 raft_consensus.cc:2468] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 0.
I20260709 13:08:14.303588 20844 raft_consensus.cc:493] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.304708 20844 raft_consensus.cc:515] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.307557 20933 raft_consensus.cc:359] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.308518 20933 raft_consensus.cc:385] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.308975 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c5385cd8d0742d7ab0cdd98e49e3a5f" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:14.308738 20937 raft_consensus.cc:359] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.309756 20937 raft_consensus.cc:385] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.309850 20789 raft_consensus.cc:2468] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 0.
I20260709 13:08:14.310122 20937 raft_consensus.cc:740] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.311959 20605 leader_election.cc:304] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:14.313203 20939 raft_consensus.cc:2804] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:14.313635 20939 raft_consensus.cc:493] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.313475 20937 consensus_queue.cc:260] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.308832 20933 raft_consensus.cc:740] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.315717 20933 consensus_queue.cc:260] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.316536 20715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c5385cd8d0742d7ab0cdd98e49e3a5f" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:14.312311 20537 catalog_manager.cc:5697] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 reported cstate change: term changed from 0 to 1, leader changed from <none> to fabb72df3c844f928dc062705b599484 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "fabb72df3c844f928dc062705b599484" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:14.314184 20939 raft_consensus.cc:3060] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.319494 20937 ts_tablet_manager.cc:1434] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.002s
I20260709 13:08:14.320907 20937 tablet_bootstrap.cc:492] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
W20260709 13:08:14.323273 20604 leader_election.cc:343] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.325467 20604 leader_election.cc:304] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:14.332304 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27876839904a43d0aa2d0c1c992e5738" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:14.333585 20937 tablet_bootstrap.cc:654] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:14.334398 20605 leader_election.cc:343] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.326434 20844 leader_election.cc:290] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.336023 20844 raft_consensus.cc:2804] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:14.338946 20933 ts_tablet_manager.cc:1434] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.006s
I20260709 13:08:14.341823 20844 raft_consensus.cc:493] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.342427 20844 raft_consensus.cc:3060] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.342588 20933 tablet_bootstrap.cc:492] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:14.342893 20950 leader_election.cc:290] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.347915 20919 raft_consensus.cc:493] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.348853 20937 tablet_bootstrap.cc:492] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:14.349573 20937 ts_tablet_manager.cc:1403] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.000s
I20260709 13:08:14.349001 20919 raft_consensus.cc:515] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.366906 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5967cbef3b48bc87d03588f9b4981d" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:14.367830 20714 raft_consensus.cc:2468] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 0.
I20260709 13:08:14.369310 20753 leader_election.cc:304] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: 
I20260709 13:08:14.375357 20919 leader_election.cc:290] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:14.378057 20919 raft_consensus.cc:2804] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:14.378539 20919 raft_consensus.cc:493] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.378979 20919 raft_consensus.cc:3060] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.379503 20935 tablet_bootstrap.cc:492] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:14.382992 20935 ts_tablet_manager.cc:1403] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.121s	user 0.022s	sys 0.006s
W20260709 13:08:14.384539 20586 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:14.381644 20939 raft_consensus.cc:515] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
W20260709 13:08:14.388312 20604 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42475, user_credentials={real_user=slave}} blocked reactor thread for 62976us
I20260709 13:08:14.390107 20639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5967cbef3b48bc87d03588f9b4981d" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:14.390919 20639 raft_consensus.cc:2468] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 0.
I20260709 13:08:14.391233 20935 raft_consensus.cc:359] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.392007 20935 raft_consensus.cc:385] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.392328 20935 raft_consensus.cc:740] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.392694 20933 tablet_bootstrap.cc:654] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.393224 20935 consensus_queue.cc:260] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.395807 20935 ts_tablet_manager.cc:1434] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:08:14.395407 20937 raft_consensus.cc:359] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.397226 20844 raft_consensus.cc:515] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.399323 20844 leader_election.cc:290] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Requested vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.399941 20935 tablet_bootstrap.cc:492] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:14.403291 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27876839904a43d0aa2d0c1c992e5738" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:14.396747 20937 raft_consensus.cc:385] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:08:14.405336 20605 leader_election.cc:343] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.403252 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27876839904a43d0aa2d0c1c992e5738" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:14.407126 20714 raft_consensus.cc:3060] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.413414 20935 tablet_bootstrap.cc:654] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.405707 20937 raft_consensus.cc:740] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.415026 20937 consensus_queue.cc:260] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.417982 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c5385cd8d0742d7ab0cdd98e49e3a5f" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:14.419257 20789 raft_consensus.cc:3060] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.419909 20715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c5385cd8d0742d7ab0cdd98e49e3a5f" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
W20260709 13:08:14.421875 20604 leader_election.cc:343] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.423565 20939 leader_election.cc:290] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Requested vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.430472 20919 raft_consensus.cc:515] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.446323 20933 tablet_bootstrap.cc:492] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:14.447281 20933 ts_tablet_manager.cc:1403] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.105s	user 0.022s	sys 0.015s
I20260709 13:08:14.449254 20937 ts_tablet_manager.cc:1434] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.099s	user 0.006s	sys 0.009s
I20260709 13:08:14.450078 20714 raft_consensus.cc:2468] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 1.
I20260709 13:08:14.451859 20604 leader_election.cc:304] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 602a0d7ba5984442865daf5400b33251
I20260709 13:08:14.451682 20933 raft_consensus.cc:359] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.452740 20933 raft_consensus.cc:385] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.453122 20933 raft_consensus.cc:740] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.453471 20939 raft_consensus.cc:2804] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:14.454628 20939 raft_consensus.cc:697] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Becoming Leader. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:14.455813 20939 consensus_queue.cc:237] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.456609 20935 tablet_bootstrap.cc:492] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:14.457403 20935 ts_tablet_manager.cc:1403] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.058s	user 0.023s	sys 0.023s
I20260709 13:08:14.461243 20789 raft_consensus.cc:2468] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 1.
I20260709 13:08:14.461612 20919 leader_election.cc:290] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:14.454178 20933 consensus_queue.cc:260] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.462962 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5967cbef3b48bc87d03588f9b4981d" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:14.484894 20714 raft_consensus.cc:3060] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.465476 20933 ts_tablet_manager.cc:1434] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:08:14.466657 20639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5967cbef3b48bc87d03588f9b4981d" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:14.487614 20639 raft_consensus.cc:3060] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.467459 20961 raft_consensus.cc:493] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.490434 20961 raft_consensus.cc:515] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.493742 20605 leader_election.cc:304] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:14.495672 20713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3b313ca1784f4194bae419112cb6cb" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:14.496219 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3b313ca1784f4194bae419112cb6cb" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:14.496682 20713 raft_consensus.cc:2468] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 0.
I20260709 13:08:14.496804 20933 tablet_bootstrap.cc:492] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:14.496896 20640 raft_consensus.cc:2468] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 0.
I20260709 13:08:14.495973 20961 leader_election.cc:290] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:14.498559 20753 leader_election.cc:304] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: 
I20260709 13:08:14.473778 20937 tablet_bootstrap.cc:492] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:14.502430 20961 raft_consensus.cc:2804] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:14.503146 20961 raft_consensus.cc:493] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.503858 20958 raft_consensus.cc:2804] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:14.505731 20933 tablet_bootstrap.cc:654] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.504518 20958 raft_consensus.cc:697] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Becoming Leader. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:14.461282 20935 raft_consensus.cc:359] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.506815 20935 raft_consensus.cc:385] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.507097 20935 raft_consensus.cc:740] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.507287 20958 consensus_queue.cc:237] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.513166 20961 raft_consensus.cc:3060] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.513778 20937 tablet_bootstrap.cc:654] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.517647 20639 raft_consensus.cc:2468] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 1.
I20260709 13:08:14.504524 20537 catalog_manager.cc:5697] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 reported cstate change: term changed from 0 to 1, leader changed from <none> to fabb72df3c844f928dc062705b599484 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "fabb72df3c844f928dc062705b599484" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:14.519451 20754 leader_election.cc:304] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:14.520498 20919 raft_consensus.cc:2804] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:14.521977 20935 consensus_queue.cc:260] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.529771 20714 raft_consensus.cc:2468] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 1.
I20260709 13:08:14.539078 20935 ts_tablet_manager.cc:1434] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.000s
I20260709 13:08:14.540934 20935 tablet_bootstrap.cc:492] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170: Bootstrap starting.
I20260709 13:08:14.540731 20933 tablet_bootstrap.cc:492] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:14.542717 20933 ts_tablet_manager.cc:1403] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.046s	user 0.021s	sys 0.007s
I20260709 13:08:14.550583 20919 raft_consensus.cc:697] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Becoming Leader. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Running, Role: LEADER
I20260709 13:08:14.551937 20919 consensus_queue.cc:237] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.552973 20933 raft_consensus.cc:359] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.553936 20933 raft_consensus.cc:385] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.554240 20933 raft_consensus.cc:740] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.555104 20933 consensus_queue.cc:260] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.555110 20535 catalog_manager.cc:5697] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 reported cstate change: term changed from 0 to 1, leader changed from <none> to fabb72df3c844f928dc062705b599484 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "fabb72df3c844f928dc062705b599484" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:14.560014 20935 tablet_bootstrap.cc:654] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.559998 20961 raft_consensus.cc:515] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.565678 20933 ts_tablet_manager.cc:1434] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 13:08:14.567443 20933 tablet_bootstrap.cc:492] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:14.567644 20921 raft_consensus.cc:493] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.568269 20921 raft_consensus.cc:515] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.569590 20639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3b313ca1784f4194bae419112cb6cb" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:14.570662 20639 raft_consensus.cc:3060] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.575961 20933 tablet_bootstrap.cc:654] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.578332 20921 leader_election.cc:290] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.583218 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c5385cd8d0742d7ab0cdd98e49e3a5f" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:14.584640 20789 raft_consensus.cc:2393] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9d91f828e5fa453982afb0ada2c45170 in current term 1: Already voted for candidate fabb72df3c844f928dc062705b599484 in this term.
I20260709 13:08:14.588591 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c5385cd8d0742d7ab0cdd98e49e3a5f" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:14.591081 20680 leader_election.cc:304] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170; no voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484
I20260709 13:08:14.593099 20921 raft_consensus.cc:3060] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.598822 20935 tablet_bootstrap.cc:492] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170: No bootstrap required, opened a new log
I20260709 13:08:14.599756 20935 ts_tablet_manager.cc:1403] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170: Time spent bootstrapping tablet: real 0.059s	user 0.021s	sys 0.003s
I20260709 13:08:14.603583 20933 tablet_bootstrap.cc:492] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:14.604318 20933 ts_tablet_manager.cc:1403] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.001s
I20260709 13:08:14.607609 20933 raft_consensus.cc:359] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.608496 20933 raft_consensus.cc:385] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.607926 20935 raft_consensus.cc:359] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.608810 20933 raft_consensus.cc:740] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.608914 20935 raft_consensus.cc:385] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.609282 20935 raft_consensus.cc:740] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.609694 20933 consensus_queue.cc:260] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.610093 20935 consensus_queue.cc:260] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.610445 20934 raft_consensus.cc:493] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.611377 20934 raft_consensus.cc:515] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.616365 20934 leader_election.cc:290] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.618794 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50538deb9d0641d6811ee0d600ec6c24" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:14.620527 20935 ts_tablet_manager.cc:1434] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.005s
W20260709 13:08:14.620752 20680 leader_election.cc:343] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.624670 20933 ts_tablet_manager.cc:1434] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:08:14.625584 20958 raft_consensus.cc:493] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.625952 20933 tablet_bootstrap.cc:492] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484: Bootstrap starting.
I20260709 13:08:14.626509 20958 raft_consensus.cc:515] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.629981 20958 leader_election.cc:290] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.632014 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d74c21c7a154be5a0057771d3e63f0c" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:14.632498 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50538deb9d0641d6811ee0d600ec6c24" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:14.633841 20640 raft_consensus.cc:2468] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 0.
I20260709 13:08:14.635120 20680 leader_election.cc:304] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 602a0d7ba5984442865daf5400b33251
I20260709 13:08:14.641611 20713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3b313ca1784f4194bae419112cb6cb" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:14.642515 20713 raft_consensus.cc:3060] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.642568 20974 raft_consensus.cc:2804] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:14.643908 20974 raft_consensus.cc:493] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.644258 20974 raft_consensus.cc:3060] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.643311 20961 leader_election.cc:290] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:14.632993 20714 raft_consensus.cc:2468] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 0.
I20260709 13:08:14.648389 20604 leader_election.cc:304] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:14.649005 20933 tablet_bootstrap.cc:654] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.650012 20921 raft_consensus.cc:2749] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.653774 20958 raft_consensus.cc:2804] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:14.659825 20958 raft_consensus.cc:493] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.660964 20958 raft_consensus.cc:3060] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.661278 20713 raft_consensus.cc:2468] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 1.
I20260709 13:08:14.662843 20753 leader_election.cc:304] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: 
I20260709 13:08:14.664481 20933 tablet_bootstrap.cc:492] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484: No bootstrap required, opened a new log
I20260709 13:08:14.665055 20933 ts_tablet_manager.cc:1403] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.001s
I20260709 13:08:14.653993 20974 raft_consensus.cc:515] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.632014 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d74c21c7a154be5a0057771d3e63f0c" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:14.667096 20789 raft_consensus.cc:2468] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 0.
I20260709 13:08:14.669160 20974 leader_election.cc:290] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.670666 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50538deb9d0641d6811ee0d600ec6c24" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:14.671013 20933 raft_consensus.cc:359] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.671680 20640 raft_consensus.cc:3060] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.672158 20933 raft_consensus.cc:385] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.672654 20933 raft_consensus.cc:740] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.673784 20933 consensus_queue.cc:260] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.670583 20789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50538deb9d0641d6811ee0d600ec6c24" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251"
W20260709 13:08:14.677619 20680 leader_election.cc:343] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.680420 20933 ts_tablet_manager.cc:1434] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.002s
I20260709 13:08:14.688704 20640 raft_consensus.cc:2468] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 1.
I20260709 13:08:14.670496 20961 raft_consensus.cc:2804] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:14.692507 20680 leader_election.cc:304] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 602a0d7ba5984442865daf5400b33251
I20260709 13:08:14.694895 20974 raft_consensus.cc:2804] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:14.695741 20974 raft_consensus.cc:697] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Becoming Leader. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Running, Role: LEADER
I20260709 13:08:14.696705 20974 consensus_queue.cc:237] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.700150 20961 raft_consensus.cc:697] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Becoming Leader. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Running, Role: LEADER
I20260709 13:08:14.701517 20961 consensus_queue.cc:237] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.702781 20969 raft_consensus.cc:493] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.703472 20969 raft_consensus.cc:515] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.707828 20639 raft_consensus.cc:2468] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 1.
I20260709 13:08:14.708068 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e8109831d6b4ada87ba80fa510b715d" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:14.708761 20714 raft_consensus.cc:2468] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 0.
I20260709 13:08:14.710258 20753 leader_election.cc:304] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: 
I20260709 13:08:14.711374 20969 leader_election.cc:290] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:14.712116 20969 raft_consensus.cc:2804] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:14.712446 20969 raft_consensus.cc:493] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.712714 20939 consensus_queue.cc:1048] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:14.712788 20969 raft_consensus.cc:3060] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.719978 20937 tablet_bootstrap.cc:492] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:14.720953 20937 ts_tablet_manager.cc:1403] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.247s	user 0.014s	sys 0.006s
I20260709 13:08:14.723388 20639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e8109831d6b4ada87ba80fa510b715d" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484" is_pre_election: true
I20260709 13:08:14.728506 20639 raft_consensus.cc:2468] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 0.
I20260709 13:08:14.739167 20939 consensus_queue.cc:1048] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:14.738876 20535 catalog_manager.cc:5697] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 reported cstate change: term changed from 0 to 1, leader changed from <none> to 602a0d7ba5984442865daf5400b33251 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "602a0d7ba5984442865daf5400b33251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:14.739645 20969 raft_consensus.cc:515] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.742861 20639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e8109831d6b4ada87ba80fa510b715d" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:14.743587 20639 raft_consensus.cc:3060] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.727487 20937 raft_consensus.cc:359] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.750133 20937 raft_consensus.cc:385] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.750559 20937 raft_consensus.cc:740] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.751547 20937 consensus_queue.cc:260] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.733700 20958 raft_consensus.cc:515] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.766983 20713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d74c21c7a154be5a0057771d3e63f0c" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:14.767833 20713 raft_consensus.cc:3060] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.770958 20937 ts_tablet_manager.cc:1434] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.049s	user 0.010s	sys 0.000s
I20260709 13:08:14.771229 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d74c21c7a154be5a0057771d3e63f0c" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:14.771926 20788 raft_consensus.cc:3060] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.772351 20937 tablet_bootstrap.cc:492] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:14.774312 20958 leader_election.cc:290] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Requested vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.775454 20537 catalog_manager.cc:5697] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130). New cstate: current_term: 1 leader_uuid: "9d91f828e5fa453982afb0ada2c45170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:14.780078 20535 catalog_manager.cc:5697] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 reported cstate change: term changed from 0 to 1, leader changed from <none> to 602a0d7ba5984442865daf5400b33251 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "602a0d7ba5984442865daf5400b33251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:14.783907 20639 raft_consensus.cc:2468] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 1.
I20260709 13:08:14.785936 20754 leader_election.cc:304] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:14.787648 20919 raft_consensus.cc:2804] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:14.788506 20919 raft_consensus.cc:697] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Becoming Leader. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Running, Role: LEADER
I20260709 13:08:14.789955 20919 consensus_queue.cc:237] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.760532 20969 leader_election.cc:290] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 1 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
W20260709 13:08:14.794562 20753 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.89.130:42475, user_credentials={real_user=slave}} blocked reactor thread for 84435.3us
I20260709 13:08:14.812920 20715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e8109831d6b4ada87ba80fa510b715d" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:14.814034 20715 raft_consensus.cc:3060] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.832858 20939 raft_consensus.cc:493] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.833926 20939 raft_consensus.cc:515] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.838897 20939 leader_election.cc:290] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.839232 20787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0ebe3dbfe4a3b9d21682bf14d1333" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251" is_pre_election: true
I20260709 13:08:14.840261 20787 raft_consensus.cc:2468] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 0.
I20260709 13:08:14.841148 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0ebe3dbfe4a3b9d21682bf14d1333" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170" is_pre_election: true
I20260709 13:08:14.842037 20605 leader_election.cc:304] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:14.841971 20714 raft_consensus.cc:2468] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 0.
I20260709 13:08:14.843202 20939 raft_consensus.cc:2804] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:14.843622 20939 raft_consensus.cc:493] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.844008 20939 raft_consensus.cc:3060] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.840476 20937 tablet_bootstrap.cc:654] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.853930 20939 raft_consensus.cc:515] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.857448 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0ebe3dbfe4a3b9d21682bf14d1333" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:14.857749 20787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0ebe3dbfe4a3b9d21682bf14d1333" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:14.858372 20714 raft_consensus.cc:3060] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.858434 20787 raft_consensus.cc:3060] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.872925 20939 leader_election.cc:290] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Requested vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:14.877291 20714 raft_consensus.cc:2468] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 1.
I20260709 13:08:14.880496 20604 leader_election.cc:304] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:14.882792 20844 raft_consensus.cc:2804] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:14.883343 20844 raft_consensus.cc:697] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Becoming Leader. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:14.884191 20844 consensus_queue.cc:237] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.887202 20787 raft_consensus.cc:2468] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 1.
I20260709 13:08:14.892746 20713 raft_consensus.cc:2468] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 1.
I20260709 13:08:14.894263 20604 leader_election.cc:304] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:14.895820 20958 raft_consensus.cc:2804] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:14.896544 20958 raft_consensus.cc:697] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Becoming Leader. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:14.896184 20985 consensus_queue.cc:1048] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:14.897801 20958 consensus_queue.cc:237] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.886549 20715 raft_consensus.cc:2468] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 1.
I20260709 13:08:14.910142 20537 catalog_manager.cc:5697] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 reported cstate change: term changed from 0 to 1, leader changed from <none> to fabb72df3c844f928dc062705b599484 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "fabb72df3c844f928dc062705b599484" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:14.928678 20937 tablet_bootstrap.cc:492] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:14.929729 20937 ts_tablet_manager.cc:1403] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.158s	user 0.014s	sys 0.007s
I20260709 13:08:14.903317 20788 raft_consensus.cc:2468] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 1.
I20260709 13:08:14.935264 20937 raft_consensus.cc:359] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.936264 20937 raft_consensus.cc:385] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.936617 20937 raft_consensus.cc:740] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.937501 20937 consensus_queue.cc:260] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.940320 20937 ts_tablet_manager.cc:1434] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:08:14.941771 20937 tablet_bootstrap.cc:492] T 50538deb9d0641d6811ee0d600ec6c24 P 602a0d7ba5984442865daf5400b33251: Bootstrap starting.
I20260709 13:08:14.949153 20985 consensus_queue.cc:1048] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:14.952029 20535 catalog_manager.cc:5697] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 reported cstate change: term changed from 0 to 1, leader changed from <none> to 602a0d7ba5984442865daf5400b33251 (127.7.89.131). New cstate: current_term: 1 leader_uuid: "602a0d7ba5984442865daf5400b33251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:14.958175 20937 tablet_bootstrap.cc:654] T 50538deb9d0641d6811ee0d600ec6c24 P 602a0d7ba5984442865daf5400b33251: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.971629 20937 tablet_bootstrap.cc:492] T 50538deb9d0641d6811ee0d600ec6c24 P 602a0d7ba5984442865daf5400b33251: No bootstrap required, opened a new log
I20260709 13:08:14.972214 20937 ts_tablet_manager.cc:1403] T 50538deb9d0641d6811ee0d600ec6c24 P 602a0d7ba5984442865daf5400b33251: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.004s
I20260709 13:08:14.974388 20536 catalog_manager.cc:5697] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 reported cstate change: term changed from 0 to 1, leader changed from <none> to fabb72df3c844f928dc062705b599484 (127.7.89.129). New cstate: current_term: 1 leader_uuid: "fabb72df3c844f928dc062705b599484" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:14.975497 20937 raft_consensus.cc:359] T 50538deb9d0641d6811ee0d600ec6c24 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.976586 20937 raft_consensus.cc:385] T 50538deb9d0641d6811ee0d600ec6c24 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.976878 20937 raft_consensus.cc:740] T 50538deb9d0641d6811ee0d600ec6c24 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.977717 20937 consensus_queue.cc:260] T 50538deb9d0641d6811ee0d600ec6c24 P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:14.980222 20937 ts_tablet_manager.cc:1434] T 50538deb9d0641d6811ee0d600ec6c24 P 602a0d7ba5984442865daf5400b33251: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:08:14.996934 20919 consensus_queue.cc:1048] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.012723 20919 consensus_queue.cc:1048] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.042089 20985 consensus_queue.cc:1048] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:15.046154 20788 raft_consensus.cc:3060] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.062984 20950 consensus_queue.cc:1048] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.089110 20787 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e8109831d6b4ada87ba80fa510b715d"
dest_uuid: "602a0d7ba5984442865daf5400b33251"
mode: GRACEFUL
new_leader_uuid: "fabb72df3c844f928dc062705b599484"
 from {username='slave'} at 127.0.0.1:44546
I20260709 13:08:15.089848 20787 raft_consensus.cc:606] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Received request to transfer leadership to TS fabb72df3c844f928dc062705b599484
I20260709 13:08:15.099061 20639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27876839904a43d0aa2d0c1c992e5738"
dest_uuid: "fabb72df3c844f928dc062705b599484"
mode: GRACEFUL
new_leader_uuid: "9d91f828e5fa453982afb0ada2c45170"
 from {username='slave'} at 127.0.0.1:38856
I20260709 13:08:15.099852 20639 raft_consensus.cc:606] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Received request to transfer leadership to TS 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:15.107280 20788 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee5967cbef3b48bc87d03588f9b4981d"
dest_uuid: "602a0d7ba5984442865daf5400b33251"
mode: GRACEFUL
new_leader_uuid: "9d91f828e5fa453982afb0ada2c45170"
 from {username='slave'} at 127.0.0.1:44546
I20260709 13:08:15.108078 20788 raft_consensus.cc:606] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Received request to transfer leadership to TS 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:15.116660 20788 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c3b313ca1784f4194bae419112cb6cb"
dest_uuid: "602a0d7ba5984442865daf5400b33251"
mode: GRACEFUL
new_leader_uuid: "9d91f828e5fa453982afb0ada2c45170"
 from {username='slave'} at 127.0.0.1:44546
I20260709 13:08:15.117560 20788 raft_consensus.cc:606] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Received request to transfer leadership to TS 9d91f828e5fa453982afb0ada2c45170
I20260709 13:08:15.128932 20639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58d0ebe3dbfe4a3b9d21682bf14d1333"
dest_uuid: "fabb72df3c844f928dc062705b599484"
mode: GRACEFUL
new_leader_uuid: "602a0d7ba5984442865daf5400b33251"
 from {username='slave'} at 127.0.0.1:38856
I20260709 13:08:15.129796 20639 raft_consensus.cc:606] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Received request to transfer leadership to TS 602a0d7ba5984442865daf5400b33251
I20260709 13:08:15.158697 20969 consensus_queue.cc:1048] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.159922 20919 consensus_queue.cc:1048] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:15.166405 20974 consensus_queue.cc:1048] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.179996 20969 raft_consensus.cc:993] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251: : Instructing follower 9d91f828e5fa453982afb0ada2c45170 to start an election
I20260709 13:08:15.180827 20969 raft_consensus.cc:1081] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Signalling peer 9d91f828e5fa453982afb0ada2c45170 to start an election
I20260709 13:08:15.183548 20714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c3b313ca1784f4194bae419112cb6cb"
dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
 from {username='slave'} at 127.0.0.1:43996
I20260709 13:08:15.184234 20714 raft_consensus.cc:493] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:15.184621 20714 raft_consensus.cc:3060] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.196029 20714 raft_consensus.cc:515] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.197201 20713 raft_consensus.cc:1240] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Rejecting Update request from peer 602a0d7ba5984442865daf5400b33251 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:15.201011 20638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3b313ca1784f4194bae419112cb6cb" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:15.201705 20919 consensus_queue.cc:1059] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.202553 20714 leader_election.cc:290] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 2 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:15.203227 20969 raft_consensus.cc:3055] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:15.203758 20969 raft_consensus.cc:740] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Running, Role: LEADER
I20260709 13:08:15.205729 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3b313ca1784f4194bae419112cb6cb" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:15.205928 20969 consensus_queue.cc:260] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.209574 20969 raft_consensus.cc:3060] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.217480 20787 raft_consensus.cc:3060] T 50538deb9d0641d6811ee0d600ec6c24 P 602a0d7ba5984442865daf5400b33251 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.236614 20999 consensus_queue.cc:1048] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.243292 20788 raft_consensus.cc:2468] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 2.
I20260709 13:08:15.249887 20680 leader_election.cc:304] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: fabb72df3c844f928dc062705b599484
I20260709 13:08:15.251071 20999 raft_consensus.cc:2804] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:15.251641 20999 raft_consensus.cc:697] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [term 2 LEADER]: Becoming Leader. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Running, Role: LEADER
I20260709 13:08:15.252584 20999 consensus_queue.cc:237] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.262262 20537 catalog_manager.cc:5697] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 reported cstate change: term changed from 1 to 2, leader changed from 602a0d7ba5984442865daf5400b33251 (127.7.89.131) to 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "9d91f828e5fa453982afb0ada2c45170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:15.318589 20969 consensus_queue.cc:1048] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:15.330960 20919 consensus_queue.cc:1048] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.342382 20919 raft_consensus.cc:993] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251: : Instructing follower fabb72df3c844f928dc062705b599484 to start an election
I20260709 13:08:15.342999 20919 raft_consensus.cc:1081] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Signalling peer fabb72df3c844f928dc062705b599484 to start an election
I20260709 13:08:15.345343 20640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e8109831d6b4ada87ba80fa510b715d"
dest_uuid: "fabb72df3c844f928dc062705b599484"
 from {username='slave'} at 127.0.0.1:38842
I20260709 13:08:15.345997 20640 raft_consensus.cc:493] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:15.346391 20640 raft_consensus.cc:3060] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.353000 20640 raft_consensus.cc:515] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.355080 20640 leader_election.cc:290] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 2 election: Requested vote from peers 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:15.356273 20714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e8109831d6b4ada87ba80fa510b715d" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:15.357124 20714 raft_consensus.cc:3060] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.356313 20787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e8109831d6b4ada87ba80fa510b715d" candidate_uuid: "fabb72df3c844f928dc062705b599484" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:15.358000 20787 raft_consensus.cc:3055] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:15.358342 20787 raft_consensus.cc:740] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Running, Role: LEADER
I20260709 13:08:15.359313 20787 consensus_queue.cc:260] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.360757 20787 raft_consensus.cc:3060] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.367080 20714 raft_consensus.cc:2468] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 2.
I20260709 13:08:15.369132 20787 raft_consensus.cc:2468] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabb72df3c844f928dc062705b599484 in term 2.
I20260709 13:08:15.369233 20604 leader_election.cc:304] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:15.370659 20985 raft_consensus.cc:2804] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:15.371229 20985 raft_consensus.cc:697] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [term 2 LEADER]: Becoming Leader. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:15.372217 20985 consensus_queue.cc:237] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.380874 20535 catalog_manager.cc:5697] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 reported cstate change: term changed from 1 to 2, leader changed from 602a0d7ba5984442865daf5400b33251 (127.7.89.131) to fabb72df3c844f928dc062705b599484 (127.7.89.129). New cstate: current_term: 2 leader_uuid: "fabb72df3c844f928dc062705b599484" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:15.392765 20985 consensus_queue.cc:1048] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.399808 20958 consensus_queue.cc:1048] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.411993 20958 raft_consensus.cc:993] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484: : Instructing follower 602a0d7ba5984442865daf5400b33251 to start an election
I20260709 13:08:15.412621 20958 raft_consensus.cc:1081] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Signalling peer 602a0d7ba5984442865daf5400b33251 to start an election
I20260709 13:08:15.429287 20789 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58d0ebe3dbfe4a3b9d21682bf14d1333"
dest_uuid: "602a0d7ba5984442865daf5400b33251"
 from {username='slave'} at 127.0.0.1:44530
I20260709 13:08:15.430110 20789 raft_consensus.cc:493] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:15.430653 20789 raft_consensus.cc:3060] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.430990 20950 consensus_queue.cc:1048] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.467734 20789 raft_consensus.cc:515] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.471184 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0ebe3dbfe4a3b9d21682bf14d1333" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:15.471895 20640 raft_consensus.cc:3055] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:15.472214 20640 raft_consensus.cc:740] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:15.473049 20640 consensus_queue.cc:260] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.474009 20640 raft_consensus.cc:3060] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.476914 20950 consensus_queue.cc:1048] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.490010 20789 leader_election.cc:290] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 2 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:15.494194 20713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0ebe3dbfe4a3b9d21682bf14d1333" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:15.495898 20713 raft_consensus.cc:3060] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.506314 20713 raft_consensus.cc:2468] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 2.
I20260709 13:08:15.509076 20753 leader_election.cc:304] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, 9d91f828e5fa453982afb0ada2c45170; no voters: 
I20260709 13:08:15.510197 20977 raft_consensus.cc:2804] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:15.510767 20977 raft_consensus.cc:697] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [term 2 LEADER]: Becoming Leader. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Running, Role: LEADER
I20260709 13:08:15.511660 20977 consensus_queue.cc:237] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.513306 20640 raft_consensus.cc:2468] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 2.
I20260709 13:08:15.522253 20537 catalog_manager.cc:5697] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 reported cstate change: term changed from 1 to 2, leader changed from fabb72df3c844f928dc062705b599484 (127.7.89.129) to 602a0d7ba5984442865daf5400b33251 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "602a0d7ba5984442865daf5400b33251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:15.594064 20919 raft_consensus.cc:993] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251: : Instructing follower 9d91f828e5fa453982afb0ada2c45170 to start an election
I20260709 13:08:15.594645 20977 raft_consensus.cc:1081] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Signalling peer 9d91f828e5fa453982afb0ada2c45170 to start an election
I20260709 13:08:15.596271 20713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee5967cbef3b48bc87d03588f9b4981d"
dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
 from {username='slave'} at 127.0.0.1:43996
I20260709 13:08:15.596818 20713 raft_consensus.cc:493] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:15.597214 20713 raft_consensus.cc:3060] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.603626 20713 raft_consensus.cc:515] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.605587 20713 leader_election.cc:290] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 2 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:15.606822 20640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5967cbef3b48bc87d03588f9b4981d" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:15.606964 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5967cbef3b48bc87d03588f9b4981d" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:15.607666 20640 raft_consensus.cc:3060] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.607782 20788 raft_consensus.cc:3055] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:15.608389 20788 raft_consensus.cc:740] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Running, Role: LEADER
I20260709 13:08:15.609323 20788 consensus_queue.cc:260] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.610826 20788 raft_consensus.cc:3060] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.612001 20844 raft_consensus.cc:993] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484: : Instructing follower 9d91f828e5fa453982afb0ada2c45170 to start an election
I20260709 13:08:15.612596 20844 raft_consensus.cc:1081] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Signalling peer 9d91f828e5fa453982afb0ada2c45170 to start an election
I20260709 13:08:15.614441 20713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "27876839904a43d0aa2d0c1c992e5738"
dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
 from {username='slave'} at 127.0.0.1:44002
I20260709 13:08:15.615195 20713 raft_consensus.cc:493] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:15.615553 20713 raft_consensus.cc:3060] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.617074 20640 raft_consensus.cc:2468] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 2.
I20260709 13:08:15.618499 20680 leader_election.cc:304] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:15.619573 20999 raft_consensus.cc:2804] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:15.620090 20999 raft_consensus.cc:697] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 2 LEADER]: Becoming Leader. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Running, Role: LEADER
I20260709 13:08:15.620956 20999 consensus_queue.cc:237] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.623082 20788 raft_consensus.cc:2468] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 2.
I20260709 13:08:15.627292 20713 raft_consensus.cc:515] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.629657 20713 leader_election.cc:290] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 2 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897)
I20260709 13:08:15.632705 20638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27876839904a43d0aa2d0c1c992e5738" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:15.633443 20638 raft_consensus.cc:3055] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:15.633818 20638 raft_consensus.cc:740] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:15.634164 20788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27876839904a43d0aa2d0c1c992e5738" candidate_uuid: "9d91f828e5fa453982afb0ada2c45170" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "602a0d7ba5984442865daf5400b33251"
I20260709 13:08:15.633797 20537 catalog_manager.cc:5697] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 reported cstate change: term changed from 1 to 2, leader changed from 602a0d7ba5984442865daf5400b33251 (127.7.89.131) to 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "9d91f828e5fa453982afb0ada2c45170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:15.635059 20788 raft_consensus.cc:3060] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.635243 20638 consensus_queue.cc:260] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.636994 20638 raft_consensus.cc:3060] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.646566 20638 raft_consensus.cc:2468] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 2.
I20260709 13:08:15.647868 20680 leader_election.cc:304] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d91f828e5fa453982afb0ada2c45170, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:15.648717 20999 raft_consensus.cc:2804] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:15.648573 20788 raft_consensus.cc:2468] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d91f828e5fa453982afb0ada2c45170 in term 2.
I20260709 13:08:15.649577 20999 raft_consensus.cc:697] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [term 2 LEADER]: Becoming Leader. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Running, Role: LEADER
I20260709 13:08:15.650470 20999 consensus_queue.cc:237] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:15.661989 20537 catalog_manager.cc:5697] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 reported cstate change: term changed from 1 to 2, leader changed from fabb72df3c844f928dc062705b599484 (127.7.89.129) to 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130). New cstate: current_term: 2 leader_uuid: "9d91f828e5fa453982afb0ada2c45170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:15.694339 20788 raft_consensus.cc:1275] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Refusing update from remote peer 9d91f828e5fa453982afb0ada2c45170: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.696249 20999 consensus_queue.cc:1048] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:15.709090 20638 raft_consensus.cc:3060] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.724395 20638 raft_consensus.cc:1275] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Refusing update from remote peer 9d91f828e5fa453982afb0ada2c45170: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.726418 21007 consensus_queue.cc:1048] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:15.859234 20788 raft_consensus.cc:1275] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Refusing update from remote peer fabb72df3c844f928dc062705b599484: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.860677 20844 consensus_queue.cc:1048] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:15.870828 20713 raft_consensus.cc:1275] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Refusing update from remote peer fabb72df3c844f928dc062705b599484: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.873041 20950 consensus_queue.cc:1048] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:15.955193 20713 raft_consensus.cc:1275] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Refusing update from remote peer 602a0d7ba5984442865daf5400b33251: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.956902 20919 consensus_queue.cc:1048] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:15.958534 20638 raft_consensus.cc:1275] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Refusing update from remote peer 602a0d7ba5984442865daf5400b33251: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.960389 20919 consensus_queue.cc:1048] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:16.065454 20638 raft_consensus.cc:1275] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Refusing update from remote peer 9d91f828e5fa453982afb0ada2c45170: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:16.067068 21007 consensus_queue.cc:1048] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:16.078941 20788 raft_consensus.cc:1275] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Refusing update from remote peer 9d91f828e5fa453982afb0ada2c45170: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:16.081396 20974 consensus_queue.cc:1048] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:16.202716 20638 raft_consensus.cc:1275] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Refusing update from remote peer 9d91f828e5fa453982afb0ada2c45170: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:16.204892 20999 consensus_queue.cc:1048] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:16.220948 20788 raft_consensus.cc:1275] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Refusing update from remote peer 9d91f828e5fa453982afb0ada2c45170: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:16.222882 20999 consensus_queue.cc:1048] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:18.134950 20537 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37338:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:08:18.163967 20537 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37338:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:08:18.216674  7526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:08:18.602335 21047 log.cc:927] Time spent T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:08:18.631788 21054 tablet.cc:1724] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:08:18.635358 21044 log.cc:927] Time spent T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:08:18.725987 21067 tablet.cc:1724] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:18.771201 21068 tablet.cc:1724] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:08:18.805339 21060 log.cc:927] Time spent T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170: Append to log took a long time: real 0.122s	user 0.006s	sys 0.000s
I20260709 13:08:18.820629 21066 tablet.cc:1724] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:18.851027 21074 tablet.cc:1724] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:18.852140 20640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40d9711960b64959a444115884f4a687"
dest_uuid: "fabb72df3c844f928dc062705b599484"
mode: GRACEFUL
new_leader_uuid: "602a0d7ba5984442865daf5400b33251"
 from {username='slave'} at 127.0.0.1:36626
I20260709 13:08:18.853343 20640 raft_consensus.cc:606] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Received request to transfer leadership to TS 602a0d7ba5984442865daf5400b33251
I20260709 13:08:18.867835 21075 tablet.cc:1724] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:08:18.870770 21051 log.cc:927] Time spent T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484: Append to log took a long time: real 0.187s	user 0.000s	sys 0.003s
I20260709 13:08:18.911017 21072 tablet.cc:1724] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:18.916599 21068 tablet.cc:1724] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:08:18.947144 21064 log.cc:927] Time spent T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251: Append to log took a long time: real 0.253s	user 0.003s	sys 0.000s
I20260709 13:08:18.951979 21054 tablet.cc:1724] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:18.952826 21053 raft_consensus.cc:993] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484: : Instructing follower 602a0d7ba5984442865daf5400b33251 to start an election
I20260709 13:08:18.954035 21053 raft_consensus.cc:1081] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Signalling peer 602a0d7ba5984442865daf5400b33251 to start an election
I20260709 13:08:18.968169 21068 tablet.cc:1724] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:18.988296 21068 tablet.cc:1724] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:18.989413 20787 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40d9711960b64959a444115884f4a687"
dest_uuid: "602a0d7ba5984442865daf5400b33251"
 from {username='slave'} at 127.0.0.1:44530
I20260709 13:08:18.990362 20787 raft_consensus.cc:493] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:18.990932 20787 raft_consensus.cc:3060] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:18.998169 21054 tablet.cc:1724] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.007558 21077 tablet.cc:1724] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.020000 21081 tablet.cc:1724] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:18.989418 21083 tablet.cc:1724] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.042659 21066 tablet.cc:1724] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.047392 21084 tablet.cc:1724] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.056964 20787 raft_consensus.cc:515] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:19.060190 20787 leader_election.cc:290] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 2 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:19.061839 20710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d9711960b64959a444115884f4a687" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:19.063190 20710 raft_consensus.cc:3060] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.066105 20635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40d9711960b64959a444115884f4a687" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:19.067006 20635 raft_consensus.cc:3055] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:19.067354 20635 raft_consensus.cc:740] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fabb72df3c844f928dc062705b599484, State: Running, Role: LEADER
I20260709 13:08:19.068480 20635 consensus_queue.cc:260] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:19.070973 20635 raft_consensus.cc:3060] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.073436 20537 catalog_manager.cc:5789] TS 602a0d7ba5984442865daf5400b33251 (127.7.89.131:42897) does not have the latest schema for tablet 58d0ebe3dbfe4a3b9d21682bf14d1333 (table soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99]). Expected version 1 got 0
I20260709 13:08:19.067704 21074 tablet.cc:1724] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.104574 21067 tablet.cc:1724] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.098752 21086 tablet.cc:1724] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.093376 21075 tablet.cc:1724] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.090837 21072 tablet.cc:1724] T 50538deb9d0641d6811ee0d600ec6c24 P 602a0d7ba5984442865daf5400b33251: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.139219 20712 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf8d4d44b9b943c5b5858f393fc5c8fb"
dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
mode: GRACEFUL
new_leader_uuid: "602a0d7ba5984442865daf5400b33251"
 from {username='slave'} at 127.0.0.1:59108
I20260709 13:08:19.140116 20712 raft_consensus.cc:606] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Received request to transfer leadership to TS 602a0d7ba5984442865daf5400b33251
I20260709 13:08:19.143671 21066 tablet.cc:1724] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.146178 20635 raft_consensus.cc:2468] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 2.
I20260709 13:08:19.146656  7526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:08:19.159642  7526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:19.152809 20710 raft_consensus.cc:2468] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 2.
I20260709 13:08:19.160713 20754 leader_election.cc:304] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:19.162027 21079 raft_consensus.cc:2804] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:19.162657 21079 raft_consensus.cc:697] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [term 2 LEADER]: Becoming Leader. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Running, Role: LEADER
I20260709 13:08:19.163786 21077 tablet.cc:1724] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.163884 21079 consensus_queue.cc:237] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:19.145709 21067 tablet.cc:1724] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.154363 21075 tablet.cc:1724] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.153216 21086 tablet.cc:1724] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:19.211217 20537 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a: soft_deleted_table [id=c9e2fcf295bf4af4b1f814e2d5d71b99] alter complete (version 1)
I20260709 13:08:19.221740 21078 raft_consensus.cc:993] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170: : Instructing follower 602a0d7ba5984442865daf5400b33251 to start an election
I20260709 13:08:19.222486 21040 raft_consensus.cc:1081] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Signalling peer 602a0d7ba5984442865daf5400b33251 to start an election
I20260709 13:08:19.225148 20787 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf8d4d44b9b943c5b5858f393fc5c8fb"
dest_uuid: "602a0d7ba5984442865daf5400b33251"
 from {username='slave'} at 127.0.0.1:44516
I20260709 13:08:19.225855 20787 raft_consensus.cc:493] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:19.226261 20787 raft_consensus.cc:3060] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.228619 20537 catalog_manager.cc:5697] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 reported cstate change: term changed from 1 to 2, leader changed from fabb72df3c844f928dc062705b599484 (127.7.89.129) to 602a0d7ba5984442865daf5400b33251 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "602a0d7ba5984442865daf5400b33251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:19.235117 20787 raft_consensus.cc:515] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:19.238171 20787 leader_election.cc:290] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 2 election: Requested vote from peers fabb72df3c844f928dc062705b599484 (127.7.89.129:45101), 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475)
I20260709 13:08:19.238672 20635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8d4d44b9b943c5b5858f393fc5c8fb" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fabb72df3c844f928dc062705b599484"
I20260709 13:08:19.238974 20710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf8d4d44b9b943c5b5858f393fc5c8fb" candidate_uuid: "602a0d7ba5984442865daf5400b33251" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d91f828e5fa453982afb0ada2c45170"
I20260709 13:08:19.239620 20635 raft_consensus.cc:3060] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.239674 20710 raft_consensus.cc:3055] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:19.240259 20710 raft_consensus.cc:740] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d91f828e5fa453982afb0ada2c45170, State: Running, Role: LEADER
I20260709 13:08:19.241175 20710 consensus_queue.cc:260] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:19.242506 20710 raft_consensus.cc:3060] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:19.248346 20635 raft_consensus.cc:2468] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 2.
I20260709 13:08:19.249384 20710 raft_consensus.cc:2468] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602a0d7ba5984442865daf5400b33251 in term 2.
I20260709 13:08:19.250036 20754 leader_election.cc:304] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602a0d7ba5984442865daf5400b33251, fabb72df3c844f928dc062705b599484; no voters: 
I20260709 13:08:19.251143 21079 raft_consensus.cc:2804] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:19.251593 21079 raft_consensus.cc:697] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [term 2 LEADER]: Becoming Leader. State: Replica: 602a0d7ba5984442865daf5400b33251, State: Running, Role: LEADER
I20260709 13:08:19.252501 21079 consensus_queue.cc:237] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } }
I20260709 13:08:19.263608 20537 catalog_manager.cc:5697] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 reported cstate change: term changed from 1 to 2, leader changed from 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130) to 602a0d7ba5984442865daf5400b33251 (127.7.89.131). New cstate: current_term: 2 leader_uuid: "602a0d7ba5984442865daf5400b33251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602a0d7ba5984442865daf5400b33251" member_type: VOTER last_known_addr { host: "127.7.89.131" port: 42897 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:19.687587 20710 raft_consensus.cc:1275] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Refusing update from remote peer 602a0d7ba5984442865daf5400b33251: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:19.689142 21079 consensus_queue.cc:1048] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:19.703791 20635 raft_consensus.cc:1275] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Refusing update from remote peer 602a0d7ba5984442865daf5400b33251: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:19.705771 21095 consensus_queue.cc:1048] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:19.825403 20710 raft_consensus.cc:1275] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Refusing update from remote peer 602a0d7ba5984442865daf5400b33251: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:19.827008 21095 consensus_queue.cc:1048] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d91f828e5fa453982afb0ada2c45170" member_type: VOTER last_known_addr { host: "127.7.89.130" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:19.837646 20635 raft_consensus.cc:1275] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Refusing update from remote peer 602a0d7ba5984442865daf5400b33251: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:19.840725 21079 consensus_queue.cc:1048] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb72df3c844f928dc062705b599484" member_type: VOTER last_known_addr { host: "127.7.89.129" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:20.225317  7526 tablet_server.cc:179] TabletServer@127.7.89.129:0 shutting down...
W20260709 13:08:20.250957 20754 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 11 similar messages]
W20260709 13:08:20.270596 20754 consensus_peers.cc:597] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 -> Peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Couldn't send request to peer fabb72df3c844f928dc062705b599484. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.272326 20754 consensus_peers.cc:597] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 -> Peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Couldn't send request to peer fabb72df3c844f928dc062705b599484. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.273342 20680 consensus_peers.cc:597] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 -> Peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Couldn't send request to peer fabb72df3c844f928dc062705b599484. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.290264 20754 consensus_peers.cc:597] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 -> Peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Couldn't send request to peer fabb72df3c844f928dc062705b599484. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.291296  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:20.292163  7526 tablet_replica.cc:333] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.293087  7526 raft_consensus.cc:2243] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.293695  7526 raft_consensus.cc:2272] T 2c3b313ca1784f4194bae419112cb6cb P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.296636  7526 tablet_replica.cc:333] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.297603  7526 raft_consensus.cc:2243] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.298142  7526 raft_consensus.cc:2272] T 50538deb9d0641d6811ee0d600ec6c24 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.300657  7526 tablet_replica.cc:333] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.301331  7526 raft_consensus.cc:2243] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.302416  7526 raft_consensus.cc:2272] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.305102  7526 tablet_replica.cc:333] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.305800  7526 raft_consensus.cc:2243] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.307013  7526 raft_consensus.cc:2272] T 5d74c21c7a154be5a0057771d3e63f0c P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.309711  7526 tablet_replica.cc:333] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.310516  7526 raft_consensus.cc:2243] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.311535  7526 raft_consensus.cc:2272] T c2f5838bfc4240de8f14073d0c85fcab P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.314478  7526 tablet_replica.cc:333] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.315306  7526 raft_consensus.cc:2243] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.315791  7526 raft_consensus.cc:2272] T 27876839904a43d0aa2d0c1c992e5738 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.318163  7526 tablet_replica.cc:333] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.319078  7526 raft_consensus.cc:2243] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.319633  7526 raft_consensus.cc:2272] T ee5967cbef3b48bc87d03588f9b4981d P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.322533  7526 tablet_replica.cc:333] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484: stopping tablet replica
W20260709 13:08:20.322921 20680 consensus_peers.cc:597] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 -> Peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Couldn't send request to peer fabb72df3c844f928dc062705b599484. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.323460  7526 raft_consensus.cc:2243] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.324561  7526 raft_consensus.cc:2272] T 81ca28ac68414a7fafb169b5d088b6d6 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.327908  7526 tablet_replica.cc:333] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.328814  7526 raft_consensus.cc:2243] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.329591  7526 raft_consensus.cc:2272] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:20.332679 20680 consensus_peers.cc:597] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 -> Peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Couldn't send request to peer fabb72df3c844f928dc062705b599484. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.333091  7526 tablet_replica.cc:333] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.334178  7526 raft_consensus.cc:2243] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.334892  7526 raft_consensus.cc:2272] T eea8f92e67394209ae7554aa9f4a0ac6 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.338440  7526 tablet_replica.cc:333] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.339262  7526 raft_consensus.cc:2243] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.339921  7526 raft_consensus.cc:2272] T 63ac558c2e934cbdb38fa0c88858f4b7 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.342998  7526 tablet_replica.cc:333] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.343727  7526 raft_consensus.cc:2243] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.344328  7526 raft_consensus.cc:2272] T 16975b7e56d04f8783a7b1250c87e478 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.347203  7526 tablet_replica.cc:333] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.347896  7526 raft_consensus.cc:2243] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.348696  7526 raft_consensus.cc:2272] T 9f44644e1f5f4610b1dab75f07c2993e P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.351730  7526 tablet_replica.cc:333] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.352453  7526 raft_consensus.cc:2243] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.352919  7526 raft_consensus.cc:2272] T 40d9711960b64959a444115884f4a687 P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.356254  7526 tablet_replica.cc:333] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.357137  7526 raft_consensus.cc:2243] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.358400  7526 raft_consensus.cc:2272] T 98ba4deec7b446128f9c734b613ca13d P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.361433  7526 tablet_replica.cc:333] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.362238  7526 raft_consensus.cc:2243] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.363386  7526 raft_consensus.cc:2272] T 6e8109831d6b4ada87ba80fa510b715d P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.366315  7526 tablet_replica.cc:333] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.367182  7526 raft_consensus.cc:2243] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.368248  7526 raft_consensus.cc:2272] T 769284c8e4f34e83a6ed1163c5ca3a85 P fabb72df3c844f928dc062705b599484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.371433  7526 tablet_replica.cc:333] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484: stopping tablet replica
I20260709 13:08:20.372092  7526 raft_consensus.cc:2243] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.372673  7526 raft_consensus.cc:2272] T cf8d4d44b9b943c5b5858f393fc5c8fb P fabb72df3c844f928dc062705b599484 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.449558  7526 tablet_server.cc:196] TabletServer@127.7.89.129:0 shutdown complete.
W20260709 13:08:20.453454 20680 consensus_peers.cc:597] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 -> Peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Couldn't send request to peer fabb72df3c844f928dc062705b599484. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:45101: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.456965 20680 consensus_peers.cc:597] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 -> Peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Couldn't send request to peer fabb72df3c844f928dc062705b599484. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:45101: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.481900 20680 consensus_peers.cc:597] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 -> Peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Couldn't send request to peer fabb72df3c844f928dc062705b599484. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:45101: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.501060  7526 tablet_server.cc:179] TabletServer@127.7.89.130:0 shutting down...
W20260709 13:08:20.506691 20754 consensus_peers.cc:597] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 -> Peer fabb72df3c844f928dc062705b599484 (127.7.89.129:45101): Couldn't send request to peer fabb72df3c844f928dc062705b599484. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.129:45101: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.550806  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:20.551930  7526 tablet_replica.cc:333] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.553211  7526 raft_consensus.cc:2243] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.553992  7526 raft_consensus.cc:2272] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.556732  7526 tablet_replica.cc:333] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.557447  7526 raft_consensus.cc:2243] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.558408  7526 raft_consensus.cc:2272] T 50538deb9d0641d6811ee0d600ec6c24 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.561233  7526 tablet_replica.cc:333] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.562219  7526 raft_consensus.cc:2243] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.563310  7526 raft_consensus.cc:2272] T 2c3b313ca1784f4194bae419112cb6cb P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.566340  7526 tablet_replica.cc:333] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.567330  7526 raft_consensus.cc:2243] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.568300  7526 raft_consensus.cc:2272] T 27876839904a43d0aa2d0c1c992e5738 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.571084  7526 tablet_replica.cc:333] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.571831  7526 raft_consensus.cc:2243] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.572388  7526 raft_consensus.cc:2272] T c2f5838bfc4240de8f14073d0c85fcab P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.575112  7526 tablet_replica.cc:333] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.575877  7526 raft_consensus.cc:2243] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.576471  7526 raft_consensus.cc:2272] T 5d74c21c7a154be5a0057771d3e63f0c P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.579119  7526 tablet_replica.cc:333] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.579813  7526 raft_consensus.cc:2243] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.580865  7526 raft_consensus.cc:2272] T 16975b7e56d04f8783a7b1250c87e478 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.583416  7526 tablet_replica.cc:333] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.584050  7526 raft_consensus.cc:2243] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.585026  7526 raft_consensus.cc:2272] T 9f44644e1f5f4610b1dab75f07c2993e P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.587575  7526 tablet_replica.cc:333] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.588194  7526 raft_consensus.cc:2243] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.589112  7526 raft_consensus.cc:2272] T ee5967cbef3b48bc87d03588f9b4981d P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.591604  7526 tablet_replica.cc:333] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.592243  7526 raft_consensus.cc:2243] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.592756  7526 raft_consensus.cc:2272] T 81ca28ac68414a7fafb169b5d088b6d6 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.595319  7526 tablet_replica.cc:333] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.595932  7526 raft_consensus.cc:2243] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.596422  7526 raft_consensus.cc:2272] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.598742  7526 tablet_replica.cc:333] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.599406  7526 raft_consensus.cc:2243] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.599965  7526 raft_consensus.cc:2272] T 63ac558c2e934cbdb38fa0c88858f4b7 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.602577  7526 tablet_replica.cc:333] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.603261  7526 raft_consensus.cc:2243] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.604279  7526 raft_consensus.cc:2272] T eea8f92e67394209ae7554aa9f4a0ac6 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.606786  7526 tablet_replica.cc:333] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.608142  7526 raft_consensus.cc:2243] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.608664  7526 raft_consensus.cc:2272] T 769284c8e4f34e83a6ed1163c5ca3a85 P 9d91f828e5fa453982afb0ada2c45170 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.611405  7526 tablet_replica.cc:333] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.612102  7526 raft_consensus.cc:2243] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.612672  7526 raft_consensus.cc:2272] T 6e8109831d6b4ada87ba80fa510b715d P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:20.614073 20753 consensus_peers.cc:597] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 -> Peer 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475): Couldn't send request to peer 9d91f828e5fa453982afb0ada2c45170. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.615322  7526 tablet_replica.cc:333] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.616146  7526 raft_consensus.cc:2243] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.616664  7526 raft_consensus.cc:2272] T cf8d4d44b9b943c5b5858f393fc5c8fb P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.619738  7526 tablet_replica.cc:333] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.620373  7526 raft_consensus.cc:2243] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.620939  7526 raft_consensus.cc:2272] T 98ba4deec7b446128f9c734b613ca13d P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.623379  7526 tablet_replica.cc:333] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170: stopping tablet replica
I20260709 13:08:20.623932  7526 raft_consensus.cc:2243] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.624410  7526 raft_consensus.cc:2272] T 40d9711960b64959a444115884f4a687 P 9d91f828e5fa453982afb0ada2c45170 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.670424  7526 tablet_server.cc:196] TabletServer@127.7.89.130:0 shutdown complete.
W20260709 13:08:20.711081 20753 consensus_peers.cc:597] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 -> Peer 9d91f828e5fa453982afb0ada2c45170 (127.7.89.130:42475): Couldn't send request to peer 9d91f828e5fa453982afb0ada2c45170. Status: Network error: Client connection negotiation failed: client connection to 127.7.89.130:42475: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.717262  7526 tablet_server.cc:179] TabletServer@127.7.89.131:0 shutting down...
I20260709 13:08:20.759738  7526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:20.760500  7526 tablet_replica.cc:333] T 50538deb9d0641d6811ee0d600ec6c24 P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.761291  7526 raft_consensus.cc:2243] T 50538deb9d0641d6811ee0d600ec6c24 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.761857  7526 raft_consensus.cc:2272] T 50538deb9d0641d6811ee0d600ec6c24 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.764549  7526 tablet_replica.cc:333] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.765214  7526 raft_consensus.cc:2243] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.765704  7526 raft_consensus.cc:2272] T 2c3b313ca1784f4194bae419112cb6cb P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.768141  7526 tablet_replica.cc:333] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.768865  7526 raft_consensus.cc:2243] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.769335  7526 raft_consensus.cc:2272] T ee5967cbef3b48bc87d03588f9b4981d P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.771840  7526 tablet_replica.cc:333] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.772564  7526 raft_consensus.cc:2243] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.773120  7526 raft_consensus.cc:2272] T 81ca28ac68414a7fafb169b5d088b6d6 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.775969  7526 tablet_replica.cc:333] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.776752  7526 raft_consensus.cc:2243] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.777343  7526 raft_consensus.cc:2272] T 3c5385cd8d0742d7ab0cdd98e49e3a5f P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.780040  7526 tablet_replica.cc:333] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.781040  7526 raft_consensus.cc:2243] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.782248  7526 raft_consensus.cc:2272] T 58d0ebe3dbfe4a3b9d21682bf14d1333 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.784983  7526 tablet_replica.cc:333] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.785741  7526 raft_consensus.cc:2243] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.786775  7526 raft_consensus.cc:2272] T 63ac558c2e934cbdb38fa0c88858f4b7 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.789302  7526 tablet_replica.cc:333] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.789983  7526 raft_consensus.cc:2243] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.790503  7526 raft_consensus.cc:2272] T eea8f92e67394209ae7554aa9f4a0ac6 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.793090  7526 tablet_replica.cc:333] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.793757  7526 raft_consensus.cc:2243] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.794293  7526 raft_consensus.cc:2272] T 6e8109831d6b4ada87ba80fa510b715d P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.796706  7526 tablet_replica.cc:333] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.797398  7526 raft_consensus.cc:2243] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.797907  7526 raft_consensus.cc:2272] T 769284c8e4f34e83a6ed1163c5ca3a85 P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.800402  7526 tablet_replica.cc:333] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.801103  7526 raft_consensus.cc:2243] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.801651  7526 raft_consensus.cc:2272] T 9f44644e1f5f4610b1dab75f07c2993e P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.804394  7526 tablet_replica.cc:333] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.805157  7526 raft_consensus.cc:2243] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.805759  7526 raft_consensus.cc:2272] T 16975b7e56d04f8783a7b1250c87e478 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.808635  7526 tablet_replica.cc:333] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.809317  7526 raft_consensus.cc:2243] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.810289  7526 raft_consensus.cc:2272] T cf8d4d44b9b943c5b5858f393fc5c8fb P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.813402  7526 tablet_replica.cc:333] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.814128  7526 raft_consensus.cc:2243] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.814759  7526 raft_consensus.cc:2272] T 27876839904a43d0aa2d0c1c992e5738 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.817414  7526 tablet_replica.cc:333] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.818068  7526 raft_consensus.cc:2243] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.818652  7526 raft_consensus.cc:2272] T 5d74c21c7a154be5a0057771d3e63f0c P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.821041  7526 tablet_replica.cc:333] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.821682  7526 raft_consensus.cc:2243] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.822227  7526 raft_consensus.cc:2272] T c2f5838bfc4240de8f14073d0c85fcab P 602a0d7ba5984442865daf5400b33251 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.824692  7526 tablet_replica.cc:333] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.825477  7526 raft_consensus.cc:2243] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.826485  7526 raft_consensus.cc:2272] T 40d9711960b64959a444115884f4a687 P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.828912  7526 tablet_replica.cc:333] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251: stopping tablet replica
I20260709 13:08:20.829495  7526 raft_consensus.cc:2243] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:20.830070  7526 raft_consensus.cc:2272] T 98ba4deec7b446128f9c734b613ca13d P 602a0d7ba5984442865daf5400b33251 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.876343  7526 tablet_server.cc:196] TabletServer@127.7.89.131:0 shutdown complete.
I20260709 13:08:20.919831  7526 master.cc:562] Master@127.7.89.190:40917 shutting down...
I20260709 13:08:20.946497  7526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:20.947294  7526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.947669  7526 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6f0c6ba509b540dd9c2f43ef3431126a: stopping tablet replica
I20260709 13:08:20.972090  7526 master.cc:584] Master@127.7.89.190:40917 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (12900 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (26564 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (438852 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:08:21.036046  7526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 422 messages since previous log ~16 seconds ago
I20260709 13:08:21.036338  7526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 422 messages since previous log ~16 seconds ago
I20260709 13:08:21.036522  7526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 420 messages since previous log ~16 seconds ago
I20260709 13:08:21.036772  7526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~35 seconds ago
I20260709 13:08:21.036959  7526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~35 seconds ago
I20260709 13:08:21.037169  7526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/master/catalog_manager.cc:4755: suppressed but not reported on 2 messages since previous log ~93 seconds ago
I20260709 13:08:21.037469  7526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 7 messages since previous log ~214 seconds ago
I20260709 13:08:21.037761  7526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 14 messages since previous log ~0 seconds ago
I20260709 13:08:21.038103  7526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 49 messages since previous log ~11 seconds ago
